Skip to content

pgml.embed trust_remote_code #1169

@kczimm

Description

@kczimm

A user on Discord submitted the following query with subsequent error:

SELECT pgml.embed(
    transformer => 'jinaai/jina-embeddings-v2-base-en',
    text => 'Dynamical Scalar Degree of Freedom in Horava-Lifshitz Gravity"}',
    kwargs => '{"trust_remote_code": true}'
);
ERROR:  ValueError: Loading this model requires you to execute custom code contained in the model repository on your localmachine. Please set the option trust_remote_code=True to permit loading of this model.

We are currently using sentence_transformers and InstructorEmbedding Python packages for the pgml.embed function. This model gets dispatched to the sentence_transformers package which does not appear to support the trust_remote_code keyword.

Huggingface shows, for this model, to use transformers.AutoModel. We should probably add this in, but how do we determine which model should use which package? Can they all use transformers? These questions need answering.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions