The model choice is a trust choice

After an index is kept current, the embedding model determines what kind of similarity zvec-grep can compute and where the data must travel. The useful decision is not “which model has the largest number?” but which combination of language coverage, input length, content type, hardware, compatibility, privacy, and operating cost fits the workspace; that judgment also determines whether an existing index can be reused or must be rebuilt.

This matters because embeddings sit on both sides of the evidence loop. During indexing, the model converts extracted fragments into vectors. During semantic search, it converts the query into the same vector space so related content can be compared. If the model cannot accept a kind of content, cannot see enough of a long fragment, or is replaced by an incompatible model later, the quality and meaning of indexed evidence change. A model setting is therefore part of the index’s contract, not a decoration added after the search design is finished.

Start with workload, not model name

The first criterion is the material being searched. A code repository benefits from a model that can represent symbols, signatures, and technical language. General documents may need stronger coverage for ordinary prose or multiple languages. Long sections need a larger input limit or a narrower extraction scope. Images require explicit inclusion and a model that accepts image content. These needs overlap, but no single catalog entry is presented as a universal winner.

Zvec-grep’s catalog includes several local model families and remote Qwen models. The local choices use different runtimes: compact static Model2Vec models, quantized Transformer models, and GGUF models are all available. The practical consequence is that “local” does not describe one performance profile. A small code-oriented model may make a fast first index on a repository. A retrieval-tuned or multilingual model may better match a document collection. A longer-context model may reduce truncation for large entities, while requiring more memory or slower local inference.

The relevant properties are concrete. Language coverage determines whether a query and a stored passage can meet in the model’s learned space. The maximum input-token limit determines how much of each extracted entity or fragment can influence its vector. The dimension determines the shape of the stored vector, but a larger dimension by itself does not establish better retrieval. Input kinds determine whether text, images, or both may be embedded. Batch size and device support affect throughput and resource use. These properties should be evaluated against representative workspace questions rather than read as an abstract ranking of models.

Consider three deployments. A private TypeScript repository with mostly short symbols and functions may start with a small local code-oriented model. A multilingual knowledge base may need a multilingual model even if its vectors are less compact. A workspace containing long technical manuals may need a model with a larger input limit, or a deliberate scope and chunking strategy that keeps each fragment within a usable boundary. For a collection that includes images, a text-only model cannot add image fragments to the vector index; selecting a multimodal option changes both capability and data-handling considerations.

Local models have a distinct operational path. Their inference keeps workspace content and query text on the machine, while model files are downloaded and cached for later use. Zvec-grep pins the model artifacts it uses. If a pinned Hugging Face artifact cannot be downloaded, the project documents a fallback to a pinned, integrity-checked ModelScope copy, with a separate cache location.