Skip to content

refactor(model): move api-based embeddings/reranking calls out of model server #5216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Aug 20, 2025
Merged

refactor(model): move api-based embeddings/reranking calls out of model server #5216

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ebd91a9
move api-based embeddings/reranking calls to api server out of model …
Aug 19, 2025
05420eb
ran pre-commit
Aug 19, 2025
347f8cc
fix mypy errors
Aug 19, 2025
84f9da5
mypy and precommit
Aug 19, 2025
fea3111
move utils to right place and add requirements
Aug 19, 2025
e55cf42
precommit check
Aug 19, 2025
11deb5c
removed extra constants, changed error msg
Aug 19, 2025
787f0a7
Update backend/onyx/utils/search_nlp_models_utils.py
jessicasingh7 Aug 19, 2025
8244324
greptile
Aug 19, 2025
c4d6577
addressed comments
Aug 20, 2025
7d2a53a
added code enforcement to throw error
Aug 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions backend/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
__pycache__/
.mypy_cache
.idea/
venv/
site_crawls/
.ipynb_checkpoints/
api_keys.py
Expand Down
29 changes: 0 additions & 29 deletions backend/model_server/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,34 +1,5 @@
from shared_configs.enums import EmbeddingProvider
from shared_configs.enums import EmbedTextType


MODEL_WARM_UP_STRING = "hi " * 512
INFORMATION_CONTENT_MODEL_WARM_UP_STRING = "hi " * 16
DEFAULT_OPENAI_MODEL = "text-embedding-3-small"
DEFAULT_COHERE_MODEL = "embed-english-light-v3.0"
DEFAULT_VOYAGE_MODEL = "voyage-large-2-instruct"
DEFAULT_VERTEX_MODEL = "text-embedding-005"


class EmbeddingModelTextType:
PROVIDER_TEXT_TYPE_MAP = {
EmbeddingProvider.COHERE: {
EmbedTextType.QUERY: "search_query",
EmbedTextType.PASSAGE: "search_document",
},
EmbeddingProvider.VOYAGE: {
EmbedTextType.QUERY: "query",
EmbedTextType.PASSAGE: "document",
},
EmbeddingProvider.GOOGLE: {
EmbedTextType.QUERY: "RETRIEVAL_QUERY",
EmbedTextType.PASSAGE: "RETRIEVAL_DOCUMENT",
},
}

@staticmethod
def get_type(provider: EmbeddingProvider, text_type: EmbedTextType) -> str:
return EmbeddingModelTextType.PROVIDER_TEXT_TYPE_MAP[provider][text_type]


class GPUStatus:
Expand Down
Loading