-
Notifications
You must be signed in to change notification settings - Fork 6
Glossary
A secret issued by LangRoute to authenticate client requests to the gateway. Sent as Authorization: Bearer <access_key>. Stored securely in the database and treated as a secret. Can be revoked, rotated, and expired.
Previously: API Key (ApiKey → AccessKey)
Non-sensitive display form of an Access Key (e.g., lr_...ABCD) that helps users distinguish keys without revealing the full secret. Typically “prefix + last 4”.
Replacing an existing Access Key with a new secret while keeping logical continuity (same project/user). Often implemented as “create new, then revoke old after a grace window.”
Invalidating an Access Key immediately. Requests using a revoked key must be rejected with 401. Previously: sometimes “delete key”; now we prefer revoke (soft delete).
An optional timestamp after which the key is unusable. Encouraged for good hygiene and least-privilege practices. Previously: “expiresAt” existed; semantics unchanged.
A server-side vendor credential used by LangRoute to call upstream LLM providers (e.g., OpenAI, Google, Anthropic). Lives in environment variables only; never exposed to clients. Previously: “provider API key”, “vendor key”.
An upstream LLM vendor (e.g., openai, google, anthropic).
Previously: same, but sometimes conflated with “adapter.”
String identifier for a provider (e.g., 'openai' | 'google' | 'anthropic' | ...'). Defined in the model/provider registry.
A module that translates LangRoute’s normalized Chat Completions request into a provider’s API call, then normalizes the response back to the OpenAI-compatible shape. Previously: “provider client,” “integration.”
Factory/selector that returns the appropriate Adapter given a Provider ID (and optionally a Model ID). Example: getAdapter(providerId, modelId?).
Previously: “adapters/index.ts” (avoid using index.ts as an implementation file).
The internal, provider-agnostic request/response format for Chat Completions (aligned to OpenAI’s schema). Adapters convert between this shape and the provider’s native API.
Single source of truth for available models and capabilities (provider, context window, max tokens, streaming/vision/functions support, pricing metadata). Implemented in lib/config/llmConfig.ts.
The registry entry for a specific model (e.g., id, provider, maxTokens, contextWindow, supportsStreaming, supportsFunctions, supportsVision, pricing).
A typed list derived from the registry; used for validation and routing.
Allowed message roles in chat requests (currently system, user, assistant). Centralized in the registry and reused by Zod schemas and services.
Provider-agnostic bounds for parameters such as temperature, top_p, and max_tokens. Used for request shape validation; model-specific constraints are enforced in services.
The canonical text-generation operation compatible with OpenAI’s /v1/chat/completions. Accepts a list of role-tagged messages and returns a completion from the assistant.
Previously: often called just “chat.” We now use Chat Completions or just Completions.
POST /v1/chat/completions — the OpenAI-compatible path exposed for maximum SDK/tool compatibility.
Previously: /api/chat (Kept as an internal alias).
POST /api/chat — the internal application route that maps to the same Chat Completions operation.
Structured identity resolved by middleware from the Access Key (and session, if applicable), attached to each request, e.g. { userId, accessKeyId }. Used for attribution, usage metering, and policy enforcement.
Canonical exception type for business-rule failures (e.g., invalid model, exceeded limits). Routes catch and convert into standardized error envelopes.
Standard JSON error shape:
{ error: { message: string, code: string }, requestId: string, ts: string }, produced by errorService.
Input shape validation at the route boundary (e.g., z.string().uuid(), z.string().email()). Business rules belong in services.
Returning partial tokens progressively when supported by the chosen model/adapter. Requires supportsStreaming in the model config and a streaming-capable adapter.
Provider capability that allows the model to emit structured tool calls during a chat completion. Modeled via capability flags (e.g., supportsFunctions) and handled in adapters/services.
Image input support for chat completions, indicated via supportsVision in the model config.
The total token budget (input + output) a model can consider.
Upper bound for generated tokens set per model and enforced by services (modelCfg.maxTokens and related logic).
A record of prompt/completion tokens (and cost) attributed to { userId, accessKeyId }. Foundation for budgets, analytics, and rate-limit decisions.
Policy that restricts request frequency or token volume per { userId, accessKeyId }. Commonly implemented via token buckets (e.g., Redis).
Configurable ceilings per user or Access Key (e.g., daily/monthly token or dollar limits). Enforced in services using Usage Events and Request Context.
Structured logs and metrics for requests/responses and adapter activity, optionally streamed in real time (e.g., via pub/sub) to dashboards or websocket clients.
Admin UI where vendor credentials (OpenAI/Google/Anthropic) are configured. Credentials remain server-side and are never sent to clients. Previously: “Integrations,” “Providers,” or “API keys.”
Admin UI where Access Keys are created, listed (with Preview), revoked, rotated, and optionally assigned expiries. Typical fields: Preview, Created, Expires, Status. Previously: “API Keys”.
NextAuth-based session for the admin dashboard (web UI). Independent from Access Key authentication used for API requests.
Scopes to group users and Access Keys. Enables RBAC, usage aggregation, and budgets at a group level.
Authorization model across dashboard and API (e.g., ADMIN, USER), typically enforced per Team/Workspace and surfaced in UI.
Temporary token used for credential resets (e.g., /auth/password/forgot, /auth/password/reset).
Primary key strategy across tables for consistency and interoperability.
Strongly-typed loading of environment variables (e.g., with Zod) for Provider Credentials and application secrets.
Practices to ensure Provider Credentials and other secrets remain server-only (env vars or secret managers), never exposed to the client.
Backend service that enforces model/business rules and delegates execution to the appropriate Adapter for execution, producing OpenAI-compatible responses. Previously: “ChatService”; we now prefer “CompletionsService” or “ChatCompletionsService”.
TypeScript interface(s) that every provider adapter implements, defining inputs/outputs and hooks (e.g., for streaming, tool calls).