Models and providers

Different providers behind one OpenAI-compatible interface, including OpenAI and Anthropic: model selection, listing accessible models, retries, and failover.

The Gateway offers a single OpenAI-compatible interface. Which provider runs behind it is not the client’s concern: your application sends the same request shape, and the only field that changes is model.

Model selection

The model name is given in the request. Switching between the models defined for your organization takes one line; the SDK, the parameters, and the response format stay the same.

resp = client.responses.create(
    model="gpt-4o-mini",
    input="Ahmet Yılmaz, TCKN 10000000382, adres değişikliği istiyor.",
)

You get the models you can access from the /v1/models endpoint. The list returns the models defined for your organization; the name gurubase-siper, reserved for masking only, is also there. For the full reference of the endpoint, see List accessible models.

Embedding models

Embedding models turn text into a numeric vector, which you use for search, similarity, and recommendations. The endpoint is /v1/embeddings and it is OpenAI-compatible: only base_url and the key change in your SDK.

resp = client.embeddings.create(
    model="text-embedding-3-small",
    input="Ahmet Yılmaz, TCKN 10000000382, talebini iletti.",
)
vector = resp.data[0].embedding

Masking is on here too and is governed by the same key as chat: personal fields in the text become placeholders before the vector is computed. The vector you get back is therefore the vector of the masked text.

Two consequences follow, and both are worth knowing upfront.

The same document always gives the same vector. Placeholders on this endpoint are deterministic, so re-embedding a document gives you the same vector and the value in your index stays consistent. This does not by itself prevent the same record from being INSERTED twice; deduplication needs a stable per-document id with an upsert (or an explicit dedup step). What determinism gives you is that the vector behind that id does not drift over time.

Masked entities cannot be told apart. Two documents that differ only in a person’s name collapse to the same placeholder and therefore produce the SAME vector (measured: similarity 1.000). A query naming a person finds “documents that mention a person” but cannot tell which one; it scores both equally. This is silent: search does not fail, it just returns the wrong person’s document just as readily. If you need to filter by person, do it with an id field on your side, not with vector similarity.

Topic search is unaffected: content that is not masked (request type, subject, date) stays in the vector as it is.

Indexing and querying must also go through the same path: if one goes through the gateway and the other directly to the provider, the vectors are not comparable.

The input must be text. Pre-tokenized input (an array of token ids) cannot be masked, so it never reaches the provider and comes back as 415. For the full reference of the endpoint, see Create an embedding vector.

Providers

Two kinds of providers can run behind the same interface: general providers such as OpenAI, Anthropic, and Gemini, or models you host on your own servers. Which providers are open, and which models each virtual key can reach, are defined for the organization.

The provider’s real API key stays inside the Gateway. Your applications and panel users see only the virtual key; the provider key never reaches the client. For key generation, rotation, and revocation steps, see Panel.

Masking is independent of this choice: whichever provider is selected, the request passes through the same protection layers before it goes to the external model. For the layers, see Guardrails.

Retries and switching to an alternative

Providers return a rate limit from time to time, or slow down. In that case the retry decision is made in the Gateway, not in your application code: the request is retried with increasing wait intervals.

If more than one provider is defined for your organization, routing the request to an alternative provider when one cannot respond is configured in the same place. No extra code is needed on the application side.

The limits on your side (requests per minute, token budget, and 429 behavior) are a separate topic; the details are on the Limits and quotas page.