OpenRouter

An API gateway lets an application reach several model providers through a single interface. Application code knows one protocol, so switching provider becomes a configuration change rather than a code change.

KobiGPT uses this approach for embedding generation. The multilingual embedding model is called through the gateway and the resulting vectors are written to Qdrant collections. On the chat side, the default model is from the Gemini family.

That separation matters: embedding and chat are different layers, use different models, and carry different cost profiles. The gateway unifies the access path, not the layers themselves.

Connecting directly to a single provider is the simplest setup, but it leaves no room to manoeuvre when prices change, a version is retired, or access is interrupted. A gateway layer reduces that risk because switching to an alternative provider does not touch application code.

In exchange it adds a dependency of its own: the gateway. When the gateway is unreachable, every model call is affected. For that reason, defining a fallback path straight to the provider is a common practice in critical deployments.

On the KobiGPT side, an on-premise scenario can run embedding with the same model inside your own infrastructure. That option removes the gateway entirely for organisations with data-residency requirements.

Three things should be measured when choosing a gateway: latency, catalogue breadth, and pricing transparency. Latency matters most on the embedding side, because a bulk document upload issues hundreds of calls back to back.

The second criterion is whether the model you use is genuinely available on the gateway. A model listed in a catalogue may not be the same version or the same dimensionality; since embedding dimensionality is a system-wide invariant, that detail is critical.

The third is failure behaviour. When the gateway returns a transient error, the queue must retry and document status must not hang. This behaviour should be tested during a bulk upload.