LLM freedom: Breaking free from vendor lock-in with AI gateways

During our recent Friday Talk, our software engineer, Nemanja Marić, presented “LLM freedom,” exploring practical methods to avoid being locked into a single Large Language Model (LLM) provider. He demonstrated how engineering teams can use AI gateways and flexible coding agents to maintain architectural independence, reduce administrative overhead, and optimize token costs.

The problem of vendor lock-in

Many teams find themselves restricted by subscription-based models tied exclusively to a single provider, such as OpenAI or Anthropic. Relying on a single ecosystem limits flexibility. Furthermore, attempting to integrate new or alternative LLMs traditionally requires maintaining multiple complex subscriptions and managing a tangled web of API keys, which significantly increases administrative complexity and security risks.

The AI gateway solution

To solve this, Nemanja introduced the concept of an AI gateway, specifically highlighting Requesty. An AI gateway serves as a unified interface that routes requests to various LLMs without requiring changes to the underlying code.

Using Requesty provides access to over 600 models through a single API key. Developers can switch models instantly via a simple command without altering their coding agent. Beyond flexibility, this centralized gateway gives organizations robust administrative controls, including:

• Data residency & compliance: Defining approved models and enforcing data residency requirements, such as restricting routing exclusively to EU servers.
• Cost control: Setting guardrails and spending budgets to encourage the use of more cost-effective models for routine tasks.
• Infrastructure reliability: Performing automatic load balancing across servers to ensure high availability.

Flexible coding agents: OpenCode vs. Pi

Nemanja also showcased how to pair this gateway architecture with flexible coding agents, comparing two specific tools he utilizes for his projects:

• OpenCode: An open-source AI coding agent available in both terminal and desktop formats. It offers a “batteries-included” experience featuring built-in “plan” and “build” modes. Because of its comprehensive nature, it uses a heavier system prompt of approximately 7,000 to 14,000 tokens.
• Pi Coding Agent: A minimalist alternative focused purely on core functionalities (read, write, edit, and bash). It requires extensions to add tools like “plan” and “build” modes, making it highly token-efficient with a system prompt of only about 1,000 tokens.

Usage insights and cost optimization

One of the most powerful workflows Nemanja demonstrated was mid-session model switching. Both OpenCode and Pi allow users to swap the active model on the fly. This is highly effective if an LLM begins to hallucinate or if a sudden shift in task complexity requires a different model’s reasoning capabilities.

By utilizing an AI gateway, teams can shift their cost structure from expensive, flat-rate subscriptions to efficient, API-based usage pricing. This allows developers to route complex reasoning tasks to premium models while seamlessly routing routine tasks to cheaper, high-quality open-source models like Mistral or DeepSeek—drastically optimizing overall project costs.

If you are looking to build scalable, independent, and secure AI architectures that put you in control of your models and data, discover how our team can help you implement these solutions and accelerate your digital journey.