Kimi K3 API: Where to Run It, and What a 1M-Token Context Costs

The 1M-Token Context Shift in Production Inference

Engineering teams building production LLM applications spend substantial engineering cycles maintaining Retrieval-Augmented Generation (RAG) pipelines. Chunking source code, synchronizing vector databases, and tuning embedding retrieval thresholds frequently fail when queries require full repository context or multi-document cross-referencing. When a model context window expands to 1M tokens, developers can ingest entire codebases or complete technical documentation sets into a single inference pass, bypassing vector index fragmentation completely.

Deploying 1M-token context models introduces specific operational trade-offs in inference execution. Moonshot's Kimi K3 provides a 1M-token context window for text and chat workloads hosted in the EU eu-north1 region. Processing full-context prompts avoids missing context boundaries, but raw input processing costs scale linearly with prompt length. At $3.00 per million input tokens and $15.00 per million output tokens, unoptimized long-context requests quickly dominate compute expenditure for AI-native platforms.

• Pipeline complexity: Eliminates vector database maintenance, embedding re-indexing, and chunk overlap tuning.
• Time-to-first-token latency: Prefill phase execution time increases with sequence length, requiring high-throughput attention kernels.
• Cost per call: Ingesting a large codebase scales directly on base input rates, significantly exceeding traditional vector retrieval costs.

To make 1M-token context economical for agentic workflows and multi-turn developer tools, inference engines rely on in-memory prompt caching. On Lyceum Serverless Inference, cached input tokens drop from $3.00 per million to $0.75 per million tokens. Prompts are held in GPU VRAM across sequential requests, allowing production systems to reuse static codebase context without re-paying the full prefill cost on every turn, while maintaining zero data retention guarantees across European data centres.

Kimi K3 Technical Profile: 2.8T MoE and 1M Context Window

Moonshot AI's Kimi K3 architecture uses a sparse Mixture-of-Experts (MoE) design containing 2.8 trillion total parameters. The published Kimi K3 model card, read on 3 August 2026, lists 2.8T total parameters, 104B activated per token, 16 of 896 experts routed per token, and a 1,048,576-token context window. During inference execution, the router directs incoming tokens to specific expert tracks, activating those 104 billion parameters. This gating mechanism provides the expressiveness of a multi-trillion-parameter base model while bounding single-step compute demands to roughly 104B active parameters. For AI-native product teams, this balance enables deep reasoning across multi-step tasks without incurring the severe inference latency of dense multi-trillion-parameter architectures.

Architectural Specifications and Context Limits

• Total parameter count: 2.8 trillion total open weights across sparse MoE experts.
• Active parameters: 104 billion parameters activated per token forward pass.
• Context window limit: 1,048,576 tokens (1M context) for long-sequence tasks.
• Regional hosting: Served from eu-north1 in European data centres with GDPR compliance.
• Token pricing model: $3.00 per 1M input tokens, $15.00 per 1M output tokens, and $0.75 per 1M cached input tokens.

Processing full 1M-token context windows presents substantial memory management challenges, particularly around Key-Value (KV) cache allocation across long conversation threads or codebase analysis. On Lyceum Serverless Inference, we serve Kimi K3 from European data centres in Spain, Paris and the Nordics, running NVIDIA GPUs with vLLM and NVIDIA Dynamo. To optimize token delivery costs for repetitive system prompts and long context prefixes, prompt caching retains prefix states in GPU memory for active sessions. This reduces re-evaluation costs to $0.75 per 1M cached tokens, maintaining low latency while adhering strictly to zero persistent data retention standards.

Running a 1M-token MoE model requires strict isolation between execution nodes and memory allocation pools. Rather than relying on black-box routing layers, our platform exposes vLLM state management directly to maintain predictable throughput under concurrent requests. For teams deploying production agent frameworks or deep document parsing pipelines, this architecture delivers full context capacity under sovereign EU data residency commitments, ensuring sensitive prompt payloads never leave European borders.

Kimi K3 Rate Card: $3.00 Input and $15.00 Output Breakdown

At $3.00 per million input tokens and $15.00 per million output tokens, Kimi K3 represents the top tier of open-weights pricing on Lyceum Serverless Inference. For AI-native product companies running long-horizon autonomous agents, multi-document analysis, or dense codebase processing across a 1M-token context window, calculating unit economics requires evaluating both baseline token rates and dynamic caching performance.

Although Kimi K3 is the highest per-token entry in our open model catalogue, its rates sit below the closed-source frontier APIs. OpenAI's own API pricing page lists GPT-5.5 at $5.00 input and $30.00 output per million tokens (USD, standard tier, read 3 August 2026), and Anthropic's models overview lists Claude Fable 5 at $10.00 and $50.00 and Claude Opus 5 at $5.00 and $25.00 per million tokens (USD, read 3 August 2026). Furthermore, in-memory prompt caching on our infrastructure reduces cached input processing to $0.75 per million tokens. For application architectures that maintain persistent system prompts or recurring conversation histories, this caching mechanism substantially lowers effective input expense.

Accessing Kimi K3 via Serverless Inference in European data centres (eu-north1) guarantees GDPR compliance and data residency without requiring dedicated GPU node allocations or long-term commitments. With no API model string published yet, engineering teams can request early integration access to test 1M-context workloads against open OpenAI-compatible endpoints.

Prompt Caching Economics: Lowering Effective Input Costs to $0.75

At $3.00 per million input tokens, filling Kimi K3's full 1M-token context window for every request scales API spend rapidly. AI-native product teams building agentic loops or processing static document corpuses rarely send entirely fresh context on every turn. In our Serverless Inference platform, we implement in-memory GPU prompt caching that cuts the cost of re-evaluating known token prefixes down to $0.75 per million cached tokens, lowering effective input token compute costs.

KV Cache Reuse in Multi-Turn Workloads

The underlying mechanism leverages automatic prefix caching within our open inference stack built on vLLM. vLLM's own documentation describes it as caching the KV cache of existing queries so that a new query can reuse it when it shares the same prefix, and notes that this shortens the prefill phase rather than the decode phase (read 3 August 2026). When an incoming request shares a common system prompt, agent tool definitions, or historical conversation blocks, the engine skips redundant matrix multiplications by fetching the precomputed Key-Value (KV) tensors directly from GPU memory. Prompts reside strictly in GPU VRAM during active sessions and are discarded without disk persistence, preserving our zero-data-retention policy while serving low-latency prefill responses.

• 75% input cost reduction: Prompt caching drops input token costs from $3.00 to $0.75 per million tokens when prompt prefixes match active GPU VRAM KV cache blocks.
• Latency and throughput benefits: Reusing cached prefill state eliminates redundant prefix processing across multi-turn agent execution and long-document queries.
• Zero manual configuration: Serverless Inference handles KV cache retention and prefix matching automatically without requiring dedicated cache management code.

In multi-turn agent workflows, evaluating a 1M-token context from cold memory costs $3.00 per million input tokens on every turn. In-memory prompt caching applies the full $3.00 rate only to the initial prefill; subsequent turns evaluate cached prefix tokens at $0.75 per million, cutting ongoing input spend by 75%.

Comparing 1M-Context Models in the EU: Kimi K3 vs GLM-5.2 and MiniMax-M3

When building production agents or repository-scale analysis workflows, selecting a million-token model requires balancing raw capability against token expenditure. In the Lyceum model catalogue, several 1M-context models run in the EU (eu-north1), among them Kimi K3, GLM-5.2, and MiniMax-M3. Kimi K3 commands the highest output rate in the catalogue at $15.00 per million tokens, reflecting its parameter scale and deep reasoning performance. For engineering teams evaluating high-frequency inference, understanding when to deploy Kimi K3 versus lighter 1M-context alternatives directly dictates unit economics. The rest of the Kimi lineup is compared in our guide to where to run Kimi models in Europe.

Workload Profiles and Cost Optimization

The substantial output price variance between Kimi K3 ($15.00/1M) and GLM-5.2 ($4.50/1M) shapes workload placement across your stack. For multi-step agent planning, code refactoring, and complex tool routing where precision outweighs output volume, Kimi K3 justifies its higher rate. Conversely, for document extraction or long-form output pipelines, MiniMax-M3 ($2.00/1M output) and GLM-5.2 deliver substantially lower operational cost.

On Lyceum Serverless Inference, all three options process requests with zero data retention, guaranteeing GDPR compliance in European infrastructure. Automated GPU prompt caching reduces Kimi K3 input costs to $0.75 per million tokens, enabling teams to persist large system prompts and static context without compounding API bills.

Data Sovereignty and GDPR Compliance for Long-Context Workloads

Feeding a 1M-token context window into an LLM involves transferring up to 4 MB of uncompressed text per call, including full source code repositories and sensitive financial records. Routing these context payloads to non-EU endpoints introduces cross-border data transfer risks and exposure to foreign access frameworks like the US CLOUD Act. For European product engineering teams, maintaining strict data residency within EU-hosted infrastructure is an essential architectural requirement for production deployments.

At Lyceum, we serve Kimi K3 directly from our eu-north1 data centre region with full GDPR compliance by default. Your prompt payloads and model outputs are processed entirely within European infrastructure, ensuring that sensitive IP never leaves the region. To support high-throughput product teams, our Serverless Inference engine enforces a strict zero data retention architecture where prompts are never logged, stored on disk, or used for model training.

• EU Data Isolation: Processing runs in isolated eu-north1 hardware, keeping all context data subject to European data privacy laws.
• Zero Persistent Storage: Prompts and generated completions exist in volatile memory only during inference execution and are immediately discarded.
• Transient In-Memory Caching: Prompt caching holds KV cache states in GPU VRAM per session for a few minutes at most, never writing state to a persistent database.
• Per-Token Transparency: Clear billing at $3.00 per 1M input tokens and $15.00 per 1M output tokens, with cached input charged at $0.75 per 1M tokens.

By isolating full-context inference within the EU, technical leads can deploy agentic workflows and deep document analytics without compromising user privacy or violating enterprise client compliance guarantees.

Integrating Kimi K3 via OpenAI-Compatible Serverless Inference

Integrating Kimi K3 into production application flows requires zero changes to existing OpenAI SDK infrastructure. Because Lyceum serves models via an OpenAI-compatible HTTP interface, switching an existing agent pipeline to Kimi K3 involves updating the base URL to our Serverless Inference endpoint and supplying an authorization key. The platform handles connection pooling, request queueing, and GPU cluster execution under the hood while maintaining strict zero-retention compliance in European data centres. Note that because Kimi K3 is a newly integrated model tier, the public API model string is available on request during initial deployment.

Streaming Latency and SSE Protocol Specifications

For real-time applications such as interactive coding agents or document analysis workflows, streaming output over Server-Sent Events (SSE) ensures rapid response times even across deep token chains. Our open inference stack (powered by vLLM, NVIDIA Dynamo, and TensorRT-LLM) emits each token to the client as it is produced, so the application can render output while generation is still running rather than waiting for the whole completion. Prompts are kept in GPU memory during session execution and purged immediately after completion, avoiding persistent storage overhead while accelerating context reuse.

• Base URL endpoint: Update your SDK client configuration to point directly to our serverless endpoint.
• Authentication: Pass your standard API key in the authorization header.
• Model identifier: Contact engineering for the active public API model string assigned to your deployment tenant.
• Response streaming: Set stream=True to receive continuous SSE delta chunks as tokens are generated.

By keeping the inference stack open and standard-compliant, developers avoid vendor lock-in to proprietary wrappers. You retain complete control over prompt formatting, temperature scaling, and context management while benefiting from sovereign execution across European infrastructure.