30B vs 70B vs 235B: How to Pick Open Model Size Per Task

AI This article was created with the help of AI.

Why Parameter Count Is the Wrong Number to Shop On

When evaluating open-weight models, engineering teams often rely on total parameter count as a direct proxy for inferencing capacity, quality, and cost. In the era of purely dense neural networks, that rule of thumb held true: a 70B model required roughly ten times the arithmetic floating-point operations (FLOPs) of a 7B model for every generated token, demanding linearly scaled compute resources and driving up server spend. Today, that direct link between total weight size and execution cost is broken.

The architectural divergence between dense networks and Mixture-of-Experts (MoE) designs means that total parameter count describes memory storage footprint, not the computational work performed during a forward pass. As Epoch AI puts it, an MoE is more efficient at inference than a dense model of the same total parameter count, but less efficient than a dense model with the same active parameter count. In an MoE architecture, only a small subset of the total parameters activates for any individual token. A model with hundreds of billions of total parameters can execute fewer arithmetic operations per token than a medium-sized dense network, fundamentally changing the unit economics of open-source LLMs.

• Dense architecture: Every single parameter across every layer participates in the matrix multiplication for every token, meaning compute FLOPs scale directly with total model size.
• Sparse Mixture-of-Experts: Gating routers direct token embeddings to a fixed number of top-k sub-networks (experts), so active compute remains decoupled from total parameter storage.
• Serving economics: Memory bandwidth and compute saturation govern inference pricing, meaning sparse models provide access to vast parametric capacity at a fraction of the per-token forward compute cost.

Defaulting to the largest available parameter count under the assumption that bigger always yields better results leads to significant infrastructure overprovisioning. When tasks do not exercise the broader latent knowledge stored across sparse expert layers, you pay for model weights that never execute during inference.

What 30B, 70B and 235B Actually Cost per Million Tokens

To ground the architectural difference in concrete financial metrics, we can look directly at the production pricing across three representative model scales in the Serverless Inference catalogue hosted in the EU (eu-north1). Rather than guessing theoretical GPU utilization or cluster overhead, per-token pricing establishes the exact boundary conditions for each tier.

The price ladder exposes a stark reality: Qwen3-235B-A22B costs exactly $0.20 per million input tokens and $0.60 per million output tokens, which is only double the price of the 30B model ($0.10 in / $0.30 out), despite holding nearly eight times the total parameter volume. Similarly, dense 70B models such as Llama-3.3-70B and Hermes-4-70B sit squarely between the two MoE tiers at $0.13 input and $0.40 output per million tokens.

Because Serverless Inference operates on a fully managed, pay-per-token model without provisioning commitments, it carries no formal SLA, availability tier, or service credit agreement. Infrastructure status and operational uptime metrics are tracked publicly on the status page.

Dense vs. Mixture-of-Experts: Why a 235B Model Does Not Cost Proportionally More

The non-linear pricing between 30B, 70B, and 235B models is a direct reflection of underlying hardware execution dynamics. In a dense architecture like Llama-3.3-70B, every single one of the 70 billion parameters is loaded and multiplied for every generated token. In contrast, Qwen3-30B-A3B and Qwen3-235B-A22B use fine-grained routing: both MoE models hold 128 total experts and activate 8 of them per token.

When processing a forward pass on Qwen3-30B-A3B, the inference engine executes only 3 billion active parameters per token. For Qwen3-235B-A22B, the engine executes 22 billion active parameters per token out of 235 billion in total. From a pure FLOP perspective, generating a token on the 235B MoE requires fewer floating-point operations than generating a token on the dense 70B model (22B active vs 70B active), despite the 235B model occupying significantly larger VRAM requirements across the GPU cluster.

1. Active FLOP footprint: During decoding, the mathematical compute load is governed by active parameters, keeping per-token execution times for a 22B active MoE competitive with or faster than a dense 70B network.
2. High-throughput batching: Serving frameworks such as vLLM use PagedAttention to hold KV blocks in non-contiguous physical memory, which removes fragmentation and lets the scheduler batch more sequences at once; the original paper reports a 2-4x throughput improvement at the same latency versus prior serving systems.
3. Memory bandwidth scaling: While the entire 235B model must reside in GPU memory, the memory access patterns and shallower layer depths prevent linear cost inflation, so the published output price sits close to the 30B MoE tier rather than scaling with total parameter count.

Which Task Shapes Genuinely Need the Larger Model

Because no synthetic benchmark claim or leaderboard rank tells you how a model behaves on your data, engineering teams must evaluate model capacity against their specific production workloads. A model with 235 billion total parameters offers a massive latent capacity that becomes necessary only when a task demands cross-domain synthesis, deep conceptual retrieval, or multi-step deductive reasoning.

The 235B MoE architecture excels in scenarios where the model must navigate conflicting constraints or draw upon obscure, specialized domain knowledge without task-specific fine-tuning. Its router selects 8 of 128 experts for each token, so a much larger pool of specialised weights is available across diverse domains than the per-token compute budget would suggest.

• Multi-step analytical reasoning: Complex workflows requiring intermediate chain-of-thought deductions, validation of mathematical proofs, or multi-variable logic trees.
• Broad multi-domain synthesis: Applications that ingest heterogeneous technical documents (such as legal statutes, medical literature, and financial compliance standards) and require unified analysis.
• Unconstrained conversational agents: General-purpose interactive assistants that must handle arbitrary, open-ended user queries across unpredictable technical and creative domains without breaking persona or policy guidelines.
• High-ambiguity instruction following: Prompts with nested constraints, negative conditions, and edge-case exceptions where smaller models tend to drop secondary rules.

If your workload fits these profiles, routing requests to Qwen3-235B-A22B is economically justifiable: paying $0.20 / $0.60 per million tokens prevents the cascading downstream errors and retry loops that occur when a smaller model fails to follow complex instructions.

Which Task Shapes Are Being Overpaid For

A substantial portion of enterprise LLM workloads consists of deterministic data transformation rather than creative reasoning. In these narrow task shapes, routing traffic to a 235B model represents pure financial waste. If a prompt provides complete context and requires the model to adhere to a rigid output format, the excess latent capacity of a massive parameter pool provides zero measurable utility.

For structured data extraction, document classification, or deterministic JSON transformations, smaller models like Qwen3-30B-A3B ($0.10 in / $0.30 out) or dense 70B models ($0.13 in / $0.40 out) achieve identical validation pass rates at half the compute cost. Finding the cheapest open model that meets your production threshold is the primary operational lever for controlling AI unit economics.

When evaluating a workload, check whether the task relies on external context injected directly into the prompt (such as retrieved database records or API payloads). When context is explicitly provided, the model does not need to recall facts from its pre-trained parametric memory, allowing a 30B MoE model to execute the task with high precision.

How to Size a Model Against Your Own Task in an Afternoon

Selecting the optimal model size should be an empirical, data-driven engineering process rather than an exercise in reading third-party leaderboard scores. You can construct an internal evaluation harness and benchmark your specific prompts across 30B, 70B, and 235B models in a single afternoon.

Because the Serverless Inference endpoint is OpenAI SDK compatible, you can switch between model sizes by modifying only the model string against our standard base URL (https://api.lyceum.technology/api/v2/external/serverless). This allows you to run identical test fixtures across multiple model tiers without altering client code or orchestration pipelines.

1. Sample representative inputs: Extract 100 to 200 real production prompts covering typical edge cases, schema constraints, and typical input token lengths.
2. Define programmatic validation: Implement automated checks for schema compliance (e.g. JSON validation), regex extraction targets, or deterministic assertion rules.
3. Execute the ladder test: Run the evaluation suite starting at the lowest cost tier (Qwen3-30B-A3B at $0.10/$0.30), recording schema pass rate, output accuracy, and response latency.
4. Step up selectively: Promote failed prompt subsets to Llama-3.3-70B ($0.13/$0.40) and Qwen3-235B-A22B ($0.20/$0.60), measuring the exact marginal gain in accuracy against the unit cost increase.

The code below illustrates how to execute a programmatic ladder evaluation using standard client tooling:

What to Re-Check When the Catalogue Changes

The open-weight model ecosystem evolves rapidly, with architecture releases frequently shifting the price-to-performance frontier. As open-source research teams introduce more efficient routing mechanisms, higher expert counts, and refined distillation techniques, the cost of running high-capability inference will continue to decline.

We continuously update the Serverless Inference catalogue to incorporate new dense and sparse architectures running on our sovereign European infrastructure. Whenever the catalogue updates, engineering teams should rerun their automated test harnesses against newly added efficient MoE models to verify whether workloads can be downgraded to lower-cost tiers without sacrificing production quality.

• EU data residency: 31 of 35 serverless models run strictly in eu-north1, ensuring full GDPR compliance and data sovereignty within Europe.
• Open-stack transparency: Built on open inference engines including vLLM and NVIDIA Dynamo, avoiding the vendor lock-in of proprietary black-box APIs.
• Per-token economics: Scale dynamically with zero GPU provisioning waste, paying only for the exact tokens generated by your application.

Rather than guessing which model architecture fits your application, test two sizes directly against your own task prompts on the Serverless Inference platform today to find your workload's exact cost-to-performance frontier.