How Lyceum's Serverless Inference Billing Works

AI This article was created with the help of AI.

The Two Serverless Modes

Modern machine learning infrastructure often forces developers to choose between managing raw virtual machines or accepting rigid vendor abstractions. Infrastructure teams building production systems require high compute density without paying for unutilized capacity between workload spikes. Serverless GPU architectures resolve this trade-off by decoupling hardware management from execution, allowing engineering teams to deploy models and custom runtime scripts without provisioning physical or virtual GPU nodes.

To accommodate different technical patterns, serverless operations are split into two distinct operational modes: API-driven model invocation and custom compute execution. Both mechanisms eliminate manual cluster setup, node health monitoring, and driver installation while operating under transparent, usage-based metering.

Per-Token Inference Against the Model Catalogue

For teams deploying open-weight models, Serverless Inference provides direct endpoint access to pre-hosted architectures across text generation, code completion, multimodal reasoning, and embeddings. Traffic routes through an OpenAI-compatible API layer powered by high-throughput engines such as vLLM and NVIDIA Dynamo. In this mode, billing measures active token throughput. Usage costs correlate strictly with input tokens processed, cached tokens reused, and output tokens generated during context decoding. You do not manage VRAM allocation, GPU quantization, or model weights across hardware nodes.

European data residency is enforced at the individual model level. Models designated for EU data residency run in European data centres in regions such as Berlin and Zurich, keeping request and response data inside the designated jurisdiction and aligned with GDPR obligations. Global routing options exist only as explicit, opt-in selections for specialized workloads that do not require regional pinning.

Serverless GPU Execution for Custom Workloads

When application logic requires custom CUDA kernels, fine-tuning scripts, synthetic data generation, or proprietary containerized pipelines, Serverless Execution provides direct compute execution without endpoint restrictions. Instead of calling a static model endpoint, you submit arbitrary Docker containers or Python scripts to run directly on high-performance accelerators.

Serverless Execution ingests standard OCI-compliant container images without requiring proprietary SDK modifications. The platform handles hardware allocation, image pulling, volume mounts, and driver binding automatically. Compute resources spin up immediately upon job submission and terminate as soon as the main process returns an exit code.

Per-Second, Workload-Aware Billing

Traditional public cloud providers bill GPU virtual machines on fixed hourly increments. If a batch processing job runs for twelve minutes and thirty seconds on an instance billed by the hour, you pay for sixty full minutes of compute time. This model transfers the financial burden of hardware provisioning waste directly to the customer, driving up operational overhead.

What Gets Measured

Usage-based GPU architectures replace fixed hourly blocks with granular per-second metering. The internal billing engine tracks compute resource utilization from the precise millisecond a job container initializes to the exact moment execution completes. Billed runtime reflects only the active operational window, preventing brief processing tasks from incurring whole-hour minimum charges.

Why the Increment Matters

Workload-aware execution goes beyond raw clock time by aligning resource consumption with active hardware demands for custom serverless execution. Across the cloud industry, enterprise AI clusters routinely run far below their available capacity, largely because instances remain powered on during idle queue gaps, data pre-processing steps, and model initialization phases. Workload-aware metering eliminates this financial waste by stopping the clock whenever compute hardware is not processing active kernel instructions.

• Kernel Execution Tracking: Metering activates when CUDA kernels begin execution and pauses instantly when processing queues clear.
• Memory Allocation Precision: Compute charges correspond to the exact GPU hardware tier and allocated VRAM requested by the workload container.
• Zero Idle Overhead: Time spent waiting for network data fetches or external API responses does not incur active GPU compute penalties.
• Sub-Second Accuracy: Invoicing rounds execution times to the nearest second, avoiding arbitrary minute or hour floor markups.

What Happens When Traffic Drops to Zero

In production environments, AI workloads rarely maintain flat, continuous request patterns. Enterprise applications experience pronounced traffic cycles, with heavy utilization during business hours followed by extended periods of inactivity overnight and on weekends. Asynchronous background workers and autonomous agent flows generate unpredictable request bursts interspersed with prolonged idle intervals inference cost optimization.

Under legacy cloud models, handling bursty demand forces engineering leads to choose between two inefficient choices: maintain dedicated GPU instances 24/7 to guarantee immediate response times, or manually destroy and rebuild clusters to save money. The first option wastes thousands of Euros on idle hardware, while the second introduces severe operational complexity and unacceptable latency during setup.

Scale-to-zero capabilities eliminate this compromise through automated listener orchestration. When request queues drain completely, the platform automatically offloads active model weights from high-performance VRAM and spins down active execution worker containers. During zero-traffic windows, the meter stops entirely, reducing active compute charges to exactly zero Euros.

When new requests arrive at an idle endpoint, the infrastructure framework dynamically routes incoming packets to an warm-standby pool or executes fast model loading procedures. Modern optimizations such as model weight caching on NVMe local storage and optimized container initialization routines keep cold-start delays minimal, restoring full inference throughput without requiring active paid capacity during idle hours.

No Egress Fees

Data transfer charges represent one of the most unpredictable lines on cloud infrastructure invoices. While inbound bandwidth is typically free, hyperscale public cloud platforms charge substantial fees for outbound data transfer over the public internet. For machine learning workflows involving multi-gigabyte dataset transfers, streaming video inference, or high-throughput log delivery, egress charges create significant budget variance.

What Hyperscalers Typically Charge

Legacy cloud vendors assess bandwidth surcharges on outbound traffic that typically land around ten cents per gigabyte: AWS bills data transfer out to the internet at $0.09 per GB for the first 10 TB each month, after a 100 GB free allowance, tiering down at higher volumes. Other hyperscalers publish comparable or higher first-tier rates, and some regions carry a premium on top. Last verified: June 2026. When transferring high-resolution media, fine-tuned model checkpoints, or massive embedding databases across cloud regions, data egress fees often rival or exceed the raw compute cost of the GPUs themselves egress fee comparison.

Lyceum enforces a zero-egress fee policy across all serverless compute modes. Inbound data ingestion and outbound response streaming pass over high-speed networks without volume-based bandwidth surcharges. Eliminating data transfer fees ensures that high-volume inference applications, model streaming pipelines, and multi-cloud dataset syncs run with complete financial predictability.

Total Cost of Compute

Evaluating GPU infrastructure purely on a nominal hourly card rate leads to flawed cost projections. A low nominal rate per GPU hour becomes expensive if poor hardware utilization, high data egress penalties, and compulsory monthly commitments inflate the total invoice. Effective machine learning FinOps requires evaluating the Total Cost of Compute across every layer of the infrastructure stack.

Compute, Egress, Compliance, Idle Waste

The Total Cost of Compute framework combines four distinct financial factors: raw active compute consumption, network data transfer overhead, regulatory and data sovereignty compliance management, and idle capacity provisioning waste. Standardizing infrastructure assessments around total compute cost exposes the hidden expenses associated with rigid provisioning models.

By uniting per-second metering, scale-to-zero orchestration, zero-egress networking, and native European data residency, usage-based serverless billing lowers the aggregate financial footprint of production AI systems. Engineering teams pay strictly for productive mathematical operations rather than idle hardware buffers or vendor bandwidth markups.

• Raw Compute Execution: The net spend required for active CUDA kernel execution measured in exact seconds.
• Network Egress Expenses: Bandwidth fees incurred when transferring results, logs, and checkpoints to external endpoints.
• Sovereignty & Compliance Overhead: Financial liabilities associated with transferring sensitive European citizen data into non-compliant legal jurisdictions.
• Provisioning Waste: Unused VRAM and idle GPU cycles paid for during traffic lulls, queue delays, and cluster warm-ups.

What Isn't Charged

Transparency in cloud billing requires absolute clarity regarding omitted line items. Legacy providers frequently obscure total operational costs behind complex pricing tiers, support agreements, and resource reservation fees. Serverless GPU billing strips away these ancillary charges to deliver straightforward usage-based pricing.

When deploying workloads on usage-based serverless infrastructure, multiple traditional cloud line items are permanently set to zero. This operational simplicity allows engineering leads and financial managers to forecast spend using simple operational metrics rather than multi-page fee schedules.

• No Base Monthly Fees: There are no account maintenance fees, platform subscription tiers, or recurring node management charges.
• No Storage & Data Retention Penalties for Inactive Models: Pre-hosted catalogue models incur zero storage or mounting fees.
• No Ingress or Egress Surcharges: Data transfer into and out of European endpoints incurs zero bandwidth fees.
• No API Wrapper Markups: Serverless Inference endpoints route requests through open engines without proprietary gateway surcharges.
• No Minimum Spend Commitments: Developers run production workloads without locking capital into multi-month minimum usage contracts.

Estimate Your Cost

Forecasting monthly infrastructure expenditure under a per-second, usage-based billing model relies on workload volume and execution duration rather than static node counts. By breaking down application traffic into discrete processing parameters, engineering teams can calculate anticipated compute spend before deploying code into production.

A Method You Can Run

To estimate monthly costs for custom serverless tasks, measure the average execution duration per request in seconds, multiply by the expected monthly request volume, and apply the corresponding per-second rate for your target GPU accelerator tier. For API inference workloads, multiply your projected input, output, and cached token volumes by the specific model token rates.

Rather than relying on static estimations, engineering leads can input active concurrency patterns, target GPU tiers, and expected processing volumes directly into the interactive savings calculator, which returns a forecast against current rates.

1. Determine Workload Runtime Profile: Measure average task completion time in seconds across test batches.
2. Project Monthly Request Volume: Estimate total expected invocations based on peak and off-peak user traffic.
3. Calculate Total Active Compute Seconds: Multiply average runtime by request volume to establish net billable seconds.
4. Run Interactive Math on the Savings Calculator: Input active parameters into the calculator surface to view precise operational spend forecasts.

Where to Find Current Rates

Rates change as hardware generations and model catalogues change, so this page deliberately carries no rate card. Current per-second GPU rates and per-token model rates live on the live pricing page, which lists the available accelerator tiers alongside their published figures. The savings calculator applies those same live rates to your own concurrency, runtime, and volume inputs, so use the pricing page to confirm a rate and the calculator to turn it into a monthly forecast.