An AI cloud can report strong demand and still have a difficult margin problem. Accelerators are expensive, power is not free, buildings take years to pay for, and idle capacity still creates fixed cost. The useful question is not whether AI demand exists. It is whether each unit of delivered compute earns more than the full cost of making it available.

The short answer

AI cloud economics depend on utilization, pricing, power efficiency, depreciation, financing, networking, maintenance, and support. High utilization helps spread fixed cost, but a provider also needs enough pricing power to cover hardware refreshes and unexpected capacity problems.

The cost stack

Start with the server and accelerator. Add memory, storage, networking, racks, cooling, power delivery, buildings, staff, software, and financing. Depreciation spreads owned equipment across its useful life, while a lease creates a different payment and accounting profile.

Power is especially important because dense accelerator racks consume a large amount of electricity and generate heat. A favorable energy rate can be offset by demand charges, transmission upgrades, backup systems, or cooling.

Utilization is not one number

A facility can be technically powered while its accelerators are waiting in a queue, running low-value workloads, or reserved for a customer who has not started using them. Ask whether utilization means powered-on hardware, scheduled work, revenue-producing work, or completed useful output.

I prefer three measurements: available capacity, scheduled capacity, and productive capacity. Compare them by hour and by workload. A 2-hour sample can reveal queueing, but a full month is better for understanding quiet periods.

What developers should measure

When choosing an AI cloud, compare cost per useful result, p95 latency, error rate, queue time, egress, and support quality. A low hourly price is less attractive if your service spends more on retries or needs a second provider during incidents.

Keep provider telemetry in your own system. Record region, model, hardware class when available, tokens, duration, and status. That makes a price or capacity change visible before it becomes a customer complaint.

Sources and limits

NVIDIA’s annual report discusses infrastructure expense and indirect customers. Its latest quarterly filing discusses AI cloud commitments. OpenAI’s compute infrastructure overview explains why power and facilities are central to the build. Public disclosures rarely reveal a provider’s complete unit economics.

The practical lesson is to treat utilization as a quality signal, not just a growth statistic.

A developer’s comparison table

Measure Why it matters
Queue time Shows whether capacity is available when requested
Productive utilization Separates useful work from powered-on hardware
p95 latency Shows the experience during busy periods
Cost per accepted result Includes retries and quality corrections

Collect these values before switching providers. A 2-hour test is useful for a first screen, while a month of production data reveals seasonality and quiet-period waste.

A warning about fixed commitments

Reserved hardware can improve availability while reducing flexibility. If demand falls, the provider still carries power, lease, financing, and depreciation costs. If demand rises faster than expected, the provider may need new capital before the existing fleet has paid for itself.

This is why revenue growth and margin expansion can move in opposite directions during a build-out. Developers see the result as quota changes, minimum commitments, regional limits, or a price increase. Track the service behavior instead of assuming that a larger facility automatically improves the product.

Run a 2-hour comparison during both quiet and busy periods. The same hourly price can produce a very different user experience when queueing begins.

Spend 2 hours validating the measurement definitions before publishing a margin comparison. Otherwise two providers may be reporting different things under the same word.

Use the same definitions across every quarter and provider.

Consistency makes the trend trustworthy.

It also improves planning.

Use it consistently.

Review the trend quarterly.