Estimating the cost of an AI API feature is easier when you separate request volume, token usage, model rates, retries, caching, and operational overhead. This worksheet gives you a repeatable method for measuring representative prompts, building low and high usage scenarios, comparing models, and reviewing actual spend against your forecast without relying on guesswork.
Overview
An AI API cost estimate should answer more than “What is the price per token?” The useful question is: How much will this feature cost at its expected traffic, prompt size, response length, and reliability requirements?
Most providers separate input-token and output-token charges, although billing units and pricing rules differ. Some services also apply distinct rates for cached input, batch processing, tool calls, audio, images, or other modalities. Your estimate must therefore match the exact API operation you plan to use.
The basic monthly model is:
Monthly cost = request cost + retry cost + fallback cost + related platform costs
For a text-only model, request cost can be estimated as:
Request cost = (input tokens ÷ billing unit) × input rate + (output tokens ÷ billing unit) × output rate
Use the provider’s current pricing page, billing unit, tokenization guidance, context limits, and rate-limit documentation when filling in the worksheet. Do not assume that two models count tokens or price cached content in the same way.
A cost estimate is not a promise. It is a planning range. A low scenario can represent normal, efficient usage; a high scenario should include longer prompts, larger responses, traffic variation, retries, and occasional fallback requests.
How to estimate
1. Define the feature and billing period
Write down exactly what you are estimating. Examples include document summarization, support-message classification, code assistance, retrieval-augmented question answering, or an internal research tool. Choose a period such as a day, week, or month, and record the expected number of active users and requests.
Do not begin with an abstract number of users if the feature is request-driven. A better starting point is:
Requests per period = active users × requests per user
Keep separate counts for interactive requests, background jobs, scheduled evaluations, and administrative tasks. They may use different models or have different latency requirements.
2. Measure representative prompts
Collect a small test set that reflects real inputs rather than one carefully written example. Include short, typical, and unusually long requests. Measure input and output tokens separately. If the feature uses a system prompt, retrieved documents, conversation history, tool results, or structured instructions, include those components in the input measurement.
For a first estimate, record the average and a high-use value for each:
- Input tokens per request
- Output tokens per request
- Requests per user or workflow
- Percentage of requests that fail and retry
- Percentage that use a fallback model
- Percentage that can reuse cached input or output
3. Apply the current model rates
For each candidate model, enter the current input and output rates exactly as displayed by the provider. Record the currency, billing unit, model version or identifier, and the date checked. This creates an audit trail when you compare the estimate with a later invoice.
A comparison table can use this layout:
| Input | Model A | Model B | Model C |
|---|---|---|---|
| Input rate | Enter current rate | Enter current rate | Enter current rate |
| Output rate | Enter current rate | Enter current rate | Enter current rate |
| Context limit | Verify limit | Verify limit | Verify limit |
| Latency target | Measure | Measure | Measure |
| Quality result | Score test set | Score test set | Score test set |
Price alone is not a sufficient model-selection criterion. A cheaper model may require longer prompts, more retries, additional validation, or fallback calls. Evaluate cost alongside quality and latency using a fixed test set. The Smart QBits guide on evaluating AI outputs can help structure that comparison.
4. Build low, expected, and high scenarios
Create at least three scenarios. The low case uses conservative traffic and efficient prompts. The expected case uses your best forecast. The high case increases request volume and token usage and includes operational events such as retries or fallback routing.
For each scenario, calculate:
Monthly input tokens = requests × input tokens per request
Monthly output tokens = requests × output tokens per request
Base cost = monthly input cost + monthly output cost
Then add expected retry and fallback costs. If 3% of requests retry, model the additional requests rather than merely adding 3% to the final cost; a retry can have a different prompt, response length, or model.
Inputs and assumptions
Use this worksheet as a starting point for a spreadsheet or internal cost calculator.
| Input | Value to record | Why it matters |
|---|---|---|
| Billing period | Day, week, or month | Defines the comparison window |
| Requests | Total feature requests | Drives total token usage |
| Input tokens | Average and high value | Includes instructions, history, and retrieved context |
| Output tokens | Average and maximum target | Often changes materially with verbosity |
| Input rate | Provider’s current rate | Converts input tokens to cost |
| Output rate | Provider’s current rate | Converts generated tokens to cost |
| Cache ratio | Share of reusable content | May reduce eligible input cost, depending on provider rules |
| Retry ratio | Failed or repeated calls | Captures reliability-related spend |
| Fallback ratio | Requests routed elsewhere | Captures overflow, quality, and availability paths |
| Latency | Time to first token and completion | May influence model choice and parallel calls |
Context size deserves special attention. A long conversation or retrieval result can make input tokens grow on every turn. Set a maximum history length, retrieve only relevant passages, and measure the prompt after all application logic has assembled it. Prompt templates that look short in source code can become expensive when they include repeated documents or tool outputs.
Caching can lower cost or latency when the provider supports it, but never assume every repeated string qualifies. Record which portion of the request is stable, which portion changes, and how the provider defines an eligible cache hit. Batch processing may also have different economics, but it is appropriate only when delayed results are acceptable.
Include non-token costs when they are material: vector search, storage, moderation, file processing, observability, proxy services, and application infrastructure. Keep them in separate rows so that the model cost remains understandable.
Worked examples
Consider a hypothetical summarization feature with 10,000 monthly requests. Suppose measurement shows an average of 1,200 input tokens and 300 output tokens per request. The provider’s pricing page lists an illustrative input rate of R-in currency units per billing unit and an output rate of R-out currency units per billing unit. Replace these placeholders with current rates before using the result.
Monthly usage is:
- Input: 10,000 × 1,200 = 12,000,000 tokens
- Output: 10,000 × 300 = 3,000,000 tokens
If the billing unit is one million tokens, the base estimate is:
(12 × R-in) + (3 × R-out)
Now add operations. If 4% of calls retry once, the extra volume is 400 requests. If those retries use the same token profile, add 480,000 input tokens and 120,000 output tokens. If 2% of all requests use a fallback model, calculate that 200-request path separately using the fallback model’s rates and measured token profile.
For a high scenario, increase requests to the forecasted peak, use the 90th or 95th percentile token measurements if available, and include the observed retry and fallback ratios. The result is more useful than multiplying the average case by an arbitrary buffer because each assumption remains visible.
When comparing two models, calculate both cost and test-set results. A model that costs less per request may produce longer responses or require a second validation call. Conversely, a higher-priced model may reduce retries or allow a shorter prompt. Treat the final decision as a quality-per-cost and latency trade-off, not a rate-card comparison alone. For broader model decisions, see the guide to local versus cloud LLMs for developers.
When to recalculate
Revisit the worksheet whenever a pricing input or workload assumption changes. At minimum, recalculate after:
- A provider changes input, output, cached, batch, or tool-use pricing
- You switch models, providers, model versions, or tokenization behavior
- The prompt gains conversation history, retrieval, files, or new instructions
- Traffic forecasts, user behavior, or request frequency changes
- Latency targets lead you to add parallel calls or fallback routing
- Observed retries, refusals, truncation, or output length differs from testing
- Context limits, rate limits, or provider billing policies are updated
Set a monthly review for an active production feature. Export actual request counts, input tokens, output tokens, cache hits, retries, fallback calls, and billed totals. Compare those values with the forecast by scenario, then update the assumptions that caused the largest difference.
Before launch, make the worksheet part of the feature checklist. Measure a representative test set, save the pricing snapshot and date, set a low-cost alert and a hard operational limit, and decide what the application should do when the budget or rate limit is reached. After launch, review cost per successful task rather than cost per API call alone. Finally, update the estimate whenever you change prompts, models, traffic, or routing. This turns an AI API cost calculator from a one-time spreadsheet into a practical control for ongoing AI development.