A capacity plan can stay useful by using historical prompt and completion token metrics as its baseline, then re-estimating throughput whenever the model or input/output token mix changes. The objective is a repeatable planning method rather than a permanently fixed capacity number. The relevant technical guidance identifies those two historical metrics and states that provisioned throughput depends on both the model and the input/output token mix in a given minute.
Start with two separate measurements
System-level throughput estimation requires historical data for prompt tokens and completion tokens. Keeping these measurements separate preserves the two inputs identified by the guidance; a combined token total alone would not show whether both have been measured.
The historical records should also remain associated with the model that processed the workload. Without that context, an old throughput estimate becomes difficult to interpret after the model changes.
Review the workload profile, not just total demand
Demand volume is only part of the planning picture. The input/output token mix can also affect the throughput obtained by a deployment. A practical review method derived from these dependencies is to:
- Measure historical prompt and completion token usage separately.
- Identify the model associated with each observation period.
- Record the input/output token mix for the minutes being evaluated.
- Re-estimate throughput when the model or token mix changes.
- Retain earlier assumptions so that changes to the workload profile remain visible.
This process does not require predicting the direction of demand. It ensures that each capacity estimate remains connected to the conditions under which the underlying measurements were collected.
Why a static estimate can become misleading
A throughput estimate reflects a particular workload profile. If the model changes, or the proportions of input and output tokens differ from the historical period, the previous estimate does not fully describe current conditions.
That does not, by itself, prove that capacity is inadequate. It means the estimate needs to be checked again. A plan that preserves this review process can accommodate changing demand without treating one historical configuration as permanently representative.
What operators must still confirm
Before adopting a capacity target, operators must confirm:
- Whether historical prompt and completion token measurements are complete and consistently recorded.
- Which model applies to the relevant workload period.
- What the input/output token mix was during the minutes used for estimation.
- Whether recent demand has changed enough to require a new estimate.
- Whether current deployment-specific guidance supports the resulting capacity decision.
The two cited points establish the measurement basis and the relevant dependencies, but they do not provide a universal capacity target or a fixed conversion rule for every workload. That final determination still depends on current workload records and applicable technical documentation.