One column, one price

The usage page told me about the problem before I knew I had one. It said the API pricing snapshot was three months old. That is a routine kind of warning, so I went to check the rates against the published price page, expecting a small edit.

The refresh found a real bug of its own. My rate lookup matched models by family name only, so every newer model was billed at its older sibling's rates. The plan was to correct the rate table and then reprice the session history from the raw token counts, so every stored cost would reflect the right prices.

Before a reprice writes anything, I run it dry. A dry run computes the new costs and shows the difference without touching the stored rows. That dry run is where the second problem surfaced, and it had nothing to do with rates.

With the old rates still in place, the dry run already wanted to cut the largest model's history. A rate change could explain movement. It could not explain a cut on prices that had not changed. The cut came from cache writes, and it was exactly the gap between the two tier prices on every one-hour write. That cost was real.

Cache writes are the tokens I pay for to put something into the model's prompt cache. There are two tiers. A five-minute write is billed at 1.25 times the input rate, and a one-hour write at 2 times. The session rows in my records already kept the one-hour count in its own column. One level down, inside each session's per-model breakdown, the slots stored a single total for cache writes. The repricer works at the slot level, so it saw one number and billed all of it at the five-minute rate.

No one chose the cheaper rate. A single total forced the job to assume one, and the assumption underpriced everything that belonged to the other tier by the difference between the two rates. Before the rate change merged, I put a plain warning in it:

Do NOT run reprice_sessions --apply after this merges.

Then I went to find out how much of the history was already wrong.

I ran read-only probes against production that same day. They found stored costs that were wrong in two opposite ways.

Splitting the sessions into groups made the method easier to trust. The 1,713 sessions whose stored cost had been right barely moved under the reprice, by a rounding error across all of them. Where the data had been correct, the method left it alone.

The go-forward change was small. The parser now stores the one-hour and five-minute token counts separately in every slot:

slot keys, before:  cache_creation_input_tokens   (one total for both tiers)

slot keys, after:   cache_creation_input_tokens
                    ephemeral_1h_input_tokens
                    ephemeral_5m_input_tokens

The breakdown is stored as JSON, so no schema migration was needed. New sessions carry the split from the moment they are ingested, and the repricer has nothing left to guess.

The history needed more care, because the split had never been recorded at the slot level. I had each session's one-hour total and had to apportion it across that session's slots. A plain pro-rata split would have mispriced most sessions. The main conversation thread writes one-hour cache and sub-agents write five-minute cache, so the apportioning goes lane first: main-thread slots take the one-hour budget first, then sub-agent slots, each lane split by its share of cache-write tokens. The last slot in a lane takes the integer remainder, so the totals add up exactly.

I checked the rule before trusting it. On production, the main lane's cache-write total equalled the session's one-hour column, and the sub-agent lane equalled the five-minute column, for 2,153 of 2,202 sessions. The other 49 get an apportioned split, which is an estimate and not an exact record. The apply writes the split back into every slot, so the next reprice reads it instead of apportioning again.

The apply was a separate, supervised write to production that I approved before it ran. A pre-image of every changed row was dumped first, with a restore command that puts the rows back. It changed 2,205 sessions. A second dry run afterward showed a difference of zero, and all 3,751 slots carried the split. One session on the newest model, made up entirely of one-hour writes, rose about 11 percent to its true cost, and its per-model rows still summed correctly.

The test that guards the split is deliberately awkward. Its fixture inverts the usual pattern, with five-minute writes on the main thread and one-hour writes on a sub-agent, so the lane-first rule would get it wrong. The test can only pass if the parser really kept the split. Dropping the split from the stored keys fails two tests.

Some of it is still open. A Mac on old code still posts slots without the split, and the repricer falls back to apportioning for those. A Mac on very old code can still overwrite repriced rows when a transcript changes, which is how the 54 stale sessions happened, and this change did not fix that. A separate report on wasted spend still prices by family name. When two things have two prices, I now give them two numbers at every level where something will later be priced. A merged total leaves some later job to pick one rate for it, and that job will pick wrong for one of the tiers.