Your AI feature has a gross margin. Do you know it?
Somebody in your company already knows what the feature earns. Almost nobody knows what a single successful use of it costs to produce.
Somebody in your company already knows what this feature earns. Almost nobody knows what one successful use of it costs to produce, which means nobody can tell you whether the thing is a product or a subsidy.
The number you want is not the monthly inference bill. It is the cost of one resolved outcome — one answer a user accepted, one ticket that closed, one document that came back correct — including everything you paid on the way there and did not count.
Count the retries
Retries, reranks, repairs, and the human who checks the output are all cost of goods. Leave them out and the margin looks fine right up until volume arrives.11The failed attempts are the line that surprises people. They are invisible in a demo, where everything works the first time.
| Line | Per 1,000 resolved |
|---|---|
| Model, first pass | TK |
| Retrieval and rerank | TK |
| Retries and repairs | TK |
| Human review | TK |
| Cost of goods | TK |
Fill that in with your own numbers before you argue with anyone about it. Mine are not the point, and neither is anyone else’s benchmark — the shape of the table is what travels, and the exercise usually takes an afternoon.
The cost of doing this: you may find out the feature you just launched has a negative margin at current pricing, and then you have to tell someone. That conversation is easier now than it is after a sales team has built a quarter on it. It is also the conversation that makes the eval work fundable, because quality and cost turn out to be the same lever pointed in two directions.
What does one resolved outcome cost you today, counting the retries?