we counted the wrong thing
one equation for why ai token demand exploded without more people showing up.
i handed off most of my job this year.
not to a person. to a file.
i wrote claude a spec for what done means, then a second file called learnings, where it records what it got wrong.
one sets the standard.
the other keeps it from making the same mistake twice.
on the payroll i’m still one person. same seat. same row. same line in the same spreadsheet.
the headcount didn’t move.
the workload did.
what is actually happening.
everybody with money in ai infrastructure is saying some version of the same thing.
ai demand is taking off.
and most of them benefit from you hearing one explanation.
more people are using ai.
that part is true. it just doesn’t explain the load.
benedict evans put the numbers beside each other in may. openai has more than nine hundred million weekly users, but most of them still use it lightly.
the demand for tokens is real.
the explanation is wrong.
we keep counting people while the machine does more work per person.
users × questions.
count how many showed up. count how often each one asked. one answer per request.
it’s the same engagement equation we’ve run on every internet product. users. sessions. searches. messages.
most ai adoption charts are that equation in a new outfit.
it goes blind the moment one request turns into a process.
a chatbot waits after every answer. an agent starts its own loop. it plans, reads, searches, calls tools, checks the result and goes back when something is wrong.
one round trip becomes ten, fifty, sometimes a hundred underneath one instruction.
the person asked once.
the machine kept going.
adoption × volume × depth.
adoption = share of people who hand work over
volume = workflows × frequency
depth = steps × work per step × overheadadoption is how many people hand work over.
volume is how much work gets handed over and how often it runs.
depth is everything that happens inside each task, including the checking, retrying and correcting underneath it.
what i noticed.
training is the cost of teaching a model. it happens in large, occasional runs.
inference is the cost of running it afterward. every time it reads a prompt, writes an answer, searches, calls a tool, checks its work, or tries again.
training is a project. inference is a meter.
not bought once. running on every task anyone hands over.
and the tasks keep getting longer.
jensen huang puts agent work at around fifty times the compute of a simple chat. that number comes from the company selling the chips.
the line didn’t rise because fifty times as many people arrived. it rose because one instruction set off far more machine activity underneath it.
so count the task, not the person.
agent workload = people × tasks per person × work per task.
a chat question might use a thousand tokens. say one agent task reaches sixty five thousand after its steps, carried context and retries.
same people.
fewer tasks.
twenty six times the load.
now hold everything still and move one term at a time.
the number of people has a ceiling.
volume and depth can keep expanding long after user growth slows.
once a week becomes every hour. ten steps become thirty.
nobody new had to show up.
tokens are not compute. model size, context length, caching, batching and hardware all change what a token costs. but tokens are the closest public meter we have for how much model work is being requested.
and the meter is spinning.
how i’m seeing it.
the workload equation tells you how much machine activity is being requested. it does not tell you how much physical capacity that activity needs.
for that you need a second one.
the buildout = agent workload × infrastructure per unit of work.
efficiency means one task needs less of the rack it runs on. demand means more tasks hit it.
the buildout grows when demand for tokens wins.
running a model has two halves.
prefill reads what you gave it before it starts answering and builds its working memory.
decode writes the answer piece by piece, reaching back into that memory each time.
every agent step contains both. more workflows create more model calls. deeper workflows often make later calls heavier because they carry more history and tool output.
prefill leans harder on compute and power. decode leans harder on memory bandwidth and capacity. tool use pushes part of the work beyond the gpu, into cpus, networking, storage and outside software.
grid connection waits now run four to seven years in northern virginia, phoenix and dallas.
and by one teardown, the gpu’s share of nvidia’s next rack falls from sixty three percent to fifty one.
the demand did not leave the gpu.
the rack got heavier around it.
some lingering thoughts.
cheaper tokens change the equation. they make more workflows worth running, raise frequency, and let agents spend more steps checking their own work.
the question isn’t whether the cost of a token falls. it’s whether it falls faster than the amount of work grows.
i don’t think anybody knows yet.
the term i keep coming back to is depth. not because it is the only one that matters. because it is the one that measures trust.
you ask a chatbot a question and watch the answer arrive. you give an agent a task and let it work without you.
the more it checks itself and corrects course, the more you are willing to leave with it.
the task gets longer. it becomes more useful. you hand over something harder.
depth compounds.
a new user adds demand.
a trusted agent multiplies it.
adoption tells you how many people showed up.
volume tells you how much they handed over.
depth tells you how far they were willing to walk away.
that last number is trust.
so stop watching the count.
the buildout won’t be decided by how many people open an ai app. it’ll be decided by how many tasks they hand over, how often those tasks run, and how long the machine keeps working after they leave.
that’s what we should’ve been counting.
— brylan.