Token anxiety
If you’ve used a subscription-based AI product like ChatGPT, Copilot, Codex or Claude Code, you have probably experienced a certain feeling of worry or uncertainty anchored around whatever happens between you pressing enter on a prompt, and your usage bar approaching its limit in mysterious ways.
Maybe the model was updated? Maybe your prompt spawned too many sub-agents? Maybe your provider has updated their terms for peak hour usage?
This feeling of worry and uncertainty is what some call “Token Anxiety”, and it stems from quite a foundational change in how we pay for software. And given the hype, excitement and accompanying speed of adoption when it comes to AI, it has brought with it an almost de facto acceptance of this as a trade-off for the magic of LLMs.
We have all heard occasional horror stories of AWS billing gone wild after accidental deadlocks, loops or log bombs, but with consumer-facing LLM products, this unpredictability often feels baked into the foundations of the business model. To quote anthropic’s term page for the Claude pro plan directly:
In addition, to manage capacity and ensure fair access to all users, we may limit your usage in other ways, such as weekly and monthly caps or model and feature usage, at our discretion.
Now, I don’t want to paint Anthropic as villains for trying to manage usage spikes that have clearly overwhelmed their capacity, but I can’t think of many other areas of software where we would accept such a vague and unpredictable access to what we’re paying for. At least not one that seems invoked so frequently, as people are constantly trying to figure out why e.g. Claude Code built all the right things last week, while combining the efforts of 47 sub-agents to create a token-eating black hole this week, seemingly with no clear change or reason.
If we combine this unpredictability with the pressure to adopt, and with it the all-too-common expectation of this adoption multiplying productivity aggressively, it seems an obvious recipe for anxiety. And this isn’t just true for individual developers or people using it at home; the impact quickly sums to CTOs of companies who have to justify the costs as succesful AI adoption to the board, all of which should hopefully service the business as a whole.
Using open-source models to combat token anxiety
The good news is that there is a cure for token anxiety; self-hostable, open-source (or well, some technically open-weight) LLMs. Open-source models can be downloaded, served through open-source inference tools, and there are even plenty of open-source UIs to provide things like chat interfaces, agent orchestration and more, all of which can run on infrastructure you control and decide how to pay for.
The first time I ran a simple ChatGPT-like setup on my home PC, my shoulders lowered what felt like a few centimeters. All costs - save for power - were gone, I could use it when I wanted to, and didn’t have to take a 6 hour break for a usage cycle to reset. And that’s without considering further upsides such as my data never leaving my local network, which has been a key player in keeping my collection of typescript-based twilight fan fiction safe from the prying eyes of the internet.
Now, you probably won’t be able to replace large state-of-the-art-models with your home PC overnight, but my experience is that even running smaller models for simpler tasks, or even parts of tasks before they’re handed off to a larger model, can be a great way to get started and experiment with LLM flows without anxiously staring at a usage bar throughout. This will also introduce you to - and familiarize yourself with - a community that is rapidly growing and improving.
If you want to get started working with open-source models, we recently wrote an article on how to do just that. Otherwise, you’re always welcome to reach out, and we can help you get started on your open-source LLM journey.
// Mikkel