Seven sensible ways to power AI adoption without making your finance director cry

This is a follow-on from the previous post about AI tokens and how to think about their consumption.

Here are some ways to manage consumption that don't harm outcomes:

1. Start a new session for a genuinely new task

Don't let the model be dragged down with the burden of previous conversations, pulling excess context into the next question.

Only carry forward context that has value.

2. Retrieve less, but retrieve better

Pre-process before handing data off to the LLM model. If you can narrow down 100k records to 50 using traditional logic processing, do that! Software languages and tools are exceedingly good at filtering data, so use them.

It should not be the LLM's task to find "WHERE CUSTNMBR='2343243'"

3. Select appropriate models for the task

Not every problem needs a reasoning model, or deep research, and not every problem needs the latest mode -although some efficiencies come with newer models sometimes.
Auto routers are appearing more often and are handy for routing tasks to models, but we are in the early days of that. You often have more knowledge to decide than any LLM auto router and will make a better selection than it.

Invest time into assessing the appropriate model for the task in hand.

4. Watch those automated jobs

Agents that are on schedules, scheduled prompts and autonomous workflows should be monitored. They may become redundant over time due to changes in the business, yet still run every hour.
Prompts that were once efficient, may no longer be if data grows and morphs over time, so keep checking what is running in production.

Anything capable of spending money while you sleep should have more financial supervision than a houseplant.

5. Catch and break out of failing loops early

Don't fear stopping an agent that is clearly stuck or pursuing the wrong goal or approach. Stop it now. Improve the instructions and/or context, then start again. Loops can eat tokens.

Repeatedly allowing the system to “have another attempt” can generate remarkable token consumption without improving the outcome.

6. Audit what nothing is using

Examine:

• scheduled AI jobs
• reports
• background agents
• alerts
• summaries
• automated research that is scouring the web daily

Check if anyone is actually still using the output. In old-school IT, so many reports were generated in SQL daily where the interest and focus of the business had long since moved away, resulting in no one looking at dozens of daily reports at all. This is low cost in SQL, not so with LLM calls, monitor who uses outputs.

The cheapest token is not necessarily one you didn't spend.
however a token spent producing something nobody wants has a particularly difficult case to make.

7. Caching can help

Prompt caching can help. Deduplicate and cache any content that repeats.
I am surprised how often that I find I can cache AI calls, or parts of the input and output without affecting output, but saving heaps on processing costs.

Some AI providers will even provide caching layers, reducing the token price for chache content, dealing with caching for you sometimes pretty transparently.

Cached prompt content still occupies context, so it's not free, but it is cheaper.