A long time ago, when they didn’t need to have numbers and special characters, one of my favourite passwords was GRIBNITZ. It stood for “Gargantuan Robots Idle Because Nigel Is Totally Zonked” (which is why I can remember it).

GRIBNITZ is the acronym from the beginning of a sentence I was passed when playing that game where everybody sits around the table and take turns adding one word to a story. When it gets to your turn, you have to recite the whole story back, word perfect, without making a mistake, or suffer a forfeit (use your imagination; playing was the idea of a broking team I was entertaining with my fellow underwriters).

AI plays this game every time you give it a prompt. Contrary to popular belief, the chatbot interface window on your desktop doesn’t learn anything, ever. When you ask Claude or ChatGPT (or whatever) to do it again, but make it in the style of Beaudelair (or whatever), you’re friendly desktop AI doesn’t begin where it left off. Instead, it starts from scratch. To do that, it goes back to the very beginning of your chat, reads and analyses everything over again, and replies anew to your request based on your slightly modified prompt.

That’s all well and good, unless you’re paying the bill.

At the FASE MGA conference in Barcelona last week, I had a fascinating conversation with an underwriter-manager who is convinced that AI is costing him more, not less, because of all the tokens his employees use.

No one should be surprised if this is happening to them right now, this instant.

AI isn’t free. Sure, you may be thinking, I pay £22.58 a month for ChatGPT at home, which isn’t cheap, but the free version kept cutting me off. But your employer is paying more. Probably orders of magnitude more.

They’re helping to cover the cost of all that data-centre construction, electricity consumption, water-cooling systems function, and everything else to do with data centres that we hear about so often, insurance included. They’re paying for the infrastructure that enables the cloud computing that allows you to ask a chatbot where blue tits live when they’ve left the nest after their chicks fledge (or whatever).

AI usage is measured in tokens. One token buys a little bit of AI processing. That’s backed by a little bit of computing time, which is fuelled by a tiny bit of electricity, which is why some people have rejected their AI-enhanced headshot on LinkedIn for environmental reasons. (For the record, all my firm’s headshot illustrations are created by human designers.)

According to ChatGPT, a token equates to roughly four characters of output, including punctuation, formatting, code symbols, and so forth. So my average column is about a thousand tokens (except it isn’t, because I don’t use AI to write it). The cost varies dramatically depending on the platform and the plan, but for ease let’s say 10,000 output tokens cost about 10 cents. That’s about eight times more expensive than input tokens (the cost of prompts), because generating text is computationally heavier; it includes the price of all the ‘work’ the AI does for you.

But input can still be hugely costly, and is often particularly wasteful. As ChatGPT just warned me, “the entire conversation history sent to the model counts again [that is, adds to your token usage] every request unless you trim it. Long chats therefore get progressively more expensive.” Further, the bot advised me, “A 100k-token prompt costs disproportionately more compute than a 1k-token prompt because the model must attend over much more context.”

This has important practical applications for the profitability of AI to your business. When, enthusiastically in the hope of driving innovation, AI was rolled out internally by putting an AI window on the desktop of every employee from reception to the C-Suite, everyone was in effect given an unlimited debit card against the IT budget. The mantra ‘give it to them and see what happens’ is a foregone conclusion: they will just run up the bill.

Unless you offer at least a modicum of AI training, that is. Everyone who has access to AI on the company tab should be showed how to use it. At the very least, they need to know to use the ‘New Chat’ button every time they start a fresh task, instead of simply typing into the existing string of prompts and responses, which simply flushes tokens down the virtual loo.

But that’s just cost control. To get any genuine innovation from the widespread dispersal of AI in your business, it’s essential for users to understand the different levels of AI operation (as I explained in an earlier diary entry), and how to progress through them.

At Level 1 of AI usage, AI is just a Q&A, and you may as well use a Google search for free. When usage advances to Level 3, though, the user is able to teach ‘skills’ to the AI. That’s where it can become an increasingly and incredibly useful business tool. But in practice almost no one will get to Level 3 without help from a colleague or trainer who knows how to use AI. They’ll just run up the bill.

Fortunately just a small investment in AI training is likely to be recouped through more prudent token usage. With a little more, you will dramatically increase the likelihood that some bright spark on the front line will find ways to use AI that make processes more efficient, and real money can be saved. Get there, and you’ve reached the place where the insurance sector can stop repeating the same old story.

* Like every Insurance Technology Diary entry about AI, this one is accurate only to the best of my knowledge at the time of writing. The pace of AI progress is so great that I cannot guarantee it remains so now that it’s finished, let alone when you read it.

Guillaume Bonnissent is CEO of Quotech.