Amazon Bans Tokenmaxxing: AI Usage Metrics Called Misleading

Amazon has openly corrected the way the company assesses the use of artificial intelligence in the workplace. In an internal memo circulated last September and only published on the company's official site on Tuesday (27/10), the e-commerce giant called the number of tokens and the volume of code generated "deeply flawed" indicators for measuring AI adoption. The move signals that the wave of generative AI adoption is entering a cooler evaluation phase.

The memo stresses that metrics such as token usage, the adoption rate of AI tools, and the number of code changes or reviews are not fit to serve as a single benchmark. According to Amazon, relying on those numbers can actually inflate costs and add system complexity. But the company does not simply ask employees to ignore them; such data may still be viewed, provided it comes with careful judgment and is compared against other indicators.

"We believe the best thing is to measure, and be measured by, long-term outcomes," the memo reads. The statement marks a shift in rhetoric from merely encouraging AI experimentation toward accountability for results. Amazon also stresses that if a piece of work does not benefit from the available AI tools, that is a legitimate finding, not a failure. An engineer who can explain where AI has not helped their work is seen as showing the judgment the company expects.

The practice Amazon criticizes is known as tokenmaxxing, namely using AI models as much as possible regardless of output quality. The phenomenon emerged as AI computing costs soared. Several companies have installed guardrails on token consumption or switched to cheaper open-source models for certain tasks. Amazon itself briefly shut down an internal leaderboard in May after finding employees spurring token usage to climb the rankings.

According to Marc Brooker, a vice president and distinguished engineer at Amazon Web Services, the company now weighs "feature velocity" — how many launches and changes make it into production — as a measure of engineer productivity. That approach, he says, is essentially a return-on-investment calculation: if there is a problem that matters to customers or the business, the company will invest tools, time, tokens, and human effort. What it wants to avoid is waste on solving unimportant problems, though he acknowledges there still needs to be room to experiment.

Amazon's move reflects broader pressure across the global technology industry. After two years of racing to integrate generative AI, companies are now being asked to prove that heavy spending on language models and computing infrastructure is proportional to performance. Amazon CEO Andy Jassy previously predicted that AI would change the scope and pace of day-to-day work, and could even shrink the number of corporate employees in the coming years, while creating new kinds of jobs.

For Indonesia, this dynamic is relevant to companies beginning to adopt generative AI. Many domestic organizations measure the success of digital transformation by the number of licenses or the intensity of tool usage, not by business impact. The lesson from Amazon shows that without a mature evaluation framework, AI adoption can end in inflated costs without a commensurate productivity improvement. Companies at home that want to avoid a similar trap need to set outcome indicators first, such as product launch time or service quality, before expanding their use of AI.

Going forward, competition among technology companies is likely to be less about who adopts AI fastest and more about who calculates the trade-offs most carefully. The question is whether metrics like feature velocity can truly capture the value of AI, or whether they will simply spawn a new form of measurement that can be gamed just as easily.