OpenAI’s Sam Altman recently revealed that the company’s heaviest user consumes more than 100 billion tokens per month, with the global per capita average hitting 100,000 tokens per month.
Those numbers describe a platform that has broken through. What they do not describe is the far larger population of AI tools that launched with similar ambitions and quietly faded out.
With tens of thousands of AI tools now available and more being introduced every day, they can’t all succeed. The question is how to quickly determine which ones will and which ones won’t. The answer, increasingly, lies in signals that most companies are missing.
Why standard AI adoption metrics miss what matters most
Too often, technology developers rely on basic statistics such as total user numbers or total hours spent with a tool to evaluate whether adoption is working. Those numbers only tell part of the story, however.
Not all users may be satisfied, and in some cases, a high number of hours spent with a tool is actually a sign of inefficiency and frustration rather than the tool solving meaningful problems.
More AI:
- Goldman Sachs has blunt message for AI stock investors
- Microsoft CEO sends a blunt warning on AI and the tech ecosystem
- The next AI infrastructure race has nothing to do with chips
The data reflect how widespread the gap between adoption and genuine value has become. Gartner reported that only one in 50 AI initiatives has delivered transformative value for companies, with culture dissonance and a negative impact on employee mental fitness often among the primary outcomes.
In one recent survey, 64% of working adults reported that the tools and systems they used at work actually hurt their productivity, and 12% admitted to completely abandoning tasks because of technology frustrations.
“Shadow AI” is a warning signal, not just a security concern
Among the hidden signals that indicate technology is causing problems rather than solving them is the rise of “shadow AI,” defined as unauthorized tools employees adopt on their own because the approved ones are not working for them.
Most organizations treat shadow AI primarily as a cybersecurity risk. It is also one of the clearest indicators that a technology rollout is failing.
Data analytics expert Kwan Chun Clinton Ngan has spent years studying how workforce behavior reflects the true health of a technology rollout. He describes the pattern in straightforward terms.
“There’s always a bit of resistance or hesitancy when new tech is introduced, but over time, quality tech is going to win out and replace the old methods,” Ngan, who has broad experience across people analytics and HR data, told TheStreet.
“On the other hand, if a workforce finds that new tech doesn’t solve their problems or is too difficult to adopt, they’re going to find ways to keep using the tools they’re already comfortable with. This is one of the biggest signals that new tech is less likely to succeed long-term.”
Other indicators that new technology is creating problems rather than solving them include a growing reliance on asking coworkers for help with a tool, active tool avoidance, and increased time spent fixing tool outputs. Such activities may not show up directly on a productivity report, but will affect overall output and engagement.
The absence of these behaviors is equally telling. When employees adopt a tool quickly, reduce their workarounds, and stop routing around it, those are the green lights tech developers should track as carefully as download numbers.
Tom/Getty Images
Signals of poor AI adoption are hard to find, harder to act on
Most of these signals live in places organizations are not accustomed to looking. Behavioral data, informal workarounds, and shadow tool usage rarely surface in standard reporting. The gap between what leadership believes is happening with a technology deployment and what is actually happening at the employee level is often significant.
“Employees may not tell you outright that a new tech implementation is making their work harder,” Ngan said. “Instead, they’ll show through their actions whether the tech is good or not. If they’re fixing AI outputs or using unauthorized tools, you have a problem, and you’re typically not going to find out about these signals unless you do some digging.”
Monitoring and analyzing unstructured data is part of what makes this diagnostic work difficult. Internal communications, open-ended survey comments, and behavioral patterns all require more depth than a rating scale.
“Organizations need to dive deep into how their employees are using or not using the tech, what their frustrations are and what they like. Because ultimately, if you forget about the people who are at the center of the equation, the technology is going to fail,” he added.
What distinguishes technologies that last from those that don’t:
- Technologies that succeed tend to reduce friction in a worker’s day, becoming the path of least resistance rather than an obstacle to it. Those that fail tend to accumulate workarounds and become invisible in usage data precisely because employees have found ways to route around them, according to Gartner research on AI adoption patterns.
- A company that deploys the wrong tool, or deploys the right tool poorly, is not just losing the cost of that investment. It risks disrupting workflows that were functioning and eroding the trust employees place in future technology rollouts, making each subsequent adoption harder to execute successfully, Harvard Business Review indicated in its analysis of technology transformation failures.
- With the right insights gathered early, tech developers can make needed changes before their offering falls into disfavor. Few things will derail a product’s trajectory faster than failing to account for how it can better serve the end user, and few things sustain momentum more reliably than a feedback loop that reaches into actual day-to-day usage, rather than stopping at surface-level metrics, McKinsey noted.
What broader tech-adoption data tell investors, developers
The pattern is consistent across industries. Technologies that succeed tend to generate organic advocacy among the people using them. Technologies that fail tend to generate the workarounds, informal help-seeking behavior, and shadow tool usage described above.
This distinction matters more now than it did five years ago because the volume of AI tools entering the enterprise has considerably raised the stakes of a failed implementation.
Paying attention to often-overlooked workforce signals is critical, both for the companies developing new technologies and those adopting them. A workforce that does not gain meaningful benefits from a tool or platform is not going to use it in the long run.
The longer it takes companies to discover this, the more costly the transition will be. By focusing on hidden signals before they become visible problems, tech creators and users can make crucial changes to deliver greater value and avoid falling into disuse entirely.
Related: Morgan Housel thinks investors are missing something about AI