AI promised a revolution, companies are still waiting

Last spring, CellarTracker, a wine-collection app, built an AI-powered sommelier to make unvarnished wine recommendations based on a person's palate. The problem was the chatbot was too nice.

"It's just very polite, instead of just saying, 'It's really unlikely you'll like the wine,'" CellarTracker CEO Eric LeVine said. It took six weeks of trial and error to coax the chatbot into offering an honest appraisal before the feature was launched.

Since ChatGPT exploded three years ago, companies big and small have leapt at the chance to adopt generative artificial intelligence (AI) and stuff it into as many products as possible. But so far, the vast majority of businesses are struggling to realize a meaningful return on their AI investments, according to company executives, advisors and the results of seven recent executive and worker surveys.

One survey of 1,576 executives conducted during the second quarter by research and advisory firm Forrester Research showed just 15 percent of respondents saw profit margins improve due to AI over the last year. Consulting firm BCG found that only 5 percent of 1,250 executives surveyed between May and mid-July saw widespread value from AI.

Executives say they still believe generative AI will eventually transform their businesses, but they are reconsidering how quickly that will happen within their organizations.

One well-known issue with AI models is their tendency to please the user. This bias – what's called "sycophancy" – encourages users to chat more, but can impair the model's ability to give better advice.

CellarTracker ran into this problem with its wine-recommendation feature, built on top of OpenAI's technology, CEO LeVine said. The chatbot performed well enough when asked for general recommendations. But when asked about specific vintages, the chatbot remained positive – even if all signals showed a person was highly unlikely to enjoy them. Part of the solution was designing prompts that gave the model permission to say no.

Companies have also struggled with AI's lack of consistency.

Jeremy Nielsen, general manager at North American railroad service provider Cando Rail and Terminals, said the company recently tested an AI chatbot for employees to study internal safety reports and training materials.

But Cando ran into a surprising stumbling block: the models couldn't consistently and correctly summarize the Canadian Rail Operating Rules, a roughly 100-page document that lays out the safety standards for the industry.

Sometimes the models forgot or misinterpreted the rules; other times they invented them from whole cloth. AI researchers say models often struggle to recall what appears in the middle of a long document. Cando has dropped the project for now, but is testing other ideas.

Seemingly small issues can unexpectedly trip up AI systems.

Human-staffed call centers and customer service were supposed to be heavily disrupted by AI, but companies quickly learned there are limits to the amount of human interaction that can be delegated to chatbots.

In early 2024, Swedish payments company Klarna rolled out an OpenAI-powered customer service agent that it said could do the work of 700 full-time customer service agents.

In 2025, however, CEO Sebastian Siemiathowski was forced to dial that back and acknowledge that some customers preferred to talk with humans.

Siemiathowski said AI is reliable on simple tasks and can now do the work of about 850 agents, but more complex issues quickly get referred to human agents.

Large language models are rapidly conquering complex tasks in math and coding, but can still fail at comparatively trivial tasks. Researchers call this contradiction in capabilities the "jagged frontier" of AI.

"Companies need more handholding in actually making AI tools useful for them," said May Habib, CEO of Writer, an AI application startup.