AI can transform business, but real-world failures at companies like Air Canada, Zillow, Samsung, CNET and IBM show how quickly things can go wrong.
Adobe Stock
AI is already reshaping business, but it’s also exposing some very expensive vulnerabilities.
Despite the excitement surrounding genetic AI, automation and intelligent decision-making, technology is only as good as the strategy, governance and human judgment behind it. When these things are missing, AI can damage customer trust, leak confidential data, create legal headaches, and turn small mistakes into very large bills.
Here are five real-world AI mistakes and the lessons every entrepreneur should learn from them.
Air Canada: Illusion Chatbots
In 2024, a Canadian court ordered Air Canada to pay damages when a chatbot, built into its online reservation system, hallucinated an imaginary discount. The robot according to information misadvised a fare discount to a passenger traveling for his grandmother’s funeral, assuring the passenger that he could pay the full fare and apply for the discount retroactively.
This turned out to be against company policy and Air Canada refused to honor the discount. However, the court ruled that it had to pay the passenger $812.02 due to the chatbot’s error.
The lesson, summed up by Air Passenger Rights president Gabor Lukacs, is “If you hand over part of your business to artificial intelligence, you’re responsible for what it does.”
Zillow: Machine Learning Miscalculations
When real estate services specialist Zillow used machine learning to build a tool to automatically buy homes and flip them for profit, the results weren’t what they expected. His algorithmic model, designed to find optimal buy and sell prices to maximize trading profits, proved unable to accurately predict the chaotic behavior of the real estate market. This led to overpayments as a result Losses of $500 million.
Eventually, an entire department was shut down and Zillow wrote the project off as an expensive course. AI mistakes escalate quickly, and tiny miscalculations or “rounding errors” can quickly escalate into major disasters if left unchecked.
Samsung: Governance failure
Samsung has been forced to severely curtail its workforce’s use of AI production tools after finding staff uploading confidential company information. Anything entered into cloud-based AI chatbots like ChatGPT can potentially be seen by human operators and used to further train the AI. Simply put, what happens to that data is completely out of the company’s control. In Samsung’s case, this highlighted a serious lack of governance over how AI is used. Unfortunately, this is still the case with many companies today, where shadow AI is spreading as employees use unapproved tools because they are quick and useful, while unclear guidelines and policies leave employees unsure of what AI should or shouldn’t be used for. Make sure your company is not one of them.
CNET: Poor human oversight
In journalism, trust between readers and publishers is critical, and tech news agency CNET put a dent in theirs with AI-generated articles. Complaints of inaccuracies surfaced after it began including AI explainers alongside features and reviews. An investigation then found errors in 41 of the 77 AI-generated articles. In addition to the loss of trust, the need for human authors to spend a lot of time publishing lengthy fixes no doubt added to the damage, the total cost of which is unknown. The lesson here is that robust processes must be in place to ensure that AI content is subject to human control and oversight.
IBM: Hype vs Reality Mismatch
IBM’s Watson Health platform serves as a warning to those who might get too excited about unproven capabilities. IBM spent billions building and marketing its artificial intelligence for healthcare, and while expectations were high, the results fell short. The technology delivered inconsistent results, adoption stalled and trust evaporated. IBM eventually sold Watson Health, which at one point had 7,000 employees, and learned that waiting to verify results before declaring your product ready to buy is probably a good idea.
Despite accelerating efforts to regulate, guide and responsibly practice AI, I’m sure we’ll see more companies making serious AI-related mistakes in the near future. Sometimes companies will make mistakes because they rush into AI too quickly for fear of being left behind, and sometimes it will be because organizations are not aligned from the top down on AI governance and oversight.
Understanding the common causes of mistakes and learning from those who have made them in the past will help companies be prepared to avoid them or minimize the impact damage when they cannot.
