Amazon's ai snafus echo industry woes amid rapid adoption
Amazon's recent spate of ai-induced disruptions, including a near-loss of 120,000 orders due to its coding ai tool, highlights the precarious balance companies are striking as they rapidly adopt the technology.
Ai errors multiply as firms race to leverage its power
From inadvertently gifting event tickets to coding catastrophe, ai mishaps are increasingly common as companies push the boundaries of what the technology can do.
Amazon's misstep is likely just the tip of the iceberg, experts warn, as the rapid pace of ai adoption outstrips the development of robust safeguards against errors.
“You have to know your tolerance for risk,” said Matt Rosenbaum, principal researcher at The Conference Board. “And you have to know what to do if things go wrong and how to change them so they don't happen again.”
One key challenge is that developers are now expected to review more code generated by ai than they write themselves, a skillset shift that many struggle with, said Todd Olson, CEO and co-founder of AI-powered startup Pendo.
“Those are very different skills and habits,” Olson noted. “Now, a lot of the work developers do is reviewing code written by AI.”
Another issue is that AI can churn out code at lightning speed, tempting overworked staff to expedite the process by accepting the results without fully vetting them, a recipe for further errors.
Approximately two-thirds of workers have accepted AI-generated production without careful review, and 72% have put less effort into their tasks due to AI, according to a global study by KPMG and the University of Melbourne.
“The lesson companies are learning is that speed without disciplined analysis can create systemic exposure,” said Lauren Buitta, founder and CEO of Girl Security, an organization preparing young women for careers in national security.
As AI capabilities rapidly expand, employees may experiment with its limits without fully understanding the downstream consequences. “Just because you can do something doesn't mean you should,” said Kevin Serwatka, founder of recruitment intelligence platform Benchmarket, who previously held leadership roles at companies like Google, Meta, and Robinhood.
Ultimately, experts agree, it's up to companies to establish clear boundaries on how AI is used within their organizations, learning from mistakes rather than letting them derail progress. “The small missteps are really good,” said Andrew Filev, founder and CEO of code generation firm Zencoder. “You want to identify and address them internally rather than exposing them to customers.”
Filev emphasized the importance of workers speaking up about AI errors and the need to begin AI autonomy with a combination of human and AI audits, with both processes running in parallel until AI review matches human standards.