Gartner Identifies Four Emerging Challenges to Delivering Value from AI Safely and at Scale

Analysts Explore How CIOs are Leading Today and Shaping Tomorrow During Gartner IT Symposium/Xpo 2024, October 21-24

Gartner Identifies Four Emerging Challenges to Delivering Value from AI Safely and at Scale

Matt LoDolce
Gartner
matt.lodolce@gartner.com

Catherine Howley
Gartner
catherine.howley@gartner.com

A Gartner, Inc. survey of 451 senior technology leaders in the second quarter of 2024, found that 57% of CIOs said they are tasked with leading an AI strategy in their enterprise. However, four emerging challenges are making it difficult for CIOs to deliver value with AI.

“Because of the relentless innovation happening in the tech vendor race, CIOs feel like they are always living the hype, while the reality of their AI outcomes race - how tough it is to get value - makes it feel like they are also in the trough,” said Mary Mesaglio, Distinguished VP Analyst at Gartner.

“However, CIOs can set the pace in their AI outcomes race,” said Hung LeHong, Distinguished VP Analyst and Gartner Fellow. “If you have modest AI ambitions, in an industry that isn’t being remastered by AI yet, you can afford to go at a more measured pace. This is an AI-steady pace. For those organizations with bigger AI ambitions, or in an industry that’s being reinvented by AI, the pace will be faster. This is an AI-accelerated pace. Whether you’re moving at an AI-steady or AI-accelerated pace, you have to deliver value and outcomes.”

During the opening keynote of Gartner IT Symposium/Xpo, which is taking place here through Thursday, Gartner analysts explained to the audience of over 8,000 CIOs and IT executives how to overcome four emerging challenges to deliver value from AI safely and at scale.

The Business Benefits of Using AI Don’t Always Materialize

To generate business value with generative AI (GenAI), people must consistently use GenAI tools in their workflow. In a second quarter 2024 Gartner survey of over 5,000 digital workers in the U.S., UK, India, Australia and China, employees said that they saved an average of 3.6 hours per week by using GenAI. But not all employees get the same degree of benefit from using GenAI.

“Here’s the real challenge with AI productivity,” said LeHong. “Productivity gains from GenAI are not equally distributed. Gains vary by employee, not just because of their personal interest and levels of adoptions, but according to complexity of job and level of experience.”

AI-accelerated organizations are also looking at benefits beyond productivity - operations and process-level improvements, such as automating key business processes or redesigning roles to work with chatbots; and business-level, game-changing improvements, such as outcomes that create new revenue streams or redesign the enterprise value proposition.

“In these cases, CIOs should manage AI benefits like a portfolio. Determine the size of your bet in each benefit area, and manage risks and rewards across this portfolio,” said Mesaglio.

The Cost of AI Can Quickly Spiral Out of Control

More than 90% of CIOs said that managing cost limits their ability to get value from AI for their enterprise, according to a Gartner survey of over 300 CIOs in June and July 2024. In fact, Gartner believes that cost is as big an AI risk as security or hallucinations.

If CIOs don’t understand how their GenAI costs scale, Gartner estimates that they could make a 500%-1,000% error in their cost calculations.

“As a CIO, you need to understand your AI bill,” said LeHong. “You must understand the cost components and pricing model options, and you need to know how to reduce these costs and negotiate with vendors. CIOs should create proofs of concept that test how costs will scale, not just how the technology works.”

Data and AI Everywhere Creates New Challenges and Risks

With AI and data proliferating everywhere in the enterprise, AI and data are no longer centralized assets that IT directly controls. The Gartner survey of over 300 CIOs found that on average, only 35% of their AI capabilities will be built by their IT teams. This means that new approaches are needed to manage and protect data access and govern AI inputs and outputs and safely deliver AI value.

“This is where the concept of a ‘tech sandwich’ comes in,” said LeHong in describing the AI tech stack of the future. “On the bottom of the sandwich is all the data and AI from IT, typically centralized. On the top is all the data and AI coming from everywhere, typically decentralized. And the middle contains the trust, risk, and security management (TRiSM) technologies that make it all safe. It’s what you need to create to accommodate AI and data coming from everywhere.”

“As CIO, your job is to design a tech sandwich that can handle the messiness of AI, but still keeps you open to new opportunities,” said Mesaglio. “AI-steady organizations (ten AI initiatives or fewer) will govern their tech sandwiches using human teams and committees. AI-accelerated organizations will add TRiSM technologies - a set of technologies designed to create trust, monitor risk and manage security for safe AI at scale.”

Using AI Can Both Positively and Negatively impact Employees’ Performance and Well-Being

Some employees may feel a strong affinity for AI. Others may feel threatened or resentful. These intense reactions to AI can lead to unintended behavioral outcomes that negatively impact employees’ work performance, such as jealousy of those using AI and overdependence on AI tools.

However, few organizations are actively managing these behavioral outcomes. In the June/July Gartner survey, only 20% of CIOs said they focus on mitigating potential negative impacts of GenAI on employee well-being.

“Most enterprises aren’t curious enough about how AI makes their employees feel. This matters because AI can lead to all sorts of unintended behavioral outcomes,” said Mesaglio. “The critical point is that if you use change management to manage this, be intentional about who owns which behavioral outcomes. Organizations must manage behavioral outcomes with the same rigor as technology and business outcomes.”

Gartner clients can learn more in “2024 IT Symposium/Xpo Keynote at a Glance.”

About Gartner IT Symposium/Xpo

Gartner IT Symposium/Xpo is the world's most important gathering for CIOs and other IT executives. IT executives rely on these conferences to learn how to amplify the impact of the technology, insights and trends shaping the future of IT and business. Follow news and updates from the conferences on X using #GartnerSYM and on the Gartner Newsroom.

Upcoming dates and locations for Gartner IT Symposium/Xpo include:
October 28-30 2024 | Tokyo, Japan
November 4-7 2024 | Barcelona, Spain
November 11-13 2024 | Kochi, India

About Gartner for Information Technology Executives

Gartner for Information Technology Executives provides actionable, objective insight to CIOs and IT leaders to help them drive their organizations through digital transformation and lead business growth. Additional information is available at www.gartner.com/en/information-technology.

Follow news and updates from Gartner for IT Executives on X and LinkedIn using #GartnerIT. Visit the IT Newsroom for more information and insights.

About Gartner

Gartner, Inc. (NYSE: IT) delivers actionable, objective insight that drives smarter decisions and stronger performance on an organization’s mission-critical priorities. To learn more, visit gartner.com.