The dos and don’ts and potential costs of GenAI

CEOs are under intense pressure to adopt generative artificial intelligence (GenAI) as their boards and shareholders look to them to solve the digital dichotomy. We’ve all lived through disruptive forces, but it’s the sheer rate of acceleration of the capabilities, seen in examples such as ChatGPT and Midjourney, keeping CEOs up at night.

To ease your sleep, we have identified what you should and shouldn’t do when determining how to integrate GenAI into your business—and how much you can expect to pay to use it. Yes, pay. For example, many of us have kicked the tires on ChatGPT in a free version. But ChatGPT costs $700,000 a day to run. Someone has to pay, and it’s going to be the enterprise.

What you should not do with GenAI

IBM Consulting’s response to the GenAI hype builds on the firm’s reputation in AI and analytics, placing it among the pacesetters. HFS believes IBM’s Watson has a new lease of life in the watsonx platform, which HFS identified as potentially becoming “the beating heart of the Generative Enterprise™.”

HFS agrees with IBM’s position that most structured data analysis, prediction, and prescription tasks are better served with traditional AI. These include analysis that leads to forecasting and predicting performance, such as predictive maintenance. But it also means that traditional AI is a better fit for directed conversational AI, where deterministic dialog flows for API-driven conversational AI are deployed, such as in many chatbots; computer vision AI, where machine vision is used for object and anomaly detection, such as on a production line; and in robotic process automation (RPA), for process reengineering and optimization.

What you should do with GenAI

The clue for the best candidates for GenAI is in its name: those that generate text, images, videos, or code. IBM Consulting, which partners with an ecosystem that includes IBM watsonx, OpenAI, ChatGPT, Microsoft, Salesforce, AWS, Google, Meta, and Open Source to deliver its portfolio of GenAI capabilities, says GenAI can augment existing traditional AI use cases to improve natural language interactions and summarize large volumes of data.

The four broad categories for these enhanced AI capabilities include

• Summarization: Capabilities include gathering data from various sources to generate a summary, such as call-center interactions, financial reports, analyst articles (note: not writing them, summarizing from a range of them!), and media trends in the news.
• Conversational knowledge: Enabling bots to respond to questions, drawing from reviews, product descriptions, and catalogs.
• Content creation: GenAI can generate personas and user stories; personalize marketing copy, images, and emails; and provide other content.
• Code creation: GenAI can generate code in response to design prompts, convert code from one platform to another, create technical documentation, and generate test cases.

Domain and industry use cases you should be considering now

At its European analyst event, IBM Consulting offered a range of domain and industry enterprise use cases that the four broad use cases described above could support (see Exhibit 1). These don’t include some operational enterprise use cases such as disruption in payroll services (one example EY has its managed services eyes on), management training, and other employee-experience use cases, such as legal, procurement, and logistics planning.

Exhibit 1: GenAI enterprise use cases available across domains and industries

Source: IBM Consulting and HFS Research, 2023

BFS and telcos already see benefits

IBM is already claiming success for its application of GenAI in a few confidential client use cases. It has recorded 91% accuracy with near real-time insights into customer complaints at a global payments firm; a 90% reduction in time to insights from agent conversations leading to the identification of operational improvements worth more than $20 million at a global telco; and 30,000 hours saved across more than 5,000 controls for regulatory compliance at a large global bank, achieved by analyzing control documents at scale.

What GenAI is going to cost you

Pricing is something of a dark art. GenAI is not well suited to effort-related pricing since much of the “sell” of GenAI is in its ability to get to outcomes quickly and support rapid insight for decision making. These factors will likely accelerate the trend toward outcome-based pricing. IBM’s analyst event did not include pricing coverage.

HFS notes that enterprise leaders must expect their service provider partners to price-in effort-related initial strategy, solution architecting, and maintenance, but all parties will also need to know the cost of GenAI’s “raw materials”—using the GPT model required to deliver their desired outcomes.

Both prompts and responses cost tokens, meaning every piece of text input and output has a dollar cost. Each language model is priced differently. Code-Cushman (one of OpenAI’s codex models, and yes, there is such a thing), for example, was priced at $0.024 per 1,000 tokens when we checked on the Microsoft Azure site on June 19, 2023. One token pays for roughly four English characters to be processed. And this is where you may need to engage your service provider. Thanks to partnerships and buying in bulk, they may get a better deal on the cost-per-thousand tokens than individual enterprises can achieve. Those who have gone through cloud transformations can be forgiven for thinking, “Uh-oh, here we go again…” and being wary of proceeding without clarity on what it will cost.

HFS conversations with those working with live projects suggest prices are falling, and we expect the costs to continue to fall while the complexity of the tasks GenAI can handle will continue to rise. One consulting leader told us: “It’s going to handle tasks that no one is currently even thinking about offshoring.” As HFS’ CEO and Chief Analyst Phil Fersht has pointed out, even greedy, lazy lawyers are under threat.

The Bottom Line: Place GenAI bets where they “generate” best-fit returns, and be clear about how costs may scale.

CEOs must place GenAI bets to take advantage of its specific generative capabilities, such as those listed in Exhibit 1. To avoid cloud-transformation-style surprises when the bill comes, work with your service provider to clarify how pricing is generated, which elements can be fixed, and which will scale with volume.