Is the singularity nigh? Deconstructing the ChatGPT hype

The singularity—when artificial intelligence (AI) becomes smarter than humans—must be nigh. The OpenAI ChatGPT project is coming for journalists, Google Search, and (gulp) analysts with its chatbot capabilities that can answer questions, tell stories, and write code.

Well, maybe. Let’s not forget the platform is limited to the input the algorithm was trained on. Despite initial results that got us all reaching for hyperbole, has this pony really got so many tricks that we should start laying off our knowledge workers?

ChatGPT gives us a glimpse of what progress with AI could look like

We are not computer scientists taking part in deep discussions about large language models (LLM) like GPT-3 (which stands for Generative Pretrained Transformer 3). Instead, we are analysts considering the enterprise implications amidst the hype and speculative, fun, as well as sometimes bewildering reactions to ChatGPT.

We asked the chatbot what it was, as shown in Exhibit 1. Its response says it was

…trained using a machine learning technique called unsupervised learning, which means that it is trained to generate text by predicting the next words in a sequence based on the ones that come before it, without the need for human-provided labels or annotations. This allows GPT-3 to generate text that is often indistinguishable from text written by a human. GPT-3 is considered to be one of the most advanced language processing models available, with many applications in natural language processing tasks such as language translation, text summarization, and question answering.

— ChatGPT, December 2022

The seductive nature of ChatGPT lies in the perception that it can write a poem, generate code, write college-level essays, and offer life advice. Yet, as Exhibit 1 outlines OpenAI has been at pains to teach the model in a way that highlights the limitation of the results:

Exhibit 1: ChatGPT in deep thought mode

Image and data source: https://chat.openai.com/chat
Source: HFS Research, 2022

For all its stunning early success with ChatGPT, OpenAI is not alone. All the major independent software vendors are investing heavily in similar models. Google has LaMDA, Microsoft has Godel, Meta has Galactica, and the list goes on. And there are already warning signs. Meta pulled Galactica just days after launching it amid suggestions of racist and dangerous information. In this regard, OpenAI’s CEO Sam Altman was extremely transparent by tweeting after the launch of ChatGPT:

ChatGPT is incredibly limited but good enough at some things to create a misleading impression of greatness. it’s a mistake to be relying on it for anything important right now. it’s a preview of progress; we have lots of work to do on robustness and truthfulness.

— Sam Altman, CEO, OpenAI, 10 December 2022

We must view this only as a sneak preview of fascinating technology capability. It gives us a glimpse of what progress with AI could look like.

Contextualizing AI breakthroughs—enterprises ain’t no mobile phones

Breakthroughs in AI tend to be demonstrated by games or highly accessible use cases: IBM’s win at Jeopardy, Google’s DeepMind beating the best Go player, and Arago winning at Civilization. Those games underlined the complexity of options and the lack of deterministic decisions. Or, take Google’s Duplex, which made an appointment at a hairdresser in natural language. Or IBM’s Debater, an AI system that can debate humans on complex topics. Yet, tellingly, those breakthroughs have not (yet) disrupted any industry. They lack enterprise-grade capabilities in terms of integration, governance, and ethics. An enterprise organization is not a mobile phone, and it is not a closed system. Enterprises must react to constant change. Thus, a pretrained data model is not enough to drive operational change.

So, what could an evolution of GPT-3 drive? ChatGPT might not disrupt the contact center exactly because of this lack of enterprise-grade capabilities, but it might impact a broad array of activities.

You get a use case! You get a use case! Everybody gets a use case!

Everyone – everyone – is finding valuable use cases with ChatGPT: completing university assignments, writing movie scripts, generating queries in Python, and building applications.

Talking about a lifesaving use case. A video taken by a doctor based in the US is circulating on the internet, showing how the doctor is leveraging ChatGPT to save time with insurance denials. One of the common reasons for denials is related to medical necessity. In many cases, the health insurer believes the requested services are not medically necessary because the doctor hasn’t provided enough convincing information about the case. A doctor is now using ChatGPT to write letters and gather enough supporting information, including providing references to scientific literature and listing appropriate articles, to ensure that the requested services will be approved without any delay.

What’s the outcome here? Get the patients the care they need on time and potentially save lives.

What really counts is how employees can do their jobs most efficiently to better serve their customers. ChatGPT can be used to enhance the productivity and capability of just about any profession.

Governance, accountability, and ethics will curtail early enterprise uptake

If ChatGPT can claim a role as some kind of better-than-ever knowledge management tool, it must first show its outputs can be validated and trusted. But all large language models are renowned for what can be called “response hallucination,” generating text that seems very credible and coherent but is factually false. And what about ethics here? Harvesting and scraping knowledge to generate long and elaborated answers without consistently referencing verified data sources is a debatable approach. We used to call it plagiarism.

Just think about the new ammunition the crime-as-a-service (CaaS) industry will now get thanks to ChatGPT, from writing advanced phishing emails to generating creative malicious code. Enterprises must address a wide range of security concerns before adoption begins. Take the example of Stack Overflow, the public platform nearly everyone who codes uses, which has provisionally banned the use of text generated from ChatGPT. While the temporary ban focuses on answers, it has also been extended to all content on Stack Overflow, including generated code. There is a risk of bad actors using ChatGPT in an industrialized manner to produce malicious code at speed and scale. There is also a risk of legitimate actors potentially generating code with security flaws that bad actors can subsequently exploit. OpenAI researchers mentioned that while “future code generation models may be able to be trained to produce more secure code than the average developer,” getting there “is far from certain.”

All the answers—but none of the accountability of traditional search

Many suggest that ChatGPT could supplant Google Search. Both could potentially be the “source” of the origination and transmission of information bias, which can ultimately result in bad outcomes. But there is a significant difference between the two: accountability. Google is a store of information, not the information it generates for itself. Google users can trace an information source to an accountable entity (whether genuine or malicious). ChatGPT is a channel for information, with self-generated information, implying intrinsic accountability regardless of the sources. That being said, we could see a hybrid approach with the integration of a ‘Google’ with hyperlinks to the computer-generated ChatGPT results. Thus, the wide-scale adoption of LLMs must be accompanied by a debate around what “accountability” means, especially regarding enterprise adoption.

Thus, as with all social media platforms, much public perception will hinge on how ChatGPT is governed and how its content is moderated. Elon Musk’s takeover of Twitter is playing out all those issues in front of our eyes, with uncertain outcomes. ChatGPT is just the beginning of a fundamentally different way to disseminate information.

The Bottom Line: OpenAI’s progress with ChatGPT is stupendous, but concerns about governance and ethics outweigh the advances.

Looking at and playing with ChatGPT is superficially intriguing. We get a sense of the progress of complex AI models. Yet, the crucial point is not AI’s advance but information dissemination and analysis. In a polarized world hosting parallel universes of facts, we would do well to recall Orwell’s seminal novel 1984, where the Thought Police control us by controlling the words we use. It’s a stark reminder that we must spend as much time on information governance and ethics as we do on advancing AI.