Keep your options open when it comes to public or private LLMs

Enterprise leaders are being sold generative AI (GenAI) tools on the promise of boosted productivity and more effective employees. To deliver, leaders are being asked to choose between public or private offerings when selecting a large language model (LLM). Both public and private models come with benefits and risks—but this is an extremely fast-moving landscape. Therefore, for success in the journey to The Generative Enterprise, it’s important for enterprise leaders to consider composable architectures to keep their options open.

LLMs are the engines that enable GenAI to generate novel outcomes, such as summarizations, content, and code, drawing on the data provided. Public LLMs have more comprehensive access to data sets; for example, ChatGPT has millions of users adding content and challenging it with prompts every day. Private LLMs are limited to using the data defined by their stakeholders. The enterprise might prefer to only use data it trusts—its own contracts, product documentation, or employee and customer information. For example, HFS Research, working with Humata, has deployed a private instance of an LLM across our research content. Try querying it for yourself on our website.

Public LLMs may add capabilities faster than private, thanks to the scale of adoption

The public side of the LLM argument—supported and illustrated by offerings from the likes of OpenAI, Google, and Anthropic—assumes by offering widespread access to millions of users, more prompts and more data will improve models faster. Proponents argue the rate of improvement will continue to accelerate, delivering still-to-emerge capabilities that will rapidly make private models look pedestrian and, shortly after, redundant. HFS CEO & Chief Analyst Phil Fersht points out the “10 tectonic shifts” ChatGPT 4 delivers over ChatGPT 3.5 in his recent blog post.

There are many unknowns about when and why new capabilities emerge from LLMs. However, the evidence of the journey so far is that when we add to the quantity of training data, the variety, and the number of parameters, unanticipated leaps in capability emerge.

Private LLMs deliver when security, control, and customization are important

On the private side, supporters cite control, enhanced governance, privacy, and security benefits in an argument echoing the debates over on-premises and cloud. Private models, such as those provided by Meta, IBM, and Huggingface, offer more predictability regarding accuracy, at least at this point. Stakeholders only allow models to access data sources they can validate, mainly solving the hallucination problem that can plague public models. ChatGPT’s latest (August 3) update comes with a catch-all caveat: “ChatGPT may produce inaccurate information about people, places, or facts.”

Exhibit 1 illustrates uses for which enterprises may consider private LLMs to meet their needs. Those cases tend to be where there is a need to maintain data privacy, address industry-specific requirements, or extract value from proprietary data.

Exhibit 1: Consider private LLMs where security, control, or industry-specific customization are essential

Source: HFS Research, 2023

However, not all aspects of all enterprise activities demand the level of control of a private LLM. Compared to public models, private LLMs can be costly to build and time-consuming to train, and their pace of improvement is constrained by limited access to data and lower usage. Exhibit 2 lists some leading uses for public models. You’ll note some overlap with private LLM use cases, such as customer support, IP and research, translation, and compliance. Specific aspects of each may better suit the use of a private or public LLM, as discussed in Exhibits 1 and 2.

Exhibit 2: Consider public LLMs when the scale of the data they access provides advantages

Source: HFS Research, 2023

Public LLMs get you on the GenAI fast track with an eye on emerging value

Aside from the uses described in Exhibit 2, there are benefits to gain from the fact that everyone can test and learn with a public LLM (see ChatGPT.com, for example). You don’t have to wait to set up a new tech crew capable of building your own LLM. You can start innovating to learn about the benefits today instead of making it part of your 2024 roadmap. Public LLMs meet the need for urgency that their rise in the public consciousness demands of enterprise leaders.

In examples such as “Code Interpreter” (part of GPT4), users can also upload and ask natural language questions of datasets. Likely, the most significant benefits and enterprise use cases for public LLMs are still unknown. They will emerge as the capability of public LLMs accelerates.

It’s a fast-moving space with the low-hanging fruit likely to be hoovered up by enterprise platforms

This space is developing quickly. Enterprise leaders should consider what may change in a couple of months. For example, could simple document generation be rapidly commoditized and consumed in platforms such as Salesforce and Pega?

Pega has already announced Pega GenAI, boosters built on its low-code platform. Salesforce has Einstein GPT, GenAi for CRM. Every platform enterprises use will be infused with GenAI to a greater or lesser extent, challenging the need for private LLMs. Why build your own if your partners have you covered?

Hybrid sounds like a safe bet, but it may leave you exposed

Currently, cases can be made for public and private LLMs, as Exhibits 1 and 2 illustrate. Certainly, enterprise leaders should keep their options open. The overlaps between Exhibits 1 and 2 reveal the challenges of choosing one.

Hybrid solutions are emerging, applying public and private LLMs where they make the most sense. A hybrid solution may spread your bets, but it doesn’t derisk the potential lag vs. your competitors that the slower development of private models may expose you to.

GenAI’s rapid development rate will only accelerate. The advancement of LLMs has so far scaled with their usage and the datasets they access, so it isn’t easy to conceive how private models can keep up with whatever may come next from public LLMs.

Composability keeps your LLM options open

To protect against the risk that your first-choice model is made redundant, consider a composable architecture that supports reusable components that engineers can swiftly reconfigure.

Composability in architecture is a design approach in which loosely coupled services, applications, and features act as independent building blocks in the tech stack. Based on the concept of microservices (in which applications consist of collections of independently deployable services), these building blocks communicate via APIs. The intention is to allow flexibility in the stack so you can remove elements without tearing the whole thing down.

LLM technology doesn’t lend itself to “hot swaps”—changing out one model for another without pause or impact. Models have different pre-training requirements. The mechanism that assigns scoring, commonly known as a “weight,” also varies from model to model. The many permutations of weighting and pre-training mean prompts also change between models, so some updates are inevitable if you change the model. But composability—an enabler of ongoing digital transformation—can limit which architectural layers get disrupted, reducing downtime and cost and leaving workflows intact.

The Bottom Line: The impact and possibilities of generative AI are shifting so quickly that it would be foolish to hitch your wagon to any one approach or solution.

Generative AI is turning out to be the last place you would want to suffer from vendor lock-in. Flexibility is going to be the name of the game as we strap in for a wild ride. It’s a fractured market currently, engaged in an arms race of capabilities. Consolidation will follow, but until it does, enterprise leaders must do all they can to keep their options open while managing costs.

In further reading, find more about how GenAI works and enterprise domains and use cases for it in our recent POV, How business leaders can take control of the GenAI conversation.