How to choose the right LLM for your business challenge

Large Language Models (LLMs) are driving disruption across every industry. Yet such is the confusion over what each is capable of and what it will cost to implement, run, and customize them that many enterprise leaders throw their hands up and go with whichever LLM happens to be in the same stable as the partners providing their current tech stack.

Current partner tech may be a reasonable place to start. Still, we believe LLMs will rapidly become too crucial to your competitive advantage to rely on tech choices made for you by trusted relationships formed in the pre-generative-AI era. Current suppliers and partners may have you covered, but to meet your specific needs, be prepared to look further, and don’t expect one technology to solve every problem.

Start by placing your business requirements ahead of technology.

We have short-listed the LLMs best positioned to solve 20 business challenges

To help you take your first steps, we compiled the following (see Exhibit 1) with the support of ChatGPT 4o in May 2024. The selection of ‘leading contenders’ vs. each use case is based on:

Performance and accuracy: How well the model performs in understanding and generating text relevant to the specific use case.

Adoption and popularity: The extent to which enterprises adopt and trust the model, as indicated by its widespread usage and industry recognition.

Versatility and flexibility: The model’s ability to handle various tasks and adapt to different scenarios and industries.

Integration and deployment: Ease of integration with existing systems, deployment options (cloud, on-premises, hybrid), and available tools and APIs.

Cost and accessibility: The cost-effectiveness of the model and its accessibility for businesses of different sizes.

Support and ecosystem: The level of support the vendor provides, including documentation, community support, and additional resources.
Sources are drawn from across the web. HFS caveats that this is our starting point in matching LLM capabilities to enterprise needs. We note that this is a fast-moving environment that is subject to change. In the coming weeks and months of this ongoing GenAI revolution, we are committed to updating our understanding of the ability of these emerging technologies to solve enterprise challenges and to sharing our insights to help enterprise leaders find their way, including understanding vertical specializations as they emerge.

Exhibit 1: Some LLMs are better for solving specific business challenges than others. One size does not fit all.

Source: HFS Research 2024. Sources of analysis include Zapier: The Best Large Language Models (LLMs) in 2024; Revelo: Top 15 Large Language Models in 2024 ; Signity Solutions: Top 15 Large Language Models in 2024; Wire19: Top 10 Large Language Model (LLM) vendors to look out for in 2024

Before you get technical, start with business goals and specific use cases

Five key steps will define which LLM is right for you: business alignment, technical capabilities, data & compliance, cost & ROI, and support and ethical considerations.

Business alignment: Start with the business goals you want an LLM to help you address. Be specific about the use cases. See Exhibit 1 for an initial steer. For example, if you want support for your sales teams, build a consideration set starting with Salesforce Einstein, OpenAI GPT-4, and Google Dialogflow.

From this, you can define KPIs to compare the effectiveness of one LLM to another in your specific use case. Example KPIs include accuracy, response time, cost per customer interaction, etc. While setting KPIs for the LLM, also set KPIs for the outcomes—such as customer satisfaction.

Technical capabilities: Does the LLM speak your language? Can it understand the English, Urdu, or Cantonese you may use, as well as the domain specifics and turns-of-phrase for your use case?

Can you easily integrate the LLM with your existing systems, and can it match your cloud or on-premises preferences? And what capabilities can the LLM deliver when pushed beyond POC into mass market adoption at scale—can it meet your scaled demands and still deliver consistent performance?

Data & compliance: Garbage-in: Garbage-out has never been more accurate than in the case of LLMs. GenAI works by generating new outcomes from the old. If the old sources are outdated or just plain wrong, your chances of success with GenAI are minimal. So, assess the availability and quality of the data available to the LLM for your specific use case. Is there enough, and is it of sufficient quality? What fine-tuning will be required? Will you need to train it with your own data?

And, of course, you must also factor in data privacy and security compliance needs. Does the LLM make this easier for you?

Cost & ROI: Ensure you know all the costs associated with your potential LLM. It starts with licensing, but what do the costs look like at peak use? Will you need to constrain use by controlling who can prompt and how they prompt? Could that be applied to external demand (customers) as much as to internal demand (employees)? You need a total cost of ownership to include computational resources, maintenance, and customization.

On the returns side, calculate how and where you are saving, but also map the role the LLM can play in revenue growth. This is particularly relevant to applying the LLM to support a new business model.

Support and ethical considerations: When choosing your LLM, understand how the model will be supported going forward. Is a community of experts and users already helping shape its improvement program and accelerating a proliferation of use cases from which to learn? What does the ecosystem around it look like, and does it match your ambitions?

Regarding ethical AI, look out for models that promote fairness and transparency. How does it manage biases, and can it provide explainable outputs? The approach to ethics is becoming an essential part of emerging legislation that will govern how you can use AI in general and LLMs in particular.

Validate your assumptions with demos and trials

Of course, running POCs and pilots is essential to validate assumptions, measure performance, and get greater clarity on costs before committing to long-term investments.

Most vendors will offer demos and trials when you enter more detailed discussions. Getting these trials free of charge is a real possibility where the vendors smell a strong future business opportunity.

The Bottom Line: Enterprise leaders should take control of their digital destiny by focusing on matching LLMs to business goals and specific use cases

Enterprise leaders must take control of their digital destiny and lead the decision-making on LLMs, focusing on business goals and requirements and applicability to specific business use cases. That’s right—it’s business first, tech second.

*If you are unfamiliar with LLMs and GenAI and how they relate to the broader AI ecosystem, use our primer: The HFS definition of enterprise AI.