Blog > Trending and new >

The risk of relying on LLM and generativ...

The risk of relying on LLM and generative AI alone to produce ROI (and how to avoid it)

Trending and new

9 min read

25th Nov 2024

Natalie Smithson

AI enthusiast | Tea addict | Focused on using AI assistants to win the working week

With the help of LLMs, businesses today are changing the way they operate in more drastic ways than any of us have seen in decades, but will this new technology give you positive ROI? Generative AI and the LLM technology behind it may seem like magic, but don’t be fooled. There’s a lot of hard work and technology know-how going into making these systems work well.

As exciting as it’s been for business leaders to have direct access to LLMs like GPT, Claude, Perplexity and LLaMA; to build custom GPTs, get copilots, and experiment with ‘no-code’ or open-source software, it’s important to know what’s worth serious monetary investment and your time for long-term, sustainable growth.

Find out how to apply AI directly and effectively to your unique business goals to avoid being one of the 30% of GenAI projects doomed to fail next year. There’s a clear path to success, once you know what direction to take.

Jump to section

How do LLMs help businesses? Holding your business accountable for AI Tech, tools and tactics you need alongside LLM Keep your company AI in safe hands Frequently asked questions (FAQs)

TL;DR

LLMs gift us incredible tools to make remarkable progress in how we operate and perform tasks, but don’t hold power on their own
Using LLMs alone without strict guardrails in place, or effective NLP, API integration, RAG, and a human in the loop, leaves you at risk of delivering a poor experience and AI getting things wrong
There are 13 crucial elements (and more) to consider alongside use of LLMs to make AI automation successful for your company and to generate ROI
ROI comes from having clear goals, organised data, appropriate APIs, deep AI capability including NLP, robust security, compliance and privacy handling, testing, reporting, consistent messaging, ability to scale, and much more
LLMs can’t learn or instinctively know what action to take or how to apply context to solve a business problem; they’re the soloist without an orchestra until you get everything else working alongside them to produce the right sound

How do LLMs help businesses?

It’s vital to recognise LLMs are transforming workflows including those for customer support and how we do business in general (so long as they’re introduced safely). They gift us incredible tools to make the most remarkable progress in how we operate and perform tasks to improve service and provision. LLM and generative AI is not a villain here.

LLMs are used by next-gen AI assistants to recognise and make sense of information to pass between people and follow processes, but an LLM doesn’t hold any power for enterprise on its own. Let it try and, instead of turning your business into a more efficient, well-oiled machine, you could put a spanner in the works ― or worse, break your reputation altogether.

Holding your business accountable for AI

It’s not only highly regulated industries like insurance and finance that need enterprise-grade security and robust data management. It’s every business that cares about their brand reputation, their customers, their teams, their profits, and even about protecting their brand voice.

Using LLMs alone without strict guardrails in place, or automation without effective NLP, API integration, RAG, and a human in the loop, can leave you at risk of not only delivering a subpar automation experience, but sending out incorrect or inconsistent responses too. If all you have is LLM, you’re the one responsible for making things right if things go wrong, whether it’s a technical, communication, or ethical issue at fault.

Without technical expertise or experience working with advanced AI, you’ll struggle to fix all the things that can negatively impact performance and to maximise ROI without the right tools and support.

Tech, tools and tactics you need alongside LLM

LLM is only one part of successful AI automation. A talented musician contributes to the music, but a full symphony includes multiple instruments, a conductor, sheet music, and a venue.

Who composed the music?
Who’s conducting the orchestra?
Who coordinated the rehearsal schedules?
How harmoniously do the musicians play together?

Like a soloist without an orchestra, an LLM is nothing without wider AI infrastructure and the tools you use to pull everything together. The people you put your trust in matters greatly, too.

You need everything you see in the list below and so much more, as well as ongoing management and instruction to keep your AI systems operating at peak performance. It’s this comprehensive make-up of technology, style, and approach to AI that delivers the return on your investment.

How to generate ROI from AI

Let’s consider just 13 of all the crucial elements you need to consider, alongside LLM capability, to make AI automation successful for your company, including functional and non-functional aspects:

1. Clear business objectives

First and foremost, what are you using AI for? What should it achieve for your business? Launching an AI assistant without this figured out from day one is like launching a marketing campaign with no target audience. It’ll reach and convert no-one.
– Research customer experience and your need for services
– Know what metrics you want to improve
– Set your Objectives & Key Results

2. Organised data

You’ll need to find, check, and organise your data stores (customer records, company processes, manuals, and marketing, sales, finance or customer service documents), then decide where you’ll store them for an AI assistant to access them for different tasks, and how (via a database or API, etc.). Today’s AI agents also use RAG to retrieve highly specific information about your processes and business from such sources, so they need to be accessible and accurate if you want AI responses to be right and also remove hallucinations, which LLMs are prone to if you don’t securely introduce them.

Consider too how you’ll keep this information up to date, whether your AI assistant will update documents via a team-working platform like Microsoft Teams or upload them directly into your AI platform. AI is clever but can only learn from and share what you feed it, so knowledge management systems that determine what process you use for which piece of data to train your AI effectively need to be mapped out and followed to the letter to avoid errors.

3. Appropriate API setup

An AI agent isn’t able to complete tasks or end-to-end journeys for people without being integrated with all the go-to apps and systems your teams use already. This integration is often facilitated by APIs and to use them, you’ll have to:
– Catalogue all the APIs your AI agents will interact with
– Understand API protocols (REST, SOAP) and data contracts
– Find or build OpenAPI specs for the APIs
– Understand what data and in what format the API expects and responds with
– Implement OAuth, API keys, or other security measures
– Define strategies for API failures and exception management
– Consider API Rate Limits and plan accordingly for any limitations
– Some APIs also require working with multiple endpoints and calls to carry out a task or transaction

4. Deep AI capability

Successful AI agents today must be able to recognise and understand requests and commands without human help by identifying key information and context. This means your AI agents need NLP, intent recognition, entity extraction, and context management, and you must understand what all those things are.

You’ll also need to be able to decide which AI model is best suited to each task and then instruct it appropriately for the most striking result.

5. Fluid visual design

Although it’s now commonplace to chat to and speak with AI assistants, it’s important to remember the tasks they’re assisting with and conversations they’re having are with human teams and customers. Your UI/UX interface design and messaging format is as important as the technical components that make your AI agents work, so people have a hassle-free experience and can enjoy it. You’ll need to give as much thought to browser compatibility and ease of use as you do to branding, styling, and naming.

6. Robust security

It’s impossible to cover the depths of your security, disaster planning and recovery needs here; it’s a whole separate article entirely, but it includes following best practices to prevent vulnerabilities, penetration testing and regular security assessments by third parties, and compliance audits.

You’ll need to use redaction to protect PII (personally identifiable information) in the right place at the right time for all AI-driven conversations, internally and externally.

7. Compliance handling

You not only need to be compliant with ADA and WCAG accessibility standards for inclusive design and assistive technologies, you also need to be compliant with industry standards like HIPAA and PCI DSS, GDPR regulation, legal obligations, and meet strict data storage location laws too. Absolutely no skimping.

8. Strict privacy policies

Data protection, as we already know, is a big issue in AI, and rightly so. AI agents are handling all the tasks your teams do and so should be bound by the same strict confidentiality and data protection rules. It’s critical you collect only necessary AI data and appropriately contain and manage consent for using it. Trust in business is easily broken when companies don’t pay close attention to details around data.

9. Time to test

It takes a long time to test and build any new AI-powered system, but particularly a public facing one where every word you say will be scrutinised. Yet AI is advancing so fast, if you aren’t already agile and equipped with the expert team you need to get every element of a secure, smooth AI system off the ground, you can miss opportunities or make mistakes rushing. One thing’s certain; you can’t skip the testing phase and it’s not a once and done task.

10. Timely updates

As LLMs and all AI technologies evolve quickly from here, it’s important to update your systems regularly to keep up with and in line with industry-standard innovations in AI ― yes, all of them! 😬

11. Comprehensive reporting

It’s typical to have a dashboard for reporting on all AI agent activity. You’ll need to know what customers are using it for, how effective it is at alleviating repetitive, routine, or mundane tasks from the workload of your teams, and spot straight away if a specific journey can be improved, or you need a new channel or response.

It’s likely you’ll also want to export data and share reports with other leaders to transform your insights because once the technology’s played its part, it’s time to add human reasoning and logic to turn around any problems and make a noticeable difference to your trajectory. LLMs alone simply can’t do that.

12. Consistent messaging

No matter what tech and systems you have in place, the person on the other side of your AI agent, chatting along with it, only sees the outcome of your efforts. Anyone who’s ever used ChatGPT knows AI-generated responses produced purely by GenAI are dry, generic, sometimes random, and occasionally include mistakes. LLMs are also incapable of following any sense of business logic or reasoning. If you don’t want to confuse people, bore people, give them incorrect information, or otherwise turn them off completely, it’s necessary to control every element of your outward-facing message, not just let an LLM spit something out.

This is one of the most damaging outcomes of using open source LLM we’ve seen so far, with brands not fully comprehending the future of conversational AI and how crucial it is to protect the message you put out into the world via AI.

13. Scalability, reliability and maintainability

Even though AI has been around for decades, AI experts are only now getting the tools they crave, like LLM, which, when combined with other technologies allows them to do the things they’ve always wanted to do and could envisage would one day happen. To make sure your AI stands the test of time, you’ll need to be able to scale it, maintain it, and make sure it’s reliable, with:
– Elasticity to scale resources up or down as needed
– Load balancing to distribute workloads evenly across your resources
– Stateless architecture to keep servers free of storing user session data
– Fault tolerance to keep operating despite failures
– Mean Time Between Failures (MTBF) to increase reliability
– Graceful error handling for unexpected conditions
– Modular design to simplify updates and maintenance
– Code quality that meets coding standards
– Documentation along the way that’s always up to date

The list here goes on, and we haven’t even mentioned yet the need for language translation, managing vendors and subscriptions, servers and load handling for peak time usage, logging debugging and analysis, and automating alerts for system issues…

Building systems like this takes years to master, which is why it makes sense to use a purpose-built platform that already covers all necessary bases.

"It's tempting for organisations to take their AI build in-house, but becoming an AI company on top of your primary business is a massive undertaking. It’s taken us nearly a decade of experience in application, integration, compliance, and education to get to where we are today—and AI continues to evolve daily, alongside the regulations, security requirements, and risks that come with it. Building an LLM over your data is just one part of the puzzle; there are countless other moving pieces to consider. Using a custom-built tool to keep your AI up-to-date and secure allows you to build on a solid foundation, stay compliant, and focus on what you’re great at— whether that's selling cars, serving your community, or managing your core operations."

Abbie Heslop

Keep your company AI in safe hands

Picture your best team member; the one who knows how everything works internally, understands your customers or clients externally, is efficient, pleasant, and professional in everything they do, and consider all the knowledge they hold in their heads. LLMs can’t learn that and instinctively know what action to take or how to apply context to solve a business problem. They’re the soloist without an orchestra until you get everything else working alongside them to produce the right sounds together.

Your orchestra of leaders, workers and industry experts have to recognise and understand the business need (use case) first, then apply everything they know to an AI system that facilitates the AI agent being able to put that knowledge into context and do something useful with it on behalf of your business.

You can’t take the risk an LLM used as a ‘wrapper’ ie. without all the support it needs to work well, might or might not get that right.

Make a promising start

AI Studio has all the advanced features you’ll ever need to launch and scale an AI assistant to help your business grow. With the right LLM guardrails in place, a human in the loop, enterprise-grade security, and more features than a motherboard has chips, after 10 years of us building, learning and evolving, our AI assistants now reach an average success rate of 96%, answering almost all requests without human help, and reach as high as 99.88%.

Speak to us and get a personalised demo, or talk through how to start strong, effectively and affordably, in enterprise AI.

Book a discovery call

FAQs

My IT team says I can build a chatbot online, is this not right?

You can create an AI chatbot in minutes online, but you’ll need to do your due diligence on any platform you use to produce it if you want one that generates positive ROI.

Companies like DPD and Chevrolet have been embarrassed by their introduction of AI chatbots that have gone on to swear at customers and agree to sell expensive goods off cheap. LLMs on their own can’t be controlled in terms of content they spit out and they’re prone to getting information wrong, so be sure whatever platform you use to create your AI chatbot has guardrails in place and the ability to control LLM input at every turn. Choosing the wrong platform can be costly to your business, so it’s best to first invest your time in researching the tech and tools your AI chatbot will be built on, so it serves you well long into the future to help not hurt profit.

What companies have already generated positive ROI using AI?

Barking and Dagenham Council generated 533% ROI on their AI assistant in less than a year. As well as this, they saved tens of thousands of pounds by introducing a smart AI assistant, and customer satisfaction increased by 67%.

Read the case study

How do I make sure my AI stays compliant?

There’s no established best practice for this yet, but since it’s so vital you do, why not lead the way. Get to know the teams behind the AI you’re buying or building, so you can trust in and rely on their knowledge. They should be expert in data management and security, knowledge management and compliance. Check for proven case studies too.

Learn more about AI security
Advice for highly regulated industries