How to learn from AI pilot projects quickly – part 1

Commercial AI Analyst Abbie Heslop looks at the challenges of running AI pilot projects, and how to extract the maximum business benefits and learning from them.

02/07/2019, updated 21/12/2020

Getting into the right mindset

“Traveling through hyperspace ain’t like dusting crops, boy!”

The first thing to understand is that piloting new technologies is nothing like conducting business as usual. The number one goal of business as usual is to keep the lights on; and the number one goal of AI pilot projects is to learn things.

For this reason, many businesses employ innovation and transformation teams whose sole function is to run pilot projects. While this team may operate alongside business as usual teams – and may even draw members from them – their remit is very different, and so is their approach.

In the pilot phase it is absolutely fine to try ten things and nine of them fail, as long as you learn something along the way which moves you incrementally towards your end goal, or which makes that goal a little more well-defined.

Formulating your hypothesis

Piloting technology projects is a little like the scientific method in that it follows a cycle of hypothesis setting and proving. Learn to think in statements like: We want to prove that we could do this (automate 50% of inbound phone calls, for example).

Of course, you won’t prove that with a single pilot project, so you have to break it down into smaller goals that move you towards the objective incrementally. In this case the first goal might be to use AI to analyse your incoming calls to see what percentage could lend themselves to automation.

Then prove that you could automate those interactions or transactions, and demonstrate what conditions apply; for example, what it would cost and how long it would take. Part of your pilot is to see how those conditions change with different approaches to the problem.

We should here draw a line between experimentation and genuine pilot projects. Experimentation involves deploying a technology or process on a very small-scale, perhaps in a test environment or with a very small group of people, in order to prove something about the technology. A genuine pilot project then involves proving that hypothesis in the real world on a larger scale.

Rather than start piloting quickly, some (usually larger) organisations will commission a lengthy and expensive white paper with a top 5 consultancy firm. That’s fine as it can make the destination clearer. However even at the end of such a research project, the glowing overarching vision will still need to be broken down into small sets of projects and resources that will get you there incrementally. When these organisations do get there, they often find their nimbler competitors, who just started testing things, already waiting for them.

Start with real business challenges

There is no point spending time and money to automate something unless it A) really causes customers, staff, partners or suppliers a lot of pain, B) is a time sink or costs too much the way it’s done right now, or C) enables some fundamental new way of doing business that just wasn’t possible before.

The history of technology, particularly communications technology, shows that rapid progress on a mass scale really only occurs when a cheap and scalable solution comes along that impacts on at least two but preferably all three of these factors at the same time.

It’s why virtual reality, for example, is yet to take off despite the technology being pretty good now; nobody has a use case for it that is really transformational. It’s quite the opposite with AI, which has only spent fifty years on the launchpad because we’ve had to wait patiently for the reality of what the technology can do to catch up to the vision. Everyone knows what it could be capable of yet AI currently still sits in the Trough of Disillusionment due to a lack of sophistication and technological capacity when rolling out AI pilot projects (Gartner, 2020). 

When it comes time to decide exactly what business impact you would like some new technology to have, choose something that really matters and which cannot be fixed in any other way. For example, we heard of a financial services organisation that decided to automate balance queries as these made up a significant percentage of its live agent calls. It later turned out that the reason for that was an underlying broken business process. It didn’t need a chatbot to solve the problem after all, so the whole project was based on a false premise.

To decide what areas to automate look at trends in your market, ask your customers what they want, and look at what your competitors are doing. List your main business challenges and pain points, whether they are to do with processes, sales, or customer experience.

Identify one or two challenges that technology could potentially help solve and which would make a significant impact if you did solve them. Of course, you cannot improve something unless you can measure it. For this reason, we always suggest picking a short-term goal that is as specific and narrow as possible while still giving a business benefit. It must also be a common enough problem that you can collect data on it quickly.

In part two we will look at how to build, run and assess pilot projects.

Want the ‘Ultimate Guide to Successful AI Projects for CX Leaders?’

In the ultimate guide we examine 5 common mistakes identified from pilot projects of major banks, airlines and contact centres, how to avoid them and how CX and contact centre leaders can make the most of the technologies that are going to define the next generation of customer interaction.