Chatbots have attracted much attention amid the rush to adopt artificial intelligence. This is because they are easy to build, familiar and play on the fears of robots taking jobs. In March, Klarna announced its AI assistant handled the work of 700 employees.

Chatbots are a simple interface for people to communicate with AI models. This enables text and voice prompts and responses including words, audio and video. We have a working version in WhatsApp serving over 400,000 people in the Middle East. Let us know if you’d like to see it. Feedback includes,

“This blew me away” – CIO of 5,000+ European financial services firm.

Chatbots go well beyond customer experience. As we do not expect this to be the primary purpose of AI, we view chatbots as interfaces for fully automated internal processes. This is developing rapidly in the financial services sector.

Updating Legacy Technology

Financial services are rife with legacy technologies. Often these are housed in siloes and go untouched by developers who have long-forgotten how the code works. AI is perfect for understanding old code and integrating tech stacks without the need for expensive middleware.

As a result, AI is the enabling technology that allows front-to-back automation of manual workflows. But it’s more than a buzzword to get your longstanding project approved. AI is the means to make sense of complicated workflows that standard digital transformation leave untouched.

The Importance of Open Source

AI is no panacea. Training a chatbot on internal documentation is easy but according to McKinsey, is only 15% of the costs of implementation. The associated change management and staff training are more expensive. These are up to three times more costly than digital transformation, because the change is comprehensive.

One way to control costs is to use open source software. Hugging Face hosts over 650,000 models and counting, meaning there is an abundance to choose from already trained in tasks businesses want doing. Firms need help choosing because deploying too many variations cuts into cost savings.

Next it’s important to control the cost of compute. This means reducing the time and effort required to perform a task. There are two ways to do this.

RAG and Fine-Tuning

This first is retrieval-augmented generation (RAG). This is streamlining prompts with specialist knowledge, that allows access to near real-time data and non-experts to use foundational LLMs^[1]. This overcomes the limitations of token size on uploading documents as examples and improves model efficiency with shorter prompts.

Fine-tuning is the second technique. AI models weight the probability of paths through the neural network. More layers mean the more a model can do, but the extra weights become expensive to operate. Fine-tuning strips out elements that are unnecessary for projects.

Complicated processes require more than one model. This is a plus because breaking down a process and modelling each step, results in models that may be used again. Reusable code and models are essential to scaling with AI.

Many processes in financial services require data retrieval, analysis and subsequent action. For example, checking a quote in trading, banking and insurance requires access to data. This is compared with the quote and either confirmed or corrected. As information is sensitive, the processes are overseen by a compliance model controlling data access. 

Modular building also improves the ability to switch models between service providers. This is a critical element of cost control. No business wants to be beholden to a single supplier because of rash decisions made in the early stages of AI development.

Questions to Ask

There are many other means of cost control and efficiency improvement when scaling AI. These include serverless solutions and infrastructure as code. Yet the most important is the involvement of business users at an early stage. Rejection of solutions late in the day means expensive write offs of development spending.

Here are 8 questions to ask before embarking on an AI-empowered solution:

1. Will the project deliver enough value to the business?
2. Is the project of strategic importance?
3. Are end users supportive?
4. Is the timing opportune?
5. Can we repurpose the code or elements of the solution?
6. Do we have the data prepared and secured?
7. Is the solution proven or theoretical?
8. Will the solution scale?

When you have favourable answers to those questions, contact us and we’ll provide a no obligation consultation about plans. We’d expect to find a benefit that goes far beyond building a chatbot.

[1] Large Language Models