tetrel Logo tetrel Logo

Shopping Tips for Artificial Intelligence – Part 4

2021-06-07 by Dr. Anselm Schultes (LinkedIn | Twitter)

The first three parts of this series were about the big picture. In this part I would like to introduce you to the technical aspects. Because although they are predominantly technical topics, they are closely linked to economic factors.

The devil is in the detail - you probably know this saying. Programmers can tell you a thing or two about it. So before you decide on one Artificial Intelligence (AI) solution or another, it is worth taking a very close look at the economic and technical challenges. What may be a success from a technical point of view may be a failure from a business point of view. The reverse is also true, of course. There are several reasons for this, which lie in the different stages of the AI implementation process.

But first:

As already mentioned in the first three parts, AI cannot be compared with conventional software that is purchased off-the-shelf and immediately deployed. Let me briefly go into detail here for a better understanding: Most applications of AI today are based on Machine Learning. In this process, algorithms learn to recognise patterns or complete tasks. The models created in this way replace a large amount of code that would otherwise have to be written manually.

Especially for less structured data, for example text and images, processing by computers would be virtually impossible without using machine learning. In a sense, machine learning is code that writes code. This happens during the training of the machine learning model. Through training, the model learns by itself through examples what would otherwise have to be programmed manually. But training is only one aspect. An essential component of AI-based software is the domain knowledge that goes into the development of the model and the selection and generation of training data - that of the users, yours and that of the AI provider. No algorithm, no computer, no machine can provide this.

The three stages of a typical AI project

The first step is developing a pilot solution: In this initial phase, training data is collected and analysed. This means understanding the problem, casting it into assumptions and solving it using algorithms and models. Often this includes training a machine learning model. The efficient generation of training data, for example supported by Active Learning, can bring decisive advantages here. This is the actual core task in an AI project. Large quantities of data are handled in this phase, and they usually come from a variety of sources. To put it metaphorically: you need a plumber for these different data pipelines …

This phase requires collaborative work between the AI provider and the customer, i.e. you, to jointly develop a solution as a proof of concept. These are iterative steps that have a research and development character. As soon as a model has been trained, it is tested in business operations. The outcomes are used to further improve the solution. This phase can take between 3-6 weeks.

The second phase comprises the implementation and integration, that is, the conversion into productive systems. The model is prepared for full-scale use and then integrated into relevant processes and systems. Either in existing software or as a mobile or desktop app specifically for this purpose, in order to be able to actually work with the results achieved. In this phase, which can last between 1-3 months, the model is further optimised, iterated and improved.

Finally, the third phase includes evaluation and monitoring. The performance of the AI solution is then permanently and specifically monitored and adjusted accordingly. The AI model continues to learn from the newly collected data and thus adapts to new circumstances. This phase is de facto an ongoing process from implementation and integration onwards.

You can see that the scope of the individual phases in the description gradually decreases in terms of content, but at the same time increases in duration.

Quick in results and low in risk

An AI project is often similar to research and development. Hypotheses are formed, tested and then validated or rejected. Models are developed and improved iteratively.

One central aspect is the variance of the input data. A large variety of input data is a challenge for most data-driven projects. That is true in particular for projects working with speech or text. Here’s an example: “I don’t think the offer is all that great.” You could express this sentiment in a million different ways. And that doesn’t account yet for mistakes in spelling. A rule-based code that recognises and flags such sentences may quickly achieve a hit rate of about 60%. The next 10% is more effort. For each further improvement, the effort per additional percentage in hit rate increases further. This is often called a long-tail problem. Modern AI methods can solve such problems very efficiently in many cases. It is crucial though to take this challenge into account early on.

Implementation and Integration

A successful pilot solution shows that the problem can be solved and the solution provides business value. The next step is implementing the solution as a production system. Although this phase is also time-consuming, it is less decisive from an economic point of view, as the main risks have already been resolved with a successful pilot.

The beginning after the end: model deployment, monitoring, iteration

And that’s it? No. Actually, that’s when it really starts. Once the solution is online, it processes real live data that is constantly changing, learns and constantly improves itself. However, it does not do this all by itself. It has to be closely monitored and adjusted. Not necessarily, but preferably continuously. This point is extremely important - as are the previous ones. Is this point, this phase, sufficiently taken into account in your project plan?

AI: it’s about numbers

The conclusion of this analysis and the resulting shopping tips: In addition to business KPIs, also develop technical benchmarks, both based on your current situation and your objectives - and match them. Decisions are often made on purely economic grounds. This aspect is of course important, but so is looking at the AI provider’s expertise in terms of software and machine learning, as well as the domain knowledge available. The interaction of these three factors with all individual aspects is essential for the success of your AI project, not one alone. Because the devil is in the detail …