Rule-based vs. Algorithmic Bots — who wins?

1. Okt.

Neither rule-based nor algorithmic chatbots are a silver bullet. Each of them has certain use cases where they are strong, and others where they fall short. Here, I will explain the difference between the two and share my theories on when we should use which type of bot.

Rules = Explicit programming

In this article I have used explicit programming as the key differentiator between rule-based and dynamic flow generation. This means that the bot does exactly what has been programmed by a human. These are mostly “nested if-then rules”.

Problems with explicit programming:

Limited human mental capacities: if complexity grows, humans cannot handle the definition of rules. That’s one of the core reasons why expert systems failed in the 70’s and 80’s (and machine learning can now partly solve these issues). The maintenance effort grows exponentially with the amount of knowledge being codified and thus quickly outstrips human capabilities.
Impossibility to describe: sometimes you cannot describe all rules. To demonstrate, I would challenge you to write down — rule-based — how to determine whether something that was said was “ironic.” If you try this exercise, you’ll soon discover that explicit programming simply doesn’t work for complex situations due to the sheer number of possible variables to consider. For a computer to figure out how to detect irony, it needs to learn the variables given training examples — much as how kids learn to detect irony through thousands of social interactions. Instead of these skills being explicitly taught, humans and computers both learn from examples.

Algorithm = mathematical calculations

If you use an algorithm to decide on when to display each message, humans define the general rules for the algorithm to follow; the algorithm then decides what to do within these boundaries. The algorithm is trained by learning characteristics from training data — if you change the training data, the behaviour will change automatically. If you want to learn more about machine learning, I recommend the “Machine Learning is fun” Series from Adam Geitey.

The degree of complexity of a chatbot is defined by the number of intents

In the following, I assume you know what intents and entities are. If you do not, read this primer on Chatbot vocabulary.

More intents = more connections to train

You need to differentiate every intent from every other intent. If you have two intents, then you need to train the bot what differentiates intent A from intent B. If you have four intents, you need to train the bot what differentiates all four intents from each one of the others — in other words, that A is A and A is not B or C or D, that B is B and B is not A or C or D, and so on. The number of connections explodes with the number of options.

The number of possible connections between intents does not grow linearly, but exponentially.

The more intents you want to train, the more effort you need to put in. “More effort” in training chatbots means:

If you follow a rule-based approach, you need to come up with many more rules
If you follow a machine learning approach, you need to have much more training data

It is obvious that rule-based systems are easier to set up and faster to generate results. For a small number of intents, they are incredibly useful. But the more intents they need to juggle, the more complex the rule-based maintenance gets. On the flip side, an algorithmic approach that uses Machine Learning will have higher initial setup costs but a lower cost per intent and will produce better results for a larger set of intents.

To figure out the cutoff point between “a small number” and “many” intents — in other words, the cutoff between rule-based and machine-learning approaches — let’s consider a couple cases:

~5 intents

Training a bot with 5 intents is pretty straightforward; just have a starting message prompting the user to click on their desired intent out of 5 option buttons that appear, or prompt them to type what they want and match keywords per intent. Pretty easy.

You can maintain these bots in a visual editor and can manually craft the sequence of messages that will prompt the user’s input. A human can easily maintain such a system.

~20 intents

On the other extreme, it becomes obvious that past a certain point explicitly building a rule-based bot becomes extremely difficult.

Algorithms can help sort things out. A machine learning system can ingest thousands or millions of training data points and can then learn to differentiate between intents. This requires significantly less human effort to maintain compared to a rule-based system.

If you have more than 20 intents, be inclined to use a software that leverages algorithms and machine learning instead of a rule-based system.

The more intents a bot has, the higher the return of an algorithmic approach will be.

I have chosen 20 intents as the cutoff point where you should start using an algorithmic approach, but this can of course vary based on the complexity of the intents and how similar they are to each other. Bear in mind that complex intents that are hard to differentiate might require an algorithmic approach even if the system has fewer than 20 intents; and, on the flip-side, simple intents that are very easy to distinguish might succeed with a rule-based approach even in a system with more than 20 intents.

Which bot is best for customer service?

Out of experience I can say that our customers have an average of ~70 different intents apiece.

With so many intents, our customers clearly need an algorithmic bot to handle their customer service — but are our customers the exception or the rule? I would argue that algorithmic bots are inherently a better fit for handling customer service.

Imagine a marketing bot which pops up when you open a website. What kind of intents would it need to be able to identify? Normally such bots ask their potential customers whether they want to see pricing options, product information, a demo, or a handful of other options. So a marketing bot might be able to get away with a dozen or so intents. On the other hand, think about a customer service bot that needs to identify what process or product the customer is having issues with and then diagnose the specific issue. A good customer service bot will almost always require more intents to work properly.

Then there’s the matter that customer problems change over time — new features launched, new products added, and problems appearing and disappearing every week. This further increases the need for an algorithmic bot; the level of effort needed to upkeep these changes in a rule-based system is simply too high.

So, who wins the epic battle of the chatbots? As we have seen, it depends on the complexity of the battlefield.

Erik Pfannmöller