Depending on your experience with machine learning, programming, and chatbots, your answer will vary. I’m here to answer this question as a Digital Marketing Specialist at Globalme by experimenting with the most popular chatbot platforms on the market. Although we do have experts that can configure these chatbots on a whim, this article will paint a picture of how easy (or difficult) it is to configure a chatbot with almost zero experience.
There are three key aspects that we used to compare the platforms. The documentation, configuration, and testing experience were all assessed in the eyes of someone with minimal chatbot experience.
This article is a summary of a larger chatbot comparison experiment which compares Amazon Lex, Google Dialogflow, IBM Watson, and Rasa against each other. In the downloadable white paper, you’ll see the full journey unfold when it comes to configuring each chatbot. If you’re a novice that is trying to decide on the simplest chatbot platform to use, then read the summaries below or follow along by downloading the full white paper.
Reading the documentation for each platform was simple enough to follow, and each one of them provided tutorials. We spent an average of 1 hour reading the documentation and terminologies for each platform. Rasa had the most interactive documentation out of the entire group. Rasa allowed users to play with the Python code in the Get Started guide, which followed a story-like framework.
Although Rasa had the most entertaining documentation, IBM Watson and Google Dialogflow seemed to have the more thorough but still simple resources to follow. Reading Amazon Lex’s documentation felt like the reader would need to have previous experience with chatbots and machine learning. In our case, the Amazon Lex documentation was a bit too advanced. The image below shows how a chatbot works on Amazon Lex, but the rest of the documentation wasn’t as visual.
For someone who wants to start learning about chatbots, Google Dialogflow’s documentation wins.
Comparing the configuration of each platform was more difficult. Each one provided a different experience, and each one had its quirks. One thing’s for sure: the Rasa chatbot was more difficult to configure on the fly. It wasn’t as simple as Dialogflow, Lex, or Watson, but we can appreciate that it still made configuring a chatbot with Python an easy (and fun) experience.
Amongst Amazon Lex, Google Dialogflow, and IBM Watson platforms, Amazon and IBM did a great job of providing examples that we could take and modify for our use case. It took 2 hours to configure Amazon Lex, and just over 1.5 hours to fully configure IBM Watson.
Why did IBM Watson take only 1.5 hours? Watson provided enough examples of user intents, entities, and natural language data for our chatbot to be fully configured to our needs. In fact, we dedicated another half an hour to configure our IBM Watson bot a little more. It felt so much more intuitive configuring IBM Watson versus Google Dialogflow. Amazon Lex falls between the two in terms of the configuration experience.
The IBM Watson conversational agent takes the cake in our configuration experience.
Testing the chatbots we configured allowed us to detect the elements we needed to work on, as well as any glaring mistakes. Let’s be honest here, testing with the Rasa chatbot was not as easy as the others. It was so much more satisfying though – there’s something about the more primitive look of the chat dialog that gave us multiple “Eureka!” and “Aha!” moments.
When it came down to testing our work in real-time, Google, IBM, and Amazon’s platforms were a lot more intuitive. The three platforms had conversational agents that could be trained and tested after each change or adjustment. All of them looked pretty in their own way, and each one would give us relevant error messages that pointed to where we misconfigured the bot.
IBM Watson had a tiny edge by labeling each response and command with intents and entities. This gave us the knowledge to understand our dialog and how the bot comprehends our commands.
Google Dialogflow and Amazon Lex had more similar experiences as both platforms used a real-time conversation box to show our bot’s training progress. The real-time conversation box reflected what the chatbot would look like using their platform, but Amazon Lex did a better job of labeling all variables in the responses. For example in the image below, the intent and slots were labeled to show what the chatbot “understood” out of the conversation or command.
This was great, but what if we want to share this testing experience with others? You can test our Google Dialogflow chatbot here.
This is where Google Dialogflow had an edge. If you are a novice chatbot-builder, you are likely to seek feedback from humans that understand these chatbot platforms better. In the case of testing your Google Dialogflow agent with others, it’s entirely possible. There is an option to share a link to the chatbot or embed the chatbot so that others may use it or test it immediately.
Each platform had its unique strengths in the testing phase. There was no clear winner here because each platform may have a different appeal to individuals. For example, the IBM interface provides a ton of information we can work with, but the interface might feel less familiar than Google Dialogflow and Amazon Lex.
The testing of each platform would depend on the user’s skill level as well. More advanced users would gravitate towards different platforms depending on their particular needs. For beginners, all that really differed was the look of the testing environment and how the intent or entity information was presented.
The testing experience was a tie between Google Dialogflow, Amazon Lex, and IBM Watson.
Which platform made it easiest to train and build a conversational agent? It was a very close decision. Although it took more time and resources to train Google Dialogflow, we did end up with a great end result. On the other hand, Amazon Lex provided a sample we could configure right away: a chatbot for a flower shop. This made the configuration process simple and easy to follow along. It took only 2 hours to configure the Lex chatbot before it understood a user’s requests and commands related to projector rentals.
If we were to use the chatbot in the real world, Google Dialogflow looks like it is the easiest to integrate with messaging platforms among the pack because of the number of one-click integrations that are supported. Roll out your Dialogflow bot with the click of a button.
Because of our overall experience with building a complete and fully functional agent, we crowned Amazon Lex as our winner.
Did we factor in user error? In the complete version of the experiment, there might have been a few user errors that you noticed. For example, intents and entities may have been configured inefficiently. Example utterances and prompts may have been too excessive or not thorough enough. There are certainly areas where human error could have skewed the results, but this is all a part of the overall experience.
What did we learn about configuring a chatbot?
Truth be told, the big three chatbot platforms: Amazon, Google, and IBM were all very similar in experience. Rasa was the only chatbot platform that was tested which required some hands-on Python coding.
Someone without any prior hands-on experience in coding, chatbots, and machine learning can still build conversational agents with a little time investment. The chatbot in our example may not have been developed to work in real life scenarios since it only responds to a single case, but they could be built further to work in real situations and use cases.
For the theoretical business Quick n’ Easy Projector Rentals, building a chatbot was surprisingly simple. We picked Amazon Lex as it had the most user-friendly experience
Now that we’ve picked Amazon Lex as a winner, does that mean it is the go-to platform?
This experiment taught us that the most powerful chatbot platform is the one that successfully satisfies your unique use case with as little effort as possible. It will depend on the user’s technical capabilities, experience with the overall platform (ie if you do work on AWS already, Lex will probably feel the most natural), and the complexity of the needed agent.
Each chatbot platform had its own advantages and drawbacks. What was most important to us was how these chatbots handled user intents, entities, and other variables. What we learned was that it wasn’t the chatbot platform or knowledge in coding that limited our results. It was the lack of user intents, entities, and other variables that can be fed to the platform’s algorithms so that it could learn how to communicate better.
If we spent more time with each chatbot platform, then the end result for each would be a lot more fleshed out.
Want to see how we configured Amazon Lex, Google Dialogflow, IBM Watson, and Rasa? You can follow the entire experiment in this white paper, which shows the steps we took to map out our intents, entities, and other variables.