How (and why) we build Chatbots – GLAMbot series #2

A brief introduction to chatbots

In the previous blog post in this series, we talked about creating social media-based culturebots. There’s also lots and lots of use cases for chatbots – a more conversational approach to connecting with your users.

Just a quick refresher, when we talk about bots, we mean automated processes or tasks online. Chatbots are applications to conduct conversations with users and can be either text or audio-based.

Chatbots can help you interact with users in new contexts and where you weren’t able to before, making your touchpoints more interactive and conversation-based. Chatbots can also utilise some pretty advanced Natural Language Processing (NLP) techniques to help users get the right information at the right time (and in the right words), but more on that a bit later.

There are lots of use cases for chatbots in galleries, libraries, archives and museums (GLAM), some of which we’ve talked about before. Jisc have recently published a report on chatbot applications in education which found that, while educational institutions are still at an early stage of chatbot development, the technology is developing rapidly to enable smart chatbot implementations that, most importantly, can learn how users ask questions. This might sound familiar to anyone working in a library – users aren’t always the best at asking for what they want. Read the full JISC report here. (PDF)

In this article, we want to share some of our process and recommendations for building chatbots based on our recent experiments, prototypes and the kind of services we build for the GLAM sector.

Why use a chatbot

Chatbots provide a bunch of different ways for us to help users achieve their goals, whether this is getting help with a new service, finding contact information for someone or learning something new. You can use chatbots for practical help for recurring queries or activities like booking a meeting room. But you can also use them to delight and reward your users like Tina the talking T.rex, a character bot created by National Geographic Kids that lets you ask a T.Rex all those questions you’ve always wanted to ask.

Providing interactive touchpoints for our users in an accessible and always-on way is important as our services become more distributed and the expectations of our users change.

For example, just-in-time support is a great way to provide help to the increasingly distributed audiences for our services. We can no longer expect our users to visit the nearest helpdesk if their main access to our services is remote. The expansion of BYOD approaches also requires a different approach to support. The conversational aspect to connect with people is certainly not new, it’s just that we’ve got more (and better) opportunities to do this now thanks to rapidly evolving conversational technologies. As Erika Hall puts it in her book on Conversational Design, Conversational design is truly human-centered design, every step of the way. There is no next big thing, only the next step in an unfolding story of how people use technology to be more themselves.

Our users are becoming more used to engaging with conversational interfaces and we can use this to provide better, more engaging services. Consider the library and museum tours – these could be full personalised with the help of a well-scripted chatbot. And this means that visitors can instigate a self-paced tour whenever works best for them – even in an unstaffed 24/7 library or as they explore a museum at their own pace.

When you think of an automated chat tool, you’re probably thinking of a stilted, rudimentary ‘bot’ that looks for keywords and often misunderstands even the most basic variations on what it expects to hear. Thankfully those are becoming a thing of the past. Now we can also provide a more personalised experience using smart, conversational interfaces.

Types of chatbots

There are different types of chatbots, from character bots (lke our T.Rex friend mentioned above) to helpdesk bots that often ppear when you visit a website these days.

Chatbots are often defined by the communication channel they’re created for. So, you can often find chatbots embedded into websites, but there are also bots designed to interact with users on messaging platforms such as Telegram and Facebook Messenger.

Some other kinds of bots that you might consider include Slack bots and (our speciality) SMS bots. These are different channels for what are all essentially chatbots. Again, the channel you decide to publish your bot to should be where your users are. But you can often disseminate the same chatbot across multiple channels without needing to run separate bots, depending on how you write and publish it.

There are a few things to remember to make it easier for your users to engage with your chatbot:

  • don’t pretend to be human. Honesty is always better than pretence.
  • give users hints and prompts for communicating with you. Context matters and getting lost in an interaction is not a good experience
  • do allow for human intervention when needed, so a real live person can step in if the bot is not able to help your user or if something goes awry in the conversation
  • be conversational but don’t overdo the quirkiness. Remember Clippy? Don’t be that guy.

SMS messaging is a great way to reach people without requiring the use of any particular specialised messaging app or setting up a distinct account but SMS bots are tricky and have some extra gotchas to be aware of. The order of messaging is not always reliable and there are some stop words (like STOP and HELP) that are usually implemented by phone systems and providers. This guide has more information about that.

Building smarter chatbots – Machine learning and chatbots

There’s a lot of talk about ‘Machine Learning’ in chatbot development but what do we mean by machine learning anyway? Machine learning is a broad term that encompasses Natural Language Processing (NLP), something that’s used in most chatbots to interpret conversational cues received by humans. But this is not something that computers have been particularly good at – think of your experiences with voice recognition systems when calling a company for help with something – it can be hard to get your message across to a confused computer programme. But that’s before we take into account more advanced processes such as ‘Deep Learning’. It’s Deep Learning that enables bots and other platforms to ‘learn’ by analysing huge amounts of data.

Natural Language Processing lets us move beyond keyword matching approaches to helping us understand what our users mean – what is their ‘intent’. But there’s also a lot of hype and subterfuge around artificial intelligence and machine learning technologies at the moment. And it’s early days with these technologies, you don’t have to look much beyond Microsoft’s dive into AI-powered chatbots with Tay to see that there’s a long way to go yet.

These terms are thrown around a lot but the reality when it comes to the available tools and technologies can be starkly different. If you’re still trying to get your head around the various terms, this interview with Terrence Sejnowski (author of the Deep Learning Revolution) is a pretty good primer.

Our bot-building workflow

When designed well, conversational interfaces are able to amplify the human-computer bond — and create a relationship grounded in communication. When they work poorly, they produce mistrust. When they work effectively, they promote trust. John Maeda – Conversational Design

 

Our workflow varies a bit depending on whether we are targeting a particular channel but the general approach is the same.

 

First we need clear objectives of why we are building the bot, what is the use case and the intended audience for it. Chatbots aren’t separate from user-driven processes that we use for other digital projects – including requirements gathering, user stories and personas.

There are some good protoyping tools for chatbot out there, like BotPreview and Botframe if you want to get something up and running quickly to show your team or to use for initial user testing.

We then start writing the scripts. Conversation skills is the most important part of our chatbot development – the last thing we want is to deter users from interacting with us by an awkward, stilted or confusing conversation experience. We’re also firmly of the belief that bots shouldn’t pretend to be human (and vice versa 🙂 ). The bot tools we prefer enable human intervention when needed so we’re not leaving our users trapped in a loop of bot communication when their needs extend what the bot offers.

We write scripts in a Google Docs or using Twine (which is helpful for working out the interactions and different pathways based on user responses).

Once we have a script, we divide the conversation into sections based on different questions or capabilities. We then start transferring the script to our chatbot authoring tool focusing on the intents – what the intentions or goals of the user are. The development platforms we use has changed overtime but has included Dexter and Flow XO. The chatbot authoring tool landscape changes pretty fast but right now we’re excited about BotPress, which is an open source bot development platform that has been a lot of fun to work with and Wit.ai which comes with some handy Natural Language Processing (NLP) powers. Unfortunately it’s limited to web-based bots right now.

While we’re big proponents of the Create Once, Publish Everywhere (COPE) approach where possible, there are some differences in the formatting and navigation options for different messaging platforms and not all authoring tools support all channels. We’ve recently been trying out Twilio Studio for building SMS bots since we use the Twilio API for SMS messaging anyway.

And while we’ve experimented with building chatbots for smart speakers such as Amazon Echo and Google Home, so far we’ve found that there’s not a strong use-case for these closed environments just yet and the failure experience for audio chatbots is still too much of a barrier to public use.

Build for humans

The main lesson we’ve learned in exploring chat interfaces is to focus on the conversation part of conversational interfaces – letting your users communicate with you in the way that’s most natural to them and returning the favour is the main key to a successful conversational interface. This is something that will likely take time to get right.

We do a lot of testing with chatbots, and our Treasure Hunt chatbot went through quite a lot of iterations to simply the language and provide lots of different response options before it went live. Conversation is a immensely complex and varied thing and so your chatbot will take time to get the tone right – smart small and keep building.

We’re excited about the potential of conversational interfaces for providing more accessible and flexible ways for our users to connect with services and get support when they need it. If you want to talk to us about chatbots, treasure hunts and all things in-between, please get in touch.

References

https://builtin.com/data-science/easy-introduction-natural-language-processing