Smarter bots with natural language processing

Highlight

The closer bot interaction gets to the one of a human the better the end user experience will be. See how to leverage Microsoft Cognitive Services LUIS for natural language processing so that users can type naturally while allowing bots to understand and act.

What is LUIS?

LUIS is shorthand name for Language Understanding Intelligent Service. It is one of Microsoft services available in Cognitive Services package in Azure. LUIS is very well described by a quote from Microsoft Azure website.

One of the key problems in human-computer interactions is the ability of the computers to understand what a person wants, and to find the pieces of information that are relevant to his/her intention. Our Language Understanding intelligent service, LUIS, provides simple tools that enable you to build your own language models (intents/entities) which allow any application/bot to understand your commands and act accordingly

Source: Language Understanding Intelligent Service | Microsoft Azure

Understanding Language Models

Language model is a statistical model used in natural language processing. Using probability distribution over sequence of words a well-trained model can estimate the relative likelihood of different phrases. Language models are not only for text recognition but also can be used for speed recognition, machine translation, handwritten recognition and many other. In LUIS building language models is very simple. At the very least a single intent must be defined and trained for already meaningful results. Intents are as the word describes is a general users’ intentions; actions if you will. Training models is done via utterances which any specific details required for LUIS to base its recognition on. If details of an actions are required then entities come into play.

To summarize most important LUIS concepts

Intent representation of user action. For example, user asking “book me a flight” would result in intent about flight booking like BookFlight.
Utterance textual input from the users that application needs to recognize. For example, user asking to book a hotel.
Entity detail of an action. For example, asking for to meet Tom at noon would have two entities which is name of the attendee (Tom) and time which is (noon).

Examples of intents and entities Screenshot

Although it is worth remembering that the more generic the application would be the more intents could slide towards being entities. Imagine the case where application allows users to add or remove files from the server. Of course, those could be trained as separate intents called FileAdd and FileRemove but if the list of actions would be dynamic training the bot to have intent call FileAction with entities Add, Remove etc. could have more sense. It is worth remembering that the more specific the bot the better the outcome.

Exemplary utterance predictions Screenshot

LUIS with Azure Bot Service

Azure Bot Service has built in integration with LUIS service. When selecting appropriate template bot service initializes everything including creation of new application in LUIS. Bot service also inputs LUIS app credentials into application settings.

Screenshot

Everything works together nicely thanks to Bot Framework using

Microsoft Bot Framework

Azure Bot Service

LUIS

Connected into final design

LUIS portal

LUIS comes with self-service portal available at luis.ai. Portal delivers set of capabilities including

Application management – list of applications with ability to create, delete, export and import.
Intent management and training – managing intents and inputting utterances for model to learn on
Entity management – managing entities
Prebuilt domains – off the shelf directory of intents and entities
Features – for better performance and intent/entity prediction accuracy
Train & test – panel for testing and training models
Publishing – panel for publishing application with ability to move to staging environment as needed

Creation of a LUIS chatbot

In this article a basic form bot will be built which will serve users with a basic information about bot series on this blog and serve public links to social media.

Instead of reading you can watch!

Or just follow the steps

Log into Azure Portal
Find Azure Bot Service in Marketplace
Create Bot Service
Fill in the data.

Consumption Plan will Functions pricing model (pay per execution). Preferred for most bots.
App Service Plan will use App Service (pay for reserved capacity). Preferred if using Free Tier for learning purposes.

Portal should automatically open bot editor after deployment is successful. If not open bot can be opened via either Portal > App Services, Notifications or Dashboard if ‘Pin to Dashboard’ option was selected.

Click create button redirects to bot wizard where C# should be set language and Language Understanding as a bot template.
Microsoft App Registration page where application must be first registered.

Note: Forgetting to save app password might be troublesome in some scenarios but this isn’t the case. App password will be available in bot app settings
Once bot is created developer is transferred back to Azure Development portal and after hitting Create button the bot is provisioned. Provisioning takes between 2 and 5 minutes and during that time template sets up Bot Framework application, storage account for the code and logic, bot service application and application insights.
Test the bot with IDE chat control

Model Learning

Now that the bot already exists and everything works means a framework to build intelligent bot was already established. Moving into LUIS.AI portal is the next step where the model training beings. At this point developer should already have precise and meaningful business case with a general idea of how would intent and entity modeling look like.

Before beginning figure out what naming convention to use. For purpose of this demo intents will be Camel Cased with dot between the sub categories. For instance, flight booking would have intent named Flight.Book.

Business Case

Deliver a bot which understands user and provides information about Marczak.IO Bot Services, their author and upon request gives social media links.

The plan

Identified three user intents.
Series.Info for users seeking information about bot series.
Series.Author for users seeking information about author.
Social.Location for those seeking social media links.
Identified Social media entity SocialName which would be social media website name.

No that the plan for building model is done let’s create it.

Log into LUIS.AI portal
Select application from the list
When presented with dashboard navigate to Intents section
Select Add Intent button to provide first intent name Series.Info
And enter few utterances into the field and press enter

Sentences used for this demo are

i need some information about the series
name of series
please let me know about those series
what series are those ?
what are those series about ?

Hit save
Follow the same process for Series.Author

who made you ?
whos the author
who’s the author of the series ?

Follow the same process for None intent. It’s important to teach none intent so that random phrases like hello won’t get misinterpreted.

help?
whats up
hello!
who are you?
test
hi
hello

Social.Location intent will be skipped for now to keep it simple
Go to Train and Test section and select Train Application button
At this point to avoid mistakes and pointless debugging sessions a testing panel should be used to check trained model Intents are scored from 0 to 1 which means from 0% to 100%. If bot is not responding as intended it might be that it wasn’t learned properly. Usually it means that sentences were either too similar or too generic.
Navigate to Publish App section and select Publish to production slot. Sometimes, I’m not sure why, portal fails to publish the app so in case this happens just republish the app.

Now that the bot is published each of the intents must find its own action within BasicLuisDialog.csx class. Class method name does not matter but the attribute on the class does. Each method that should be mapped to luis intent must have LuisIntent attribute with parameter being exact intent name from LUIS service. End State of the class should be looking like this with highlighted the key parts that were changed.

using System;
using System.Threading.Tasks;

using Microsoft.Bot.Builder.Azure;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Builder.Luis;
using Microsoft.Bot.Builder.Luis.Models;

[Serializable]
public class BasicLuisDialog : LuisDialog<object>
{
    public BasicLuisDialog() : base(new LuisService(
        new LuisModelAttribute(
            Utils.GetAppSetting("LuisAppId"), 
            Utils.GetAppSetting("LuisAPIKey"))))
    { }

    [LuisIntent("None")]
    public async Task NoneIntent(IDialogContext context, LuisResult result)
    {
        await context.PostAsync($"None intent. You said: {result.Query}");
        context.Wait(MessageReceived);
    }

    [LuisIntent("Series.Info")]
    public async Task SeriesInfoIntent(IDialogContext context, LuisResult result)
    {
        await context.PostAsync($"This Azure Bot Series - Smarter Bots.");
        context.Wait(MessageReceived);
    }
    
    [LuisIntent("Series.Author")]
    public async Task SeriesAuthorIntent(IDialogContext context, LuisResult result)
    {
        await context.PostAsync($"Azure Bot Series are made by Adam Marczak. Visit Marczak.IO for more information.");
        context.Wait(MessageReceived);
    }
}

Save and test the application. Notice how sentences do not have to be the same as provided in the learning model. This is exactly the power of machine learning.

In case a bot would return None intent multiple of times try
Republishing the application
Restarting the conversation

Half way there

Now that the application works there is still one business scenario that was not touched. This is the social media links scenario.

It was already planned to deliver one intent and one entity for this purpose. Starting from adding entities is the best approach.

Go back to LUIS.AI portal and navigate to entities section
Create new entity of by pressing Add Custom Entity button

Give it a name SocialName and a type of List. Reason for this is that list entities are best for predefined set of items like list of available social media. To learn more about entity types visit LUIS documenation
When created list entity must be provided with canonical form of names and optionally any synonyms or aliases. For this demo following list was used.

youtube with alias yt
google
facebook with alias fb
twitter
email with alias mail
linkedin with alias ln

Go to Intents and select Add Intent button to provide first intent name Social.Location
And enter few utterances into the field and press enter Sentences used for this demo are

give me facebook link
what is the email address
what is the facebook url?
give me twitter link
give me twitter url
give me email url
what is the linkedin address
Correctly recognized entities should be marked as [$SocialName]

Of’course link, address and url could be optimized with LUIS phrases but this isn’t part of this tutorial.

Retrain and test the bot
Publish the app
Go back to Azure Portal to the BasicLuisDialog.csx, there is couple of things that has to be done
Add new intent handler called SocialLocationIntent with attribute [LuisIntent(“Social.Location”)]
Parameter of created intent handler LuisResult result has available method for extracting entities called TryFindEntity which returns true if entity was present in the request. It can be used to search for “SocialName” entity.
If entity is present handle all possible scenarios of found values based on canonical name provided when training LUIS. Found value is available under property of the same name.

New Method should look like below.

[LuisIntent("Social.Location")]
public async Task SocialLocationIntent(IDialogContext context, LuisResult result)
{
    await context.PostAsync($"Hey social media is my thing!");
    
    EntityRecommendation socialName;
    if (result.TryFindEntity("SocialName", out socialName)) {
        
        await context.PostAsync($"I know this social media! Let me check the URL for you!");
        
        switch(socialName.Entity) {
            case "facebook":
                await context.PostAsync($"Here's the link {FacebookUrl}");
                break;
            case "twitter":
                await context.PostAsync($"Here's the link {TwitterUrl}");
                break;
            default:
                await context.PostAsync($"Sadly I don't have link for this on me :(");
                break;
        }
        
    } else {
        await context.PostAsync($"Sadly I don't recognize that social media name :(");
    }
    
    context.Wait(MessageReceived);
}