This was done in 2016 with technologies from 2016 😅
To see a real demo of this bot, go to this website and create natural language queries. The queries will be processed by LUIS and if the intent is right, it will search Twitter using the entities it identified.
In order to build the solution, you need a LUIS application and subscription and a Twitter developer app.
After you get these, create a file called appSettings.config in the root of the solution and replace the placeholders here with your keys.
<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
<add key="AppId" value="YourAppId" />
<add key="AppSecret" value="YourAppSecret" />
<add key="luisApplicationId" value="luisAppId"/>
<add key="luisSubscriptionKey" value="luisSubscriptionKey"/>
<add key="twitterAccessToken" value="twitterAccessToken"/>
<add key="twitterAccessTokenSecret" value="twitterAccessTokenSecret"/>
<add key="twitterConsumerKey" value="twitterConsumerKey"/>
<add key="twitterConsumerSecret" value="twitterConsumerSecret"/>
</appSettings
You can test the completed bot here.
Last week at its annual developer conference, Build, Microsoft announced the new Bot Framework in the attempt to get developers to build intelligent bots using Microsoft technologies.
In this article, we will introduce the concepts of Conversational AI and bots, and will create a bot using the Microsoft Bot Framework that will search Twitter for tweets containing the user query.
We will then integrate it with LUIS (Language Understanding Intelligent Service) from the new Microsoft Cognitive Services which will allow users to input natural language. Then, with the help of Machine Learning, we will extract the intent from the user’s query and search Twitter.
The Conversational AI is not a new concept. I was first introduced to the idea of a conversational bot back in the early 2000s when I was playing with ELIZA, an early example of how to use natural language processing to achieve a very near human-like conversational bot.
The idea behind conversational AI is to have a computer program respond to the user as close as possible to a real conversation.
While early programs like ELIZA were based on predefined scripts, the newer ones are based on Artificial Intelligence and Machine Learning.
(Photo source here)
ELIZA is a computer program and an early example of primitive natural language processing. ELIZA operated by processing user’s responses to scripts, the most famous of which was DOCTOR, a simulation of a Rogerian psychotherapist.Using almost no information about human thought or emotion, DOCTOR sometimes provided a startlingly human-like interaction. ELIZA was written at the MIT Artificial Intelligence Laboratory by Joseph Weizenbaum between 1964 and 1966.
After having read a lot of Medium and TechCrunch articles in the past months about how bots are the new apps (here, here or here), it became pretty clear that the next step in the user interaction is represented by intelligent bots.
A bot is a piece of software designed to automate a specific task. When talked about in the context of conversation as a platform, a bot becomes the chat interface of a regular app.
So you should allow tasks that required full UI to be performed by the user only through conversation.
You just send and receive messages from the bot. No need to learn, understand and navigate disparate interfaces or languages. Users will be able to interact with bots just as they interact with other humans. It’s the most natural way to communicate and transact.
The goal is to have the user input natural language and your bot to perfectly understand and execute the action the user wants.
Of course, you can do this by using terminal-like commands (much like inviting a user to a Slack channel), but then you compel the user into remembering commands and arguments (then, every bot will have specific commands and arguments).
Build and connect intelligent bots to interact with your users naturally wherever they are, from text/sms to Skype, Slack, Office 365 mail and other popular services.
Microsoft Bot Framework has three main components: Bot Connector, Bot Builder SDK and Bot Directory.
(Photo from the official Bot Framework documentation)
The image above is pretty much self explanatory, but in a few words:
- the Bot Conenctor allows you easily to connect your bot to Slack, Skype, via SMS or web.
-
the Bot Builder is an SDK that allows you to develop bots using .NET (C#) or Node JS. It is open-source and you can browse the repository here.
-
the Bot Directory is a collection of all approved bots connected through the Bot Connector. It is a marketplace where users can search for bots to add in their chat applications.
In our tutorial, we are going to use the Bot Builder C# SDK to create the bot and the Bot Connector to test it in a web application.
We will also use the Microsoft Cognitive Services, more exactly the Language Understanding Intelligent Service (LUIS) which will allow us to compute natural language queries from the user.
At this point, we can create a new Visual Studio 2015 solution. In order to do this, we follow the step-by-step instructions from the official documentation.
This is a step-by-step guide to writing an Bot in C# using the Bot Framework Connector SDK .Net template.
- Visual Studio 2015 Update 1 – you can downlodad the community version here for free: https://www.visualstudio.com
Important: Please update all VS extensions to their latest versions Tools->Extensions and Updates->Updates
- Download and install the Bot Application template
- Download the file from the direct download link here:
Save the zip file to your Visual Studio 2015 templates directory which is traditionally in %USERPROFILE%\Documents\Visual Studio 2015\Templates\ProjectTemplates\Visual C#- Open Visual Studio
After completing these steps, we can create a new project based on the Bot template.
As you can see, this is mainly a WebApi solution that has some custom attributes, with the MessagesController's Post method as entry point.
At this point, you can install the Bot Framework Emulator from here, run the application and start the emulator.
If we look at the code, we see that it simply returns the number of characters from the user input.
Since we are going to use Twitter data, we need to create and authenticate and a Twitter application.
In order to do this, go to the Twitter Application Management console and create an application.
After you create it, you will be able to find the consumer key, consumer secret, the access token access token secret that we will use in order to authenticate the calls to the Twitter API.
We will store the tokens Twitter generated for our app in order to use them when searching. Since we don’t want credentials in a Git repository, I am created a file called appSettings.config in which I placed the tokens from Twitter and the authentication tokens from LUIS (we will see this later).
<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
<add key="AppId" value="YourAppId" />
<add key="AppSecret" value="YourAppSecret" />
<add key="luisApplicationId" value="luisAppId"/>
<add key="luisSubscriptionKey" value="luisSubscriptionKey"/>
<add key="twitterAccessToken" value="twitterAccessToken"/>
<add key="twitterAccessTokenSecret" value="twitterAccessTokenSecret"/>
<add key="twitterConsumerKey" value="twitterConsumerKey"/>
<add key="twitterConsumerSecret" value="twitterConsumerSecret"/>
</appSettings>We are going to use a library called Linq2Twitter in order to make Twitter queries easier, and we will install int from NuGet.
At this point, we are ready to create the class that will be responsible for actually executing Twitter searches. At first, we will only execute un-intellingent queries, meaning that we will search Twitter exactly for the input from the user.
As I said, we are implementing a wrapper over Linq2Twitter that is simply going to get tweets based on the user query.
public class TwitterClient
{
private static TwitterContext TwitterContext { get; set; }
static TwitterClient()
{
TwitterContext = new TwitterContext(new SingleUserAuthorizer
{
CredentialStore = new InMemoryCredentialStore
{
ConsumerKey = ConfigurationManager.AppSettings["twitterConsumerKey"],
ConsumerSecret = ConfigurationManager.AppSettings["twitterConsumerSecret"],
OAuthToken = ConfigurationManager.AppSettings["twitterAccessToken"],
OAuthTokenSecret = ConfigurationManager.AppSettings["twitterAccessTokenSecret"]
}
});
}
public static string GetTweets(string query)
{
var search = TwitterContext.Search.Where(t => t.Type == SearchType.Search)
.Where(t => t.Query == query)
.SingleOrDefault();
return GetStringTweets(search.Statuses.Take(3));
}
private static string GetStringTweets(IEnumerable<Status> statuses)
{
string result = "";
foreach (var tweet in statuses)
result += tweet.ScreenName + "\n\n" + tweet.Text + "\n\n\n";
return result;
}
}And here is the Post method from the controller, with the HandleSystemMessage method unchanged.
[BotAuthentication]
public class MessagesController : ApiController
{
public async Task<Message> Post([FromBody]Message message)
{
if (message.Type == "Message")
{
return message.CreateReplyMessage(TwitterClient.GetTweets(message.Text));
}
else
{
return HandleSystemMessage(message);
}
}As we can see, we simply create the reply message using the string that contains the tweets received from the search.
I limited the results to 3 tweets because the message was rather large. You can have up to 200 tweets using the Search API. For more tweets, consider using the Streaming API.
When running the application and testing it with the Bot Framework Emulator, if we input Satya Nadella, we can see 3 tweets that satisfy the query:
Now let’s try to input some natural language and see how the system behaves: what does the world think about bill gates?
Let’s see what happened here: in the first query, the input was simply: satya nadella, so the TwitterClient just made a query for satya nadella.
The second time, the TwitterClient did the same thing, this time with a much more complicated query: what does the world think about bill gates?, query that returned no results from Twitter.
So far, we managed to create a bot that receives queries and makes Twitter searches based on the exact text of the query.
The goal is to have the user input natural language and for the system to extract the intention of the user, as well as the entities of the intention.
For example if a user had the following input: what does the world think about bill gates?, the system must understand that the user wants to do a Twitter search for bill gates.
For the following input: how is apple doing?, it should perform a Twitter search on apple.
One of the key problems in human-computer interactions is the ability of the computer to understand what a person wants, and to find the pieces of information that are relevant to their intent. For example, in a news-browsing app, you might say “Get news about virtual reality companies”,� in which case there is the intention to “FindNews”, and “virtual reality companies”� is the topic.
LUIS is designed to enable you to very quickly deploy an HTTP endpoint that will take the sentences you send it, and interpret them in terms of the intention they convey, and the key entities like “virtual reality companies” that are present.
LUIS lets you custom design the set of intentions and entities that are relevant to the application, and then guides you through the process of building a language understanding system.
Once your application is deployed and traffic starts to flow into the system, LUIS uses active learning to improve itself. In the active learning process, LUIS identifies the interactions that it is relatively unsure of, and asks you to label them according to intent and entities.
Before going any further, you should watch the tutorial from the LUIS Help section, since it contains all steps in creating a new LUIS application and how to setup entities and a training model.
When ready to start working with LUIS, create a free subscription here.
Then, go to the LUIS website and create a new app.
At this point, when you can see something like this, it means you are ready to define intents and start training your model.
Now we add an intent.
Then, we add an entity.
Now we can start training the model by feeding it queries and identifying the intent and the entities.
Since this is a machine learning model, at first you should feed it and classify enough queries (in my case I gave and identified 32 queries).
When we are satisfied with how our model responds to queries, we are ready to publish the model to be accessible programatically.
After hitting publish, we can actually test the model against queries and see how it handles them.
This is the result it gave us. You can see it correctly identified the intent as being GetTweets and the entity as satya nadella.
Now we need to make a C# client to query the LUIS model we just created.
We will create a new class called LuisResponse, we will copy the JSON from above and paste it as a class in Visual Studio.
After doing this, I placed each class in its own file and followed the .NET naming convention by using JsonProperty.
The Entity class
public class Entity
{
[JsonProperty(PropertyName = "entity")]
public string EntityName { get; set; }
[JsonProperty(PropertyName = "type")]
public string Type { get; set; }
[JsonProperty(PropertyName = "startIndex")]
public int StartIndex { get; set; }
[JsonProperty(PropertyName = "endIndex")]
public int EndIndex { get; set; }
[JsonProperty(PropertyName = "score")]
public float Score { get; set; }
}The Intent class
public class Intent
{
[JsonProperty(PropertyName = "intent")]
public string IntentName { get; set; }
[JsonProperty(PropertyName = "score")]
public float Score { get; set; }
[JsonProperty(PropertyName = "actions")]
public object Actions { get; set; }
}The LuisResponse class
public class LuisResponse
{
[JsonProperty(PropertyName = "query")]
public string Query { get; set; }
[JsonProperty(PropertyName = "intents")]
public Intent\[\] Intents { get; set; }
[JsonProperty(PropertyName = "entities")]
public Entity\[\] Entities { get; set; }
}Now we will create a class responsible for communicating with the LUIS HTTP endpoint and we will call it LuisClient.
Now you should grab your applicationId and subscriptionKey from LUIS in App Settings and place them in the appSettings.config.
public class LuisClient
{
private static string url = String.Format("https://api.projectoxford.ai/luis/v1/application?id={0}&subscription-key={1}", ConfigurationManager.AppSettings["luisApplicationId"], ConfigurationManager.AppSettings["luisSubscriptionKey"]);
public static async Task<LuisResponse> GetLuisResponse(string message)
{
string query = "&q=" + message;
using (var client = new HttpClient())
{
client.BaseAddress = new Uri(url);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
HttpResponseMessage response = await client.GetAsync(url + query);
return await response.Content.ReadAsAsync<LuisResponse>();
}
}
}This class is responsible for retrieving the intent and entities from LUIS based on a query. As you can see, the GetLuisResponse method accepts a message, then makes an HTTP request to your app’s endpoint, then formats the result as LuisResponse.
In the MessagesController, we will modify the result so it uses the intent from LUIS.
public async Task<Message> Post([FromBody]Message message)
{
if (message.Type == "Message")
{
LuisResponse luisResponse = await LuisClient.GetLuisResponse(message.Text);
return message.CreateReplyMessage((TwitterClient.GetTweets(luisResponse.Entities[0].EntityName)));
}
else
{
return HandleSystemMessage(message);
}
}Now let’s see how our app responds to some natural language:
Right now, we can add more stuff to our bot: we can add confirmation, remember the last query, build more complex queries, add more intents and entities and check them, stuff we will make in part 2 of this article.
We started by building a pretty straightforward bot: the user would put some exact phrase, then our bot would search Twitter for that phrase.
Then, we added support for natural language queries, meaning the user could input rather complex phrases, then, using LUIS, we would extract the intent and the entities from the phrase.
You just send and receive messages from the bot. No need to learn, understand and navigate disparate interfaces or languages. Users will be able to interact with bots just as they interact with other humans. It’s the most natural way to communicate and transact.
This phrase basically states exactly what a bot should be all about, and bots are becoming the new apps.
This way, we can build bots for the web, for Skype, Slack, SMS and many others.
For a step-by-step tutorial on how to deploy your bot and use it with the Bot Connector, check this resource.