“A man has a property in his opinions and the free communication of them” – James Madison.
Every day, millions of consumers share their opinion on a range of things, including feedback about their experiences with products and services. This volunteered feedback contains raw, unsolicited views and opinions about a brand, product, or service.
In a customer-centric culture, it is more important than ever to make use of intelligent technologies that can make sense of this data, extract insights about customers and create strategies based on these valuable insights. Organizations at the forefront of customer experience and social customer service are already making use of such techniques. By tapping into the universe of unstructured opinion data, these firms are enabled to find and prioritize their most valuable customer interactions. Such techniques can be used to improve retention & acquisition rates, if not deliver a superior customer experience.
Let’s start with a simple example: a customer tweets online about a hotel review, “I loved the hotel location but was disappointed my room wasn’t ready on arrival”. As we try to make sense of this text, we attempt to understand how the consumer feels about the topic at the broadest level. Text Analysis algorithms typically determine the overall polarity by detecting the presence of certain keywords (love, hate, dislike, etc.) and then classifying it into one of three buckets: positive, negative, or neutral. What if there are multiple polar attributes in the same text? The truth is, understanding language is not always straightforward.
What is Opinion Mining?
Last weekend I ran a poll to ask people what they thought the underlying sentiment was on the same example “I loved the hotel location but was disappointed my room wasn’t ready on arrival”. It was evident from the results and the comments that when the opinion expressed is multi-layered, it can get tricky to attach an overall sentiment. What we’d need is to drill down to the drivers and understanding why certain behaviors occur.
Traditionally sentiment analysis has helped understand how individuals feel about a given subject - favorable or unfavorable. Welcome the new kid on the block 😎- Opinion Mining which goes a step further, to explain why individuals feel the way they do, by showing what is driving the sentiment.
Opinion Mining or Aspect-based sentiment analysis (ABSA) is the science of using text analysis to understand the drivers of public sentiment. It categorizes data by aspect and identifies the sentiment attributed to each one. Finding the precise, subject-level drivers behind the negative and positive emotions, uncovers the exact territories of strength and deficiency. 🤔Imagine if our public sector agencies could engage such techniques to understand the precise areas of improvement and model campaigns around the electorate’s evolving needs! For once, they may be able to see what people truly think and feel.💡
Why should I mine (or mind) the opinions?
Opinion mining is clearly making waves and is stepping ahead of its predecessor Sentiment analysis, in more ways than one-
- Individual opinions could reflect a broader viewpoint. Even a single customer who takes up an issue with a brand on social media likely reflects the opinions of several others. When enough opinions are gathered and analyzed accurately, you can better gauge the feelings of the audience.
- Knowing which battle to fight, which issue to target – and why – is key. When mapped to customer journeys, it helps organizations to understand where they are succeeding or failing with customer experience.
- More personalized content for audiences. Previously the market was segmented based on age, sex, salary, and other surface stats. Now companies can additionally segment based on NPS-how their visitors personally think about the brand; allowing for tailor-made experiences for users.
- As online networking gets rampant, more expressive, and emotive data is gathered across multiple streams- Internal Data, Surveys, NPS (Net Promoter Score) software, Customer Service and Customer Relationship Management (CRM) Software, External Data (social media, news articles, product reviews, etc.)..whew! Opinion Mining provides broader insights into this gathered data.
-
Given the sheer volume of minable text, you will need help to carry out a large part of the deciphering work. The advancements in AI and ML capabilities can be harnessed to yield quicker results.
How to mine opinions?
To obtain more granular insights from your data, Opinion Mining must assign sentiments to specific topics. And to do so, it breaks down the data into smaller fragments. Let us go back to our original example: “I loved the hotel location but was disappointed my room wasn’t ready on arrival.” In cases like this, there is more than one sentiment and more than one topic in a single sentence, so to label the whole review as either positive or negative may not be correct. Opinion Mining extracts and separates each aspect and sentiment polarity in the sentence. In this instance, the aspects are Location and Room, resulting in the following sentiment attribution: The location was great (Location → Positive) but the room was disappointing (Room → Negative).
Opinion Mining uses (NLP) Natural Language Processing techniques to parse unstructured data and make sense of the human language. These include simplifying words (lemmatization and stemming), analyzing grammar (parsing), and understanding the contextual meaning of a word, etc. But you do not have to worry about all of that. Thanks to Azure AI’s democratized capabilities- developers can get started with Opinion Mining today. And before you ask - to do so, you would require Zero machine learning expertise!
Using the Text Analytics client library and REST API you can build applications to leverage these powerful AI capabilities. In fact, this makes a perfect case for a Conversational AI agent or Virtual Assistant that entertains user interactions in natural language. The ability to understand a user’s emotions and the specifics thereof can be leveraged in personalizing the conversational experience and even triggering human agent hand-off. What say we build an emotionally intelligent chatbot? 🤖
Using the Visual Studio IDE, create a new Echo Bot (Bot Framework v4-.NET Core 3.1) app. This will provide a good template to start with. You can build and run the solution as-is and test it in the Bot Emulator.
The Echo bot typically echoes back whatever the user inputs, but we’d like for it to analyze the emotion behind the user’s text. So let's roll.Install the Azure.AI.TextAnalytics client library by right-clicking on the solution in the Solution Explorer and selecting Manage Nuget Packages -or- using the Package Manager Console. Check the include prerelease box, select version 5.1.0-beta.5, and then Install.
-
In the Solution Explorer, find the EchoBot.cs file under the folder named Bots. Add the following using directives.
using Azure; using System; using System.Globalization; using Azure.AI.TextAnalytics; -
Create a Text Analytics resource in Azure. Once created, you can find your key and endpoint in the resource's key and endpoint page, under resource management. (In a Prod environment, consider using a secure way of storing and accessing your credentials e.g. Azure Key Vault) In EchoBot.cs, create variables for your resource's key and endpoint.
private static readonly AzureKeyCredential credentials = new AzureKeyCredential("ADD_YOUR_TEXT_ANALYTICS_KEY_HERE"); private static readonly Uri endpoint = new Uri("ADD_YOUR_TEXT_ANALYTICS_ENDPOINT_HERE"); The conversational logic, especially logic specific to when a user’s Message is received, is present in the OnMessageActivityAsync function. In response to the user’s input, the EchoBot currently echoes back what it hears. Since we're repurposing it and improving its Emotional Quotient, let’s rename the file to OpinionMiningBot.cs.
-
Create the TextAnalyticsClient object with your endpoint and credentials, created in the previous step. It authenticates to Azure using your key and provides functions to accept the text as single strings or as a batch.
var client = new TextAnalyticsClient (endpoint, credentials); -
Add a new method called ClassicSentimentAnalysis() that takes the client that you created earlier and calls its AnalyzeSentiment() function. It will return a DocumentSentiment object containing the overall sentiment score for a document or sentences therein.
static DocumentSentiment ClassicSentimentAnalysis(TextAnalyticsClient client, string inputText) { DocumentSentiment documentSentiment = client.AnalyzeSentiment(inputText); return documentSentiment; } -
Invoke the method ClassicSentimentAnalysis() within OnMessageActivityAsync()
DocumentSentiment documentSentiment = ClassicSentimentAnalysis(client, turnContext.Activity.Text); foreach (var sentence in documentSentiment.Sentences) { var replyText = $"Sentiment: \"{sentence.Sentiment}\""; replyText += $"\nPositive score: \"{sentence.ConfidenceScores.Positive:0.00}\""; replyText += $"\nNegative score: \"{sentence.ConfidenceScores.Negative:0.00}\""; replyText += $"\nNeutral score: \"{sentence.ConfidenceScores.Neutral:0.00}\""; await turnContext.SendActivityAsync(MessageFactory.Text(replyText, replyText), cancellationToken); }Run the Solution from VS IDE and connect the emulator to the bot running locally.
The overall Sentiment score detected was Negative. If this was a larger text or document this result may not have been super useful as we don’t really know what aspect drove the Negative sentiment. Let’s take this a step further. -
Add a method called SentimentAnalysisWithOpinionMining() that takes the Text Analytics client, and call it's AnalyzeSentimentBatch() function to return a collection of AnalyzeSentimentResult. The key is to set the IncludeOpinionMining option to true, in the AnalyzeSentimentOptions bag.
static AnalyzeSentimentResultCollection SentimentAnalysisWithOpinionMining(TextAnalyticsClient client, List<string> documents) { AnalyzeSentimentResultCollection reviews = client.AnalyzeSentimentBatch(documents, options: new AnalyzeSentimentOptions() { IncludeOpinionMining = true }); return reviews; } -
Invoke the method SentimentAnalysisWithOpinionMining() within OnMessageActivityAsync().
AnalyzeSentimentResultCollection reviews = SentimentAnalysisWithOpinionMining(client, new List<string> { turnContext.Activity.Text }); foreach(AnalyzeSentimentResult review in reviews) { foreach (SentenceSentiment sentence in review.DocumentSentiment.Sentences) { var sentenceText = $"Sentence Sentiment: \"{sentence.Sentiment}\""; sentenceText += $"\n\nPositive score: \"{sentence.ConfidenceScores.Positive:0.00}\""; sentenceText += $"\n\nNegative score: \"{sentence.ConfidenceScores.Negative:0.00}\""; sentenceText += $"\n\nNeutral score: \"{sentence.ConfidenceScores.Neutral:0.00}\""; await turnContext.SendActivityAsync(MessageFactory.Text(sentenceText, sentenceText), cancellationToken); foreach (SentenceOpinion sentenceOpinion in sentence.Opinions) { var sentenceOpinionText = $"\n\nTarget: {sentenceOpinion.Target.Text}"; foreach (AssessmentSentiment assessment in sentenceOpinion.Assessments) { sentenceOpinionText += $"\n\nRelated Assessment: {assessment.Text}, Value: {assessment.Sentiment}"; sentenceOpinionText += $"\n\nPositive score: {assessment.ConfidenceScores.Positive:0.00}"; sentenceOpinionText += $"\n\nNegative score: {assessment.ConfidenceScores.Negative:0.00}"; await turnContext.SendActivityAsync(MessageFactory.Text(sentenceOpinionText, sentenceOpinionText), cancellationToken); } } } }Firing up the Bot Emulator for the same input. Check this out..
The good news is that in addition to the overall sentiment, we obtain deeper insights on each analyzed target and their related assessment(s).
The better news is that the entire source code is available in Github repository for you to test your own dataset.
Emotionally Intelligent Bot
This repository contains samples to demonstrate Opinion mining - a feature of Sentiment Analysis. Also known as Aspect-Based Sentiment Analysis (ABSA) in natural language processing (NLP), this feature provides more granular information about the opinions related to aspects (such as the attributes of products or services) in text.
What's Next?
We are only scratching the surface in terms of the possibilities of Opinion Mining in interpreting customer feedback, streamlining, and prioritizing customer support. The next stage for sentiment analysis is thought to be in Video Analysis- to assess spoken language, for example in a service center (pitch changes, volume level, voice overlap to detect speakers interrupting each other). But to start with, opinion mining can analyze video chats.
Opinion Mining techniques have been improving over the years, and experts in the field have overcome many challenges associated with understanding the nuances of human language. But some challenges are relevant even today e.g. understanding sensitivity to context, humor, slang, innuendo, sarcasm, colloquialisms, figures of speech, understanding industry jargon, etc. Apparently, machines are confounded by the very things that make human language lively and humorous.😉
Conclusion
It is said that the greatest fear in the world is of the opinions of others. But if the customer is the king, the king’s opinions matter. Opinion mining is gaining traction as a valuable tool to better customer experiences as it analyzes customer interactions at every level, finding out what makes customers tick and what causes customer churn. These latest capabilities, as part of Azure AI’s Text Analytics features, help capture to pinpoint exactly what customers like and dislike. Now you know what they feel and more so, why they feel what they feel.
Happy mining!
The comment section awaits your opinions🙂
Top comments (0)