Dive into Sentiment Analysis with Opinion Mining

If you run (or a part of) a business then you must have customers and a big number of those customers are driven by the customer service. Good customer service acts as a catalyst for your customer retention and helps you get new customers, absolutely (or almost) free of charge. Not only this, the positive testimonials / reviews of your customers are proven to be more effective than your usual marketing efforts. Contrary to this, if your customers experience something bad, they're probably going to express more quicker than those who had the positive experience. Hence, customer service can make or break your business.

Millions of customers express their opinions about range of things, every day. If you're in retail business then you may experience these reviews more frequently than any other line of business. As this unstructured, raw data grows, it gets more difficult to analyze. Therefore, to address such issues, natural language processing technique (NLP) comes into play. NLP is a branch of Artificial Intelligence which allows machines to break down and interpret human language. One of the important applications of NLP is Sentiment Analysis.

Sentiment Analysis

It is a process of detecting positive or negative sentiment in the utterance (or text). It gives you an insight of what your customer is talking about, whether they are loving or hating your products or services. Sentiment Analysis is able to recognize the nuances in emotions and opinions from the real-time feed of social media or within the transcript of your call center. This can give you an overall sense of how customers feel about your offerings.

Today, there are bunch of tools available to perform sentiment analysis for you. However, as a part of AI For Every Developer series, our focus is more on the Azure Cognitive Services (ACS). In ACS, Sentiment Analysis can be performed using the Text Analytics at a sentence or document level. In return, you get positive, negative or mixed labels. For example, for below sentences, you will get these results with their scores in response when you make a query to Text Analytics, Cognitive Service.

In all above examples, you get to know your customer's fine grained sentiment. When it's negative (or depending upon the your choice of threshold, if it's mixed) then you can either analyze the feedback / review in detail by assigning it to your back office team for more investigation. Usually, this is happening with the social media analysis, surveys or general feedback.

Nowadays, customer reviews have got so much information in them that a mere positive, negative labels or even their respective scores are not completely helpful. I am not saying that they do not add value today but you still have to dig yourself into the review, manually, to learn more about it. 

Wouldn't it have been better if we had some capability to get some more meaningful information from these reviews? Like an opinion of the customer, an actionable insight or the aspects with the opinions on them. Well, that's what we're going to learn here today.

Opinion Mining

If you're not super familiar with the updates in Azure Cognitive Services; the product team has introduced the Opinion Mining ^Preview feature as a part of Text Analytics API v3.1 but before we get into the implementation part; let's learn a bit more about Opinion Mining.

Sentiment Analysis examines how people feel about your products or service (ranging score from 0 to 1 / positive to negative) whereas Opinion Mining goes a little deeper into conversation and understand why people feel that way. Basically, Opinion Mining is a technique that categorizes data by aspect (attributes / nouns) and identifies the sentiment (adjective) associated to each aspect. That's why it is also known as Aspect-based Sentiment Analysis.

Now let's analyze the 4th feedback from the table above. If you notice closely, this feedback has an overall positive sentiment which is around 68% but there's some important information in this feedback which may get missed if we do not use Opinion Mining and that's its last sentence. Desserts (aspect) were delicious (positive sentiment) but a little too sweet (negative sentiment) for us. That's the beauty of Opinion Mining.

How do I use it?

As I mentioned above that it's a part of the Text Analytics API therefore, you can either use REST API to call this or use the client library (C#, Python, JavaScript or even Java).

1. If you're using a REST API, all you have to do is to pass opinionMining = true flag in the request.
2. For client library (C#), you just have to set the AnalyzeSentimentOptions options with IncludeOpinionMining = true. For rest of the languages, you can follow this Quickstart guide.

There are multiple use-cases where Opinion Mining can be super useful. Such as customer feedback, market research, brand and social media analysis to name a few.

In this blog post, I've applied a generic approach and created an Azure Function for you. As I've kept it opensource so feel free to download and use in your own projects. All you have to do is to change the endpoint URL and API Key.

For the sake of this demo, I am using Postman to show how you can get the responses back with the opinions.

So that's how you can simply get the opinions along with the sentiments from your customers feedback, just with the feature flag. In case you get stuck somewhere or need more information on this, please feel free to comment or connect with me on Twitter and I'll be happy to help you.

Until next time.