Published June 1, 2018 by Grapple Data

Selecting the Best Sentiment Analysis Software

Are you currently interested in using Artificial Intelligence to enhance the automation of your key workflows? Are you thinking it could help your business perform the highly valuable task of detecting and monitoring customer and partner sentiment? Are you sometimes frustrated at how long it takes the business to respond to market conditions and customer/partner sentiment? Would you like your business to be much more strategically agile?

You aren't alone. More and more businesses are beginning their foray into using artificial intelligence to mine text documents, customer support requests, user feedback, and forum posts. Quite a few businesses are taking the critical step of implementing their first pilot projects with some of the biggest software provider names in this space, but the results have been pretty mixed. It's is all too common to hear complaints that things have not gone quite as well as hoped. Many first adopters have been left with the realization that there is still a good deal of hype in this area that has gone largely unrealized. It turns out that building good Natural Language Processing software is not as easy as just throwing a bunch of text data at some Machine Learning algorithms that will magically understand human writing. This is especially true with the complexities involved in automated sentiment analysis.

Although we humans naturally perform sentiment analysis on a daily basis, we still quite often misinterpret each other, so it would probably be unreasonable to expect software to be any better than us. However, most sentiment analysis software can't come anywhere close to the accuracy and precision of a human being in this regard yet. It's pretty easy to build simple software that can detect basic phrases in text that denote GENERAL sentiment (e.g. "I like this!", "That's great!", "I'm not happy", etc.). It's also pretty easy to assign some points or weight to the detected phrases and come up with an overall sentiment score metric. It's even pretty easy to apply this sentiment score to all the entities in the immediate proximity (assuming you can faithfully detect those entities). However, what's not so easy is to build software that is highly ACCURATE, PRECISE, TARGETED, and SPECIFIC.

Take for example this short sentence:

"I think the ABC product is really superior to the XYZ product."

The sentiment here is detected from the word "superior". We have two entities in close proximity, ABC and XYZ. Is the sentiment toward these positive or negative? As native English speakers it's obvious to us that ABC gets positive sentiment while XYZ gets a less than positive or negative sentiment. Saying one thing is better than another doesn't inherently imply that the other thing is bad. However, we understand that ABC should get much greater positive sentiment and that it's possible there is some negative sentiment toward XYZ, because a comparison is taking place. Obviously it would be awesome if your sentiment analysis software could be this accurate and precise in its results, but there are some key features that it needs to have to make this a reality.

First, it needs to be able to effectively disambiguate the type of sentiment that is detected in the text. Next, it needs to be able to detect where each sentiment is being directed. It must take into consideration and understand appropriate application of negation, affirmation, exclusion, inclusion, and comparative and contrasting statements, and do all of this with a large degree of accuracy. Otherwise, that big warm fuzzy sentiment score number you see in your dashboard can turn out to be very misleading or wildly incorrect. It can also lead to a lot of false positives alerts that actually result in MORE MANUAL WORK to resolve, or false negatives that cause you to miss critical events and scenarios.

Hopefully inaccuracies generated by the AI aren't catastrophic to your customers, partners, or your bottom line (recent news reports of self-driving vehicles running into pedestrians, road barriers, and other vehicles come to mind). Obviously, those kinds of results defeat the purpose and can easily become Business Disinformation rather than Intelligence.

Unfortunately, after taking a test drive, many businesses are realizing that the sentiment analysis they have implemented is not turning out to be as useful as they thought it might be. Rather than being immediately practical, the results frequently tend to be more general, ambiguous, high level, fuzzy, or academic in nature, still requiring humans to get heavily involved in the disambiguation, noise reduction, and interpretation process. At this point it should be no big surprise that frequently the next big ask is how to make the software more ACCURATE, PRECISE, TARGETED, and SPECIFIC, so that it's actually useful and actionable?

Knowing that your customers are generally 75% happy is not as useful as knowing with a high degree of precision that customers are 75% happy with product ABC, but also 68% of your customers dislike feature XYZ of that same product, and prefer the XYZ feature in your competitors product. Also, wouldn’t it be great if when a customer opens a support ticket containing negative sentiment (complaints), you also can see that the negative sentiment is directed at the sales rep who hasn’t called them in the last three months? Those are the types of results we are looking for, but the problem isn’t so simple, otherwise everyone would be able to do it already.

Written language is hugely complex. The software must understand key aspects of the specific language. It must know the precise textual features to look at and examine, understand their meaning/semantics individually, and also be able to understand how their interpretation may change within specific contexts. On top of this, it must also be able to deal with human variability. Humans express things in so many different possible ways there is no one standard for the textual representation of ideas and concepts, and these are changing/evolving all the time. Add to this things like misspellings, typos, grammatical errors, or the unexpected grammar that may come from a non-native speaker/writer, and you've got a cocktail of immense complexity.

Consider just these few small “basic” examples that can represent a very difficult problem to many sentiment analysis systems:

  • "Do I think that ABC is awesome? Well, now that I think about it, I really don't."
  • "Do I think that ABC is awesome? Sure, well, not really."
  • "ABC is not a great product, but you know what? XYZ is a different story."
  • "I really like ABC but XYZ not so much."
  • "ABC is awesome as opposed to XYZ."
  • "ABC is horrible compared to XYZ."
  • "ABC is a premium product. That applies to XYZ as well."
  • “In terms of its BLAH feature, ABC blows away XYZ."

Try running those examples through your current sentiment analysis suite of software, or any that you are currently considering, and see what happens. Good luck! For example, try going to the google text analytics demo located here:

Google Natural Language Processing

... and input some text like:

"Do I love my new phone? After having put it through the paces... no not really."

Note whether it able to detect that the sentiment toward the phone is actually negative? Additionally, you can check out the IBM Watson demo here:

IBM Watson Natural Language Processing

... and input some text like:

"I'm thinking California is a better place for us than Colorado."

Again, take a look at the results. Does Watson detect both positive and negative sentiment? Does it assign the positive to California and the negative to Colorado, or does it just split it ambiguously across the two entities?

At this point you may be feeling discouraged from using automated tools for Natural Language Processing and Sentiment Analysis. However, that isn't the objective of this article. Just because a problem is huge, and many of the biggest names haven't done a great job of solving it, doesn't mean that quality solutions don't exist.

For example, Grapple Data Technology has been doing deep, focused, specialized work on this problem for many years. This year, we released the AKIN Natural Language Processing API, which can produce highly ACCURATE, PRECISE, TARGETED, and SPECIFIC results that are immediately useful to your business.

We have several public interactive online demos that you can play with to get a good sense for how well it works in comparison to others, and the features that differentiate it from the rest. Good sentiment analysis via AI has just gone from hype to hyper-good. The AKIN NLP API delivers the stellar Sentiment Analysis results you expect, but also provides a large number of other Natural Language Processing features. We invite you to come discover for yourself:

AKIN Natural Language Processing Demos

Want more information, or just want to let us know what you think? You can contact us here:

info@grappledata.com