• Stars
    star
    520
  • Rank 83,397 (Top 2 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created over 4 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

πŸ’­ Aspect-Based-Sentiment-Analysis: Transformer & Explainable ML (TensorFlow)

Aspect Based Sentiment Analysis

The task is to classify the sentiment of potentially long texts for several aspects. The key idea is to build a modern NLP package which supports explanations of model predictions. The approximated decision explanations help you to infer how reliable predictions are. The package is standalone, scalable, and can be freely extended to your needs. We sum up thoughts in the article:

Do You Trust in Aspect-Based Sentiment Analysis? Testing and Explaining Model Behaviors


There are over 100 repositories on GitHub around sentiment analysis 1 2 3 4 5 6 7 8 9 . All of them are hard to commercialize and reuse open-source research projects. We clean up this excellent research. Please give a star if you like the project. This is important to keep this project alive.


Quick Start

The aim is to classify the sentiments of a text concerning given aspects. We have made several assumptions to make the service more helpful. Namely, the text being processed might be a full-length document, the aspects could contain several words (so may be defined more precisely), and most importantly, the service should provide an approximate explanation of any decision made, therefore, a user will be able to immediately infer the reliability of a prediction.

import aspect_based_sentiment_analysis as absa

nlp = absa.load()
text = ("We are great fans of Slack, but we wish the subscriptions "
        "were more accessible to small startups.")

slack, price = nlp(text, aspects=['slack', 'price'])
assert price.sentiment == absa.Sentiment.negative
assert slack.sentiment == absa.Sentiment.positive

Above is an example of how quickly you can start to benefit from our open-source package. All you need to do is to call the load function which sets up the ready-to-use pipeline nlp. You can explicitly pass the model name you wish to use (a list of available models is below), or a path to your model. In spite of the simplicity of using fine-tune models, we encourage you to build a custom model which reflects your data. The predictions will be more accurate and stable.


Pipeline: Keeping the Process in Shape

The pipeline provides an easy-to-use interface for making predictions. Even a highly accurate model will be useless if it is unclear how to correctly prepare the inputs and how to interpret the outputs. To make things clear, we have introduced a pipeline that is closely linked to a model. It is worth to know how to deal with the whole process, especially if you plan to build a custom model.

The diagram above illustrates an overview of the pipeline stages. As usual, at the very beginning, we pre-process the inputs. We convert the text and the aspects into a task which keeps examples (pairs of a text and an aspect) that we can then further tokenize, encode and pass to the model. The model makes a prediction, and here is a change. Instead of directly post-processing the model outputs, we have added a review process wherein the independent component called the professor supervises and explains a model prediction. The professor might dismiss a model prediction if the model internal states or outputs seem suspicious. In the article [here], we discuss in detail how the model and the professor work.

import aspect_based_sentiment_analysis as absa

name = 'absa/classifier-rest-0.2'
model = absa.BertABSClassifier.from_pretrained(name)
tokenizer = absa.BertTokenizer.from_pretrained(name)
professor = absa.Professor(...)     # Explained in detail later on.
text_splitter = absa.sentencizer()  # The English CNN model from SpaCy.
nlp = absa.Pipeline(model, tokenizer, professor, text_splitter)

# Break down the pipeline `call` method.
task = nlp.preprocess(text=..., aspects=...)
tokenized_examples = nlp.tokenize(task.examples)
input_batch = nlp.encode(tokenized_examples)
output_batch = nlp.predict(input_batch)
predictions = nlp.review(tokenized_examples, output_batch)
completed_task = nlp.postprocess(task, predictions)

Above is an example how to initialize the pipeline directly, and we revise in code the process being discussed by exposing what calling the pipeline does under the hood. We have omitted a lot of insignificant details but there's one thing we would like to highlight. The sentiment of long texts tends to be fuzzy and neutral. Therefore, you might want to split a text into smaller independent chunks, sometimes called spans. These could include just a single sentence or several sentences. It depends on how the text_splitter works. In this case, we are using the SpaCy CNN model, which splits a document into single sentences, and, as a result each sentence can then be processed independently. Note that longer spans have richer context information, so a model will have more information to consider. Please take a look at the pipeline details here.


Supervising Model Predictions

It's time to explain model reasoning, something which is extremely hard. The key concept is to frame the problem of explaining a model decision as an independent task wherein an aux. model, the pattern recognizer, predicts patterns (weighted compositions of tokens, presented below) given model inputs, outputs, and internal states. Due to time constraints, at first we did not want to research and build a trainable pattern recognizer. Instead, we decided to start with a pattern recognizer that originates from our observations, prior knowledge. The model, the aspect-based sentiment classifier, is based on the transformer architecture wherein self-attention layers hold the most parameters. Therefore, one might conclude that understanding self-attention layers is a good proxy to understanding a model as a whole. Accordingly, there are many articles that show how to explain a model decision in simple terms, using attention values (internal states of self-attention layers) straightforwardly. Inspired by these articles, we have also analyzed attention values (processing training examples) to search for any meaningful insights. This exploratory study has led us to create the BasicPatternRecognizer (details are here).

import aspect_based_sentiment_analysis as absa

recognizer = absa.aux_models.BasicPatternRecognizer()
nlp = absa.load(pattern_recognizer=recognizer)
completed_task = nlp(text=..., aspects=['slack', 'price'])
slack, price = completed_task.examples

absa.summary(slack)
absa.display(slack.review)

absa.summary(price)
absa.display(price.review)

The explanations are only useful if they are correct. To form the basic pattern recognizer, we have made several assumptions (prior beliefs), therefore we should be careful about interpreting the explanations too literally. Even if the attention values have thought-provoking properties, for example, they encode rich linguistic relationships, there is no proven chain of causation. There are a lot of articles that illustrate various concerns why drawing conclusions about model reasoning directly from attentions might be misleading. In the article here, we validate and analyse explanations in detail.


Ready-to-Use Models

In the table below, we present the State of the Art results on the SemEval 2014 evaluation dataset (dataset details are here). There are two available models for the restaurant and the laptop domains. The model implementation details here. The hyper-parameters optimization (with the explanation how to train a model) is here. You can easily reproduce our evaluations, look at the performance tests here.

Model Name Acc Rest Acc Lapt Release
LCF-ATEPC [code][paper] 90.18 82.29 Jan 2020
BERT-ADA [code][paper] 87.89 80.23 Nov 2019
BAT [code][paper] 86.03 79.35 Feb 2020
classifier-rest-0.2 85.17
classifier-lapt-0.2 79.78

Installation

You can use the pip:

pip install aspect-based-sentiment-analysis

Otherwise, clone the code and create the new environment via conda:

git clone [email protected]:ScalaConsultants/Aspect-Based-Sentiment-Analysis.git
conda env create -f=environment.yml
conda activate Aspect-Based-Sentiment-Analysis

The package works with the Python in the version 3.7 (the same as in Colab 2021).


References

How to use language models in the Aspect-Based Sentiment Analysis:

  • Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence (NAACL 2019) [code][paper]
  • BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis (NAACL 2019) [code][paper]
  • Exploiting BERT for End-to-End Aspect-based Sentiment Analysis [code][paper]

Introduction to the BERT interpretability:

  • Are Sixteen Heads Really Better than One? [code][paper]
  • A Primer in BERTology: What we know about how BERT works [paper]
  • What Does BERT Look At? An Analysis of BERT's Attention [code][paper]
  • Visualizing and Measuring the Geometry of BERT [code][paper]
  • Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment [paper]
  • Adversarial Training for Aspect-Based Sentiment Analysis with BERT [paper]
  • Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT [paper]
  • exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models [code][paper]
  • Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings [code][paper]
  • Attention is not Explanation [code][paper]
  • Attention is not not Explanation [code][paper][blog post]
  • Hierarchical interpretations for neural network predictions [code][paper]
  • Analysis Methods in Neural NLP [code][paper]
  • Visualization for Sequential Neural Networks with Attention [code]
  • NeuroX: Toolkit for finding and analyzing important neurons in neural networks [code][paper]

The State of the Art results:

  • A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction [code][paper]
  • Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification [code][paper]
  • Adversarial Training for Aspect-Based Sentiment Analysis with BERT [code][paper]

Other interesting:

  • Multi-Dimensional Explanation of Ratings from Reviews [paper]
  • Extracting Syntactic Trees from Transformer Encoder Self-Attentions [paper]
  • Master Thesis: Transfer and Multitask Learning for Aspect-Based Sentiment Analysis Using the Google Transformer Architecture [code]
  • Create interactive textual heat maps for Jupiter notebooks [code]
  • A pyTorch implementation of the DeepMoji model: state-of-the-art deep learning model for analyzing sentiment, emotion, sarcasm etc [code]
  • More you can find here.

Developed by Scalac

More Repositories

1

reactive-rabbit

Reactive Streams driver for AMQP protocol. Powered by RabbitMQ library.
Scala
184
star
2

panopticon-tui

Terminal UI observability and diagnostic tool for Scala applications
Rust
107
star
3

zio-scala2-quickstart.g8

A Giter8 template for a fully functional, ready to deploy ZIO-based microservice.
Scala
91
star
4

websocket-akka-http

Websocket Client/Server with akka-http
Scala
74
star
5

mesmer

OpenTelemetry agent for Scala applications
Scala
71
star
6

akka-persistence-eventsourcing

Example project with a simple REST API to a domain model persisted using akka-persistence with eventsourcing.
Scala
48
star
7

zio-slick-interop

Slick interop for ZIO
Scala
37
star
8

scala-slack-bot-core

Akka based library for writing Slack bots in Scala.
Scala
33
star
9

zio-kafka-showcase

Example project that demonstrates how to build Kafka based microservices with Scala and ZIO
Scala
31
star
10

zio-scala3-quickstart.g8

A Giter8 template for a fully functional, ready to deploy ZIO-based microservice in Scala 3.
Scala
28
star
11

zio-akka-http-interop

akka-http interop for ZIO
Scala
28
star
12

docker-postgres-cluster

Shell
27
star
13

scala-slack-bot

Scala
22
star
14

akka-periscope

Akka plugin to collect various data about actors
Scala
17
star
15

zio-dotty-quickstart.g8

A Giter8 template for a basic Dotty application build using ZIO
Scala
17
star
16

spark-kafka-avro

POC: Spark consumer for bottledwater-pg Kafka Avro topics
Scala
17
star
17

lagom-scala-post-example

Scala
11
star
18

wsug-akka-websockets

source code for Warsaw Scala Enthusiasts group meeting
Scala
8
star
19

akka-message-visualization

Proof of concept of a visualization of message flow within an Akka system
JavaScript
8
star
20

monix-blog-examples

Scala
7
star
21

akka-streams-graph-stage

Scala
7
star
22

rough-slick

Scala
6
star
23

Tezos-FullStack-Console-backend

Scala
4
star
24

scala-spark-ml

Machine learning using Spark and Scala
Scala
4
star
25

WhoSaidThat

Demo for Spark 101 post
Scala
3
star
26

Tezos-Micheline-Michelson-Translator-Frontend

TypeScript
3
star
27

akka-http-s3-upload

Experiments on different ways of uploading files to S3 using combinations of Akka HTTP directives, AWS SDK and Alpakka.
Scala
3
star
28

json-tools

Tools for JSON handling
Scala
3
star
29

macro-fun

Code accompanying the macro post, including polynomial differentiation.
Scala
3
star
30

vhs-data-analysis

Scala
2
star
31

zio-introduction

ZIO introduction workshop
Scala
2
star
32

ScalaWave2017ScalaBasic

JavaScript
2
star
33

panopticon-example

A sample Scala app, fully equipped to be monitored with https://github.com/ScalaConsultants/panopticon-tui
Scala
2
star
34

recru-app

Awesome recruitment app for the IT folks
JavaScript
2
star
35

newspaper

Scala
1
star
36

scala2-introduction

Scala 2 introduction workshop exercises
Scala
1
star
37

warsjawa-2014

Mobile app for the conference
Java
1
star
38

galaxy-gear2-tutorial

Galaxy Gear2 basics tutorial
Java
1
star
39

devoxx-android

Java
1
star
40

docker-java8

1
star
41

scalagram

Instagram's clone to share memes about Scala world
Scala
1
star
42

endpoints4s-vs-tapir-blog-examples

1
star
43

zionomicon

Scala
1
star
44

whisky-tango-foxtrot

Scala
1
star
45

reactive-slick

Scala
1
star