What is Tensorflow?

Tensorflow is one of the most popular tools for developing neural networks. Tensorflow is not a library for making neural networks per se. Rather, Tensorflow is for efficiently handling the mathematic operations that neural networks rely on. ANNs utilize a lot of linear algebra, matrix operations and node-edge graph manipulations. Tensorflow provides a set of functionality to accomplish these operations efficiently with C++ code and allows the developer to use these functions with a python interface.

What is Keras?

As discussed, Tensorflow itself does not directly provide neural network functionality. Instead, the programmer who wants to develop an ANN will need another library. Keras provides programmers a way to declare what kind of ANN they want and then Keras accomplishes the interfacing with Tensorflow to generate the network. Tensorflow focusses on accomplishing the mathematics efficiently, while Keras provides programmers an easy way to develop their models. Although Keras and Tensorflow are very often used together, they are actually independent. Keras can be used to interface with an alternative to Tensorflow, like Theano. Similarly, developers can use an alternatives to Keras, like TFLearn and still use Tensorflow in the backend.

The predictive features (X)

Our X value is a matrix of floats where each row is a flower and each column is a measurement.

pd.DataFrame(X,columns = feature_names).head(5)

The dependant (y)

For each row in X, there is a corresponding item in the vector y, which represents the species of flower. There are three unique species in our dataset as shown in the table below. However, our dependant y must come in the form of a One Hot Encoded (OHE) matrix, not a vector. In this format, there are three columns with a zero or one value to indicate which species of flower is this subject.

pd.DataFrame(y[:5],columns = dataset.target_names)

Let us inspect this same data when converted to an array, this will be used to ‘flatten’ the matrix form of y to a vector. Each row is a ‘setosa’ (0).

y[:5].argmax(axis=1)

array([0, 0, 0, 0, 0], dtype=int64)

Terminology

• Batch size: This parameter instructs the ANN to trial that number of cases with a set of random weights before trialing another batch with a new set of weights. A large batch size enhances computational efficiency, at the potential cost of accuracy.

• Epochs: The number of times the ANN shall repeatedly use the training data to find the optimum set of weights. A large number of epochs may increase accuracy, but takes longer to compute.

• Optimizer: In the fitting process, random weights are trialed for every neuron and the best combination is selected. However, trying every possible random combination is very inefficient. Instead, optimizers apply algorithms that aim to discover which combinations of parameters for each node is best. Instead of randomly selecting weights, the optimizer applies a guided process to discovering the best settings. These algorithms for determining the optimum weights are called ‘gradient descent’ algorithms.

• Learning rate: The incremental value by which the optimizer varies weights. A larger learning rate may be quicker to compute, but less accurate.

• Loss Function: During the fitting process, the loss function is how we calculate if a given trial was better or worse that others. Here we use ‘Binary Cross Entropy’ which is a common selection for classification problems. Mean-squared error would be an appropriate choice for regression problems.

• Metrics: Machine learning practitioners know there are many ways to evaluate a model’s performance. We can record multiple metrics to evaluate performance. For example, we can examine the model’s accuracy (the portion of correct predictions) and the recall (the portion of correctly classified cases of a given class among all cases of that class).

def get_simple_ann(
        input_shape: int|tuple,
        output_shape:int|tuple
        ) -> keras.models.Sequential:
    """
    This function returns an instance of a 
    Keras Sequential model which is ready to 
    fit with training data.
    
    Args:
    input_shape: int or tuple. Describes the 
    dimensions of a single case of input.
    output_shape: int or tuple. The desired 
    shape of output. 
    
    Returns:
    A compiled instance of a keras Sequential model
    """

    ### Step 1: Instantiate an empty Sequential model object
    ann = keras.models.Sequential()
    
    ### Step 2: Adding the input layer with same number 
    # of nodes as we have input features
    ann.add(
        keras.layers.Dense(
            units = 16,
            activation='relu', 
            input_dim=input_shape
            )
        )
    
    # Step 3: Adding 10% dropout to prevent overfitting
    ann.add(keras.layers.Dropout(rate=0.1))
    
    ### Step 4: Adding another Dense-Dropout combo to 
    # make a second layer in the network
    ann.add(
            keras.layers.Dense(
            units = 16, 
            activation='relu'
            )
        )
    ann.add(keras.layers.Dropout(rate=0.1))
    
    ### Step 5: Adding the output layer. This must match the 
    # format of our y variable
    ann.add(
            keras.layers.Dense(
            units = output_shape, 
            activation='softmax'
            )
        )
    
    ### Step 6: Once we have defined the architecture of 
    # our ANN, we call compile() to convert the model to tensors.
    ann.compile(
        optimizer='adam', 
        loss='binary_crossentropy', 
        metrics=['accuracy']
        )
    return ann

### Calling our function
simple_ann = get_simple_ann(
    input_shape = input_shape,
    output_shape = n_classes
    )

### Fitting to training data
simple_ann.fit(
    X_train,
    y_train,
    batch_size=100, 
    epochs=150,
    verbose=False
    )

### Making predictions
# argmax() here is used to 'flatten' the matrix form of the 
# dependant variable (as seen above) into a vector of integers 
# which correspond to classes. 
predictions = simple_ann.predict(X_test).argmax(axis=1) 

### Accuracy score
accuracy = accuracy_score(y_test.argmax(axis=1),predictions)
print("Accuracy of this model is: "+"{:1.2f}".format(accuracy*100)+"%")

2/2 [==============================] - 0s 0s/step
Accuracy of this model is: 70.00%

Visualizing the ANN

From the code above, we are constructing an ANN like the one in this diagram. We have four features in our data, and so we have four input nodes to receive each value. If you know the theory of ANNs, you know that each node will apply a weight and an activation function to the input to determine the number it will output to every node in the next layer. However, we use a dropout layer, which will select some nodes to nullify. We have two such layers. Finally, we have three output nodes, representing the probability that the given case is of a certain species. Therefore, for each case we predict, we output three numbers. We conclude that whichever of those three numbers is largest corresponds to the most likely species for the given case.

Evaluating the model

As we can see from the output above, this model was 70% accurate, meaning it classified 70% of cases as the correct species.

Why we need hypertuning

Is 70% accuracy the best we can do? What if we had more than 12 nodes in each layer? Or fewer? What if we added more layers? Or increased the dropout rate? Hypertuning is how we experimentally verify which parameters are best.

To accomplish this, we could try repeatedly fitting the model with every possible combination of parameters (this is called ‘Grid Search’). However, for many parameters, this would require a lot of fittings. Instead, the ‘keras tuner’ library was created to efficiently trial different hyperparameter selections.

Terminology

• Objective: One of the metrics. The ‘best’ model is the one with the highest value.
• Hypergrid: Is a dictionary-like structure that specifies what values for each parameter to trial.
• Callbacks: In this context, the callback is a special type of function called at the end of each epoch. In this example, we use ‘EarlyStopping,’ which is helpful for reducing computation time. With early stopping, once each additional epoch only improves the objective slightly, we conclude we have reached a point of diminishing return and finish the fitting process without completing every epoch.
• Width: The number of units is a particular layer.
• Depth: The number of layers in an ANN.
• Validation split: By default, the tuner selects the model with the best value for the objective as calculated on training data. However, we can optionally specify a validation split size, whereby a random sample of training data will be withheld, and we select the best model as validated against that holdout data. This can be helpful in avoiding overfitting to training data.
• Activation function: a function which each node in a layer uses to convert the inputted values into the outputted values. A sigmoid function would be appropriate for the activation function of an output node doing classification, while no function would be appropriate for a regression. For a discussion on the different types of common functions, see this article.

class ANNHyperModel(kt.HyperModel):
    """
    Given a set of hyperparameters to explore, 
    uses keras-tuner to discover the best settings 
    and returns the optimum model.
    """
    
    _default_hypergrid = {
            'width':{
                'min':12,
                'max':12,
                'step':1
                },
            'depth':{
                'min':1,
                'max':1,
                'step':1
                },
            'droprate':{
                'min':.8,
                'max':.8,
                'step':.1
                },
            'activation':{
                'values':['relu']
            },
            'learning_rate':{
                'values':[1e-2]
            }
        }
    """If the user does not input values for the 
    following hyperparameters, we use these values by default."""
    
    def __init__(
                self,
                input_shape:tuple|int,
                n_outputs:int,
                hypergrid:dict = None,
                output_activation:str = 'sigmoid',
                epochs:int = 1,
                batch_size:int = 100,
                optimizer_class = keras.optimizers.Adam,
                loss: Loss = SparseCategoricalCrossentropy(
                                        from_logits=True
                                        ),
                metrics: list[str] = ['accuracy'],
                objective:str = "accuracy",
                directory:str = None,
                project_name:str = "kt_hyperband",
                factor:int = 3,
                verbose:int = 0,
                callbacks:list = []
    ):
        """
        Args:
        - input_shape (tuple | int): Dimensions of a single input case. 
        Provide an int of n if one case is a 1-d array of length n.

        - n_outputs (int): Number of output nodes. One for regression or 
        binary-classification. Multiple for 
        multi-regression/multi-classification.

        - hypergrid (dict, optional): Dictionary describing hyperparams 
        to trial, see default '_default_hypergrid' for example. 
        Defaults to None.

        - output_activation (str, optional): Output layer activation 
        function. Defaults to 'sigmoid'.

        - epochs (int, optional): Training epochs. Defaults to 1.

        - batch_size (int, optional): training batch size. 
        Defaults to 100.

        - optimizer_class (_type_, optional): Optimizer class. 
        Input the class type, not an instantiation. Defaults to 
        keras.optimizers.Adam.

        - loss (tf.keras.losses.Loss, optional): training loss function. 
        Defaults to 
        keras.losses.SparseCategoricalCrossentropy(from_logits=True).
        
        - metrics (list[str], optional): Training metrics. Defaults 
        to ['accuracy'].
        
        - objective (str, optional): Must be included in 'metrics.' 
        Tuning process will select the model with the best performance
        by this metric. Defaults to "accuracy".
        
        - directory (str, optional): When used, tuning process creates 
        a directory to hold data on disk. Deleted afterwards. 
        Be careful not to use existing dir name. Defaults to "None".
        
        - project_name (str, optional): File naming stem in directory. 
        Defaults to "kt_hyperband".

        -factor (int, optional): Reduction factor for epochs at each 
        tuning trial. Defaults to 3.

        -verbose (int, optional): Set to true to print details of 
        fitting process. Defaults to 0.
        
        - callbacks (list, optional): A keras.callback function called 
        after each epoch. Defaults to [].
        """
        self.input_shape = input_shape
        self.n_outputs = n_outputs
        self.hp = kt.HyperParameters()
        self.hypergrid = hypergrid
        self.output_activation = output_activation
        self.epochs = epochs
        self.optimizer_class = optimizer_class
        self.loss = loss
        self.metrics = metrics
        self.objective = objective
        self.callbacks = callbacks
        self.directory = directory
        self.project_name = project_name
        self.factor = factor
        self.verbose = verbose
        self.batch_size = batch_size
        self.history = None # null before training

    def get_hypergrid(self):
        """Returns the user-specified hypergrid, plus the default items 
        (from _default_hypergrid) where the user did not specify 
        values"""
        output = copy.deepcopy(ANNHyperModel._default_hypergrid)
        if self.hypergrid is None:
             self.hypergrid = {}
        output.update(self.hypergrid)
        return output

    ### If you want to try more complex types of ANN, you should extend 
    # this class and overwrite this function with your own logic. 
    # For example, instead of adding Dense layers, you could add LSTM 
    # layers instead to create an RNN. 
    def build(self,hp:kt.HyperParameters):
        """Using an instance of kt.HyperParameters, we build and 
        compile a Keras Sequential instance with 
        some parameters filled by HyperParameter objects instead of 
        literal values. The Keras Hyperband object can then execute 
        a search function using this hypermodel to discover the 
        best model setting."""

        ### here we take the hypergrid and use it to construct a set of 
        # kt.HyperParameter objects
        hypergrid = self.get_hypergrid()
        width_hyperparam = hp.Int(
              'units', 
              min_value=hypergrid['width']['min'], 
              max_value=hypergrid['width']['max'], 
              step=hypergrid['width']['step']
              )
        depth_hyperparam = hp.Int(
              'layers', 
              min_value=hypergrid['depth']['min'], 
              max_value=hypergrid['depth']['max'], 
              step=hypergrid['depth']['step']
              )
        droprate_hyperparam = hp.Float(
              'droprate', 
              min_value=hypergrid['droprate']['min'], 
              max_value=hypergrid['droprate']['max'], 
              step=hypergrid['droprate']['step']
              )
        activation_hyperparam = hp.Choice(
              'activation', 
              values=hypergrid['activation']['values']
              )
        learning_rate_hyperparam = hp.Choice(
              'learning_rate', 
              values=hypergrid['learning_rate']['values']
              )

        model = keras.Sequential()
        model.add(
              keras.layers.Input(
              shape=self.input_shape
              )
            )
        
        ### Add a Dense-Dropout combo for each layer in the depth param
        for layer_number in range(0,depth_hyperparam):
              model.add(
                    keras.layers.Dense(
                        units=width_hyperparam,
                        activation=activation_hyperparam,
                        name = 'Dense_'+str(layer_number)
                        )
                    )
              model.add(
                   keras.layers.Dropout(droprate_hyperparam)
              )
        
        model.add(
                keras.layers.Dense(
                self.n_outputs,activation=self.output_activation
                )
            ) 

        model.compile(
            optimizer=self.optimizer_class(
                learning_rate=learning_rate_hyperparam
                ),
            loss=self.loss,
            metrics=self.metrics
            )
        ### This function must return an instance of a keras 
        # model that includes kt.Hyperparamater objects
        return model 
    
    def _hypertune(self,X_train, y_train, validation_size = 0):
        """This function executes the hypertuning of the
          built hypermodel"""
        self.build(self.hp)
        
        self.tuner = kt.Hyperband(
            ### We can input an instance of the custom class here (self) 
            # and the 'build' function will be called by keras 
            # Hyperband. 
                self, 
                objective= self.objective,
                max_epochs= self.epochs,
                factor=self.factor,
                hyperband_iterations=10,
                directory=self.directory,
                project_name=self.project_name
                )
        ### use validation split if validation_size > 0
        validation_sets = None
        if validation_size > 0:
            X_train, X_val, y_train, y_val = train_test_split(
                    X_train, 
                    y_train, 
                    test_size=validation_size
                    )
            validation_sets = (X_val,y_val)

        self.tuner.search(
               X_train,
               y_train, 
               epochs=self.epochs, 
               callbacks=self.callbacks, 
               verbose=self.verbose,
               batch_size=100,
               use_multiprocessing=True,
               validation_data=validation_sets
               )
        self.best_hyperparams=self.tuner.get_best_hyperparameters()[0]

        best_model = self.tuner.hypermodel.build(self.best_hyperparams)

        return best_model
    
    def execute(self,X,y,validation_size = 0):
        """
        Fit the model on training data.

        Args:
            - X (_type_): predictors, ndarray
            - y (_type_): dependant, ndarray or 1-d array
            - validation_size (int, optional): If > 0, tuner will 
            record validation performance of the objective. Set 
            objective to 'val_<metric name>' to select best model 
            on validation data. Defaults to 0.

        Raises:
            - e: If errors occur during fitting, a 'final' clause is 
            invoked to ensure the tuning directory is still deleted. 
        """
        ### Use finally clause to ensure the tuning dir is deleted even 
        # if there is an error.This failsafe is used because 'leftover' 
        # tuning directories from previous fittings can cause errors.
        try:
            self.model = self._hypertune(X,y,validation_size)
            self.history = self.model.fit(
                X,
                y, 
                batch_size=self.batch_size, 
                epochs=self.epochs,
                callbacks=self.callbacks, 
                verbose = self.verbose)
        except Exception as e:
            raise e
        
        finally:
            try:
                  shutil.rmtree(self.directory)
            except FileNotFoundError as e:
                 pass

Making predictions

Now that we have our fitted model, let us evaluate its predictive power. In this context, we are going to use accuracy as our evaluation metric. Accuracy is the portion of classifications which were correct. Typically, this metric is suitable when there are roughly equal numbers of classes in the data and we consider any kind of misclassification as equally undesirable.

y_pred = model.model.predict(X_test).argmax(axis=1)

accuracy = accuracy_score(y_test.argmax(axis=1),y_pred)
print("Accuracy of this model is: "+"{:1.2f}".format(accuracy*100)+"%")

2/2 [==============================] - 0s 0s/step
Accuracy of this model is: 98.00%

Success!! The best model discovered by our hypertuner was 98% accurate, a big improvement over 70% in our basic model.

Fitting history

The history parameter helps us monitor how effectively the fitting process converged on the optimum accuracy. We can construct a timeline which shows us how the accuracy metric improved at each epoch as the optimizer progressively improved the weightings. Note that once a point of diminishing return is reached, the EarlyStopping callback function will conclude the fitting process early to save computation time, which is why we cannot see the requested 150 epochs.

Practitioners should expect to see a steady increase in performance over epochs. A history showing no improvement over epochs would indicate a problem with the model.

pd.Series(
    model.history.history['accuracy']
    ).plot(
        xlabel='Epochs',
        ylabel="Accuracy"
        )

<Axes: xlabel='Epochs', ylabel='Accuracy'>

What were the best hyperparameters?

The best_hyperparams property will tell us which of the parameters we trialed was best. Note that some of the ‘tuner’ parameters are internal considerations for the tuner and not important to us for now.

model.best_hyperparams.values

{'units': 52,
 'layers': 2,
 'droprate': 0.1,
 'activation': 'relu',
 'learning_rate': 0.01,
 'tuner/epochs': 2,
 'tuner/initial_epoch': 0,
 'tuner/bracket': 4,
 'tuner/round': 0}

What does our model look like?

We can use the summary() function to print out a basic summary of our model’s architecture.

model.model.summary()

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 Dense_0 (Dense)             (None, 52)                260       
                                                                 
 dropout_1 (Dropout)         (None, 52)                0         
                                                                 
 Dense_1 (Dense)             (None, 52)                2756      
                                                                 
 dropout_2 (Dropout)         (None, 52)                0         
                                                                 
 dense_1 (Dense)             (None, 3)                 159       
                                                                 
=================================================================
Total params: 3,175
Trainable params: 3,175
Non-trainable params: 0
_________________________________________________________________

Evaluating Performance

We know the model was 98% accurate, but let us examine more deeply how well it classified each species. We shall use a confusion matrix.

matplotlib.rc('figure', figsize=(5,5))
ConfusionMatrixDisplay(
    confusion_matrix(
        y_test.argmax(axis=1), 
        y_pred)
        ).plot(
            colorbar=False
            )
plt.title('Confusion Matrix')

Text(0.5, 1.0, 'Confusion Matrix')

Interpreting the model

Most consumers of machine learning models want to know why a model gives the output it does. This can be a difficult task for a complex neural network, with many interactions nested in the algorithm. In a linear model, each feature has a positive or negative impact on the outcome. In an interactive model, the direction and size of effect is conditional and not so simple to describe. A common method for explaining why models output as they do is ‘permutational feature importance’ in which we calculate the loss of accuracy in a model when it does not have a given feature available. However, this is a lot of repeated computation when you have many features and ANNs can be slow for this.

Another approach is SHapley Additive exPlanations (SHAP), which uses an algorithm based on game-theory as well as many computational optimizations, to efficiently calculate ‘SHAP values’ for an ANN. Here we shall demonstrate SHAP. A larger SHAP value indicates the feature is more influential.

deep_explain = shap.DeepExplainer(model.model, X_test)
deep_shap_values= deep_explain.shap_values(X_test[1:5])

matplotlib.rc('figure', figsize=(10,10))
pd.Series(
    deep_shap_values[0][0],
    index=feature_names
    ).sort_values(
    ).plot.barh(
        title='SHAP Values'
        )

<Axes: title={'center': 'SHAP Values'}>

It appears that petal length and width are the most important factors when trying to classify a flower. Sepal width is also important, but sepal length is relatively inconsequential.

Loading and preprocessing the United Nations General Debate Corpus

In this tutorial, we will work with the United Nations General Debate Corpus (Baturo, Dasandi, and Mikhaylov 2017). It consists of all general debate statements from 1970 (Session 25) to 2020 (Session 75). We store the corpus in a data folder. For demonstration, we will limit it to the last 5 years to predict a sentiment category for each sentence in all speeches using BERT and create a visualization.

To achieve our goal, we include the following steps:

• Loading the speeches and doing some data wrangling
• Setting up and implementing the sentiment prediction pipeline
• Iterate over all sentences to predict their sentiment
• Visualize the results in a plot

If you want to follow the tutorial on your own machine, you can access the data here. If you want to follow the coding examples in a Jupyter notebook, you can access it in this Google Colab sandbox.

As the data comes within various .txt-files and formats differs slightly over the years, we first have to execute some data processing to bring the data into the required shape:

                                     speech_sentence session  year country
0  Let me begin by congratulating Ms. María Ferna...      73  2018     BRB
1  However, I would like to pause at this stage, ...      73  2018     BRB
2  Those events include the transit of a tropical...      73  2018     BRB
3  Those events are of great concern because the ...      73  2018     BRB
4           I ask myself, what does all that matter?      73  2018     BRB

# Load needed packages/functions
import pandas as pd
import glob
import re
import spacy
import os

# Download required spaCy model:
spacy.cli.download("en_core_web_sm")

# Function to transform an input raw speech into a sentence-level data frame
def get_speech_text_from_file(filepath):
    # Load the .txt-file
    with open(filepath) as f:
        lines = f.readlines()
    
    # Remove numerical identifiers and obsolete whitespaces
    lines = " ".join([re.sub(r'^\d{1,3}(\.|:)(\t| )|\n', '', line) for line in lines])
    # Load spaCy sentencer to split the speech on sentence-level
    nlp = spacy.load('en_core_web_sm')
    doc = nlp(lines)
    sentences = [sent.text.strip() for sent in doc.sents]
    
    # Create a data frame to store each sentence
    speech_df = pd.DataFrame(sentences, columns = ['speech_sentence'])
    title_search = re.search(r'Session (\d{1,3}) - (\d{4})/(\w{3})', filepath)
    
    # Add some meta information
    speech_df['session'] = title_search.group(1)
    speech_df['year'] = title_search.group(2)
    speech_df['country'] = title_search.group(3)
    
    return speech_df

# Data folder path
dir_path = r'./data/'

# List to store files
speeches_df = pd.DataFrame(columns = ['speech_sentence', 'session', 'year', 'country'])

# Identify files in the folder (to loop over relevant once)
folder_in_dir_path = [f.path for f in os.scandir(dir_path) if f.is_dir()]

# Extract the relevant folders
relevant_folders = []

for folder in folder_in_dir_path:
  # Extract the year (it comes with four digits) from the folder name
  year = int(''.join((str(i)) for i in re.findall('[0-9]{4}', folder)))
  # Identify the folder names of the last 5 years
  if year>2015:
    relevant_folders.append(folder)

# Iterate over the directory with relevant folders and crawl files
for folder in relevant_folders:
  for filename in glob.iglob(folder + '/**/*.txt', recursive=True):
    # Call function to transform an input raw speech into a sentence-level data frame
    speech_df = get_speech_text_from_file(filename)
    # Concatenate all resulting data frames into one
    speeches_df = pd.concat([speeches_df, speech_df], ignore_index=True)

speeches_df.head()

                                     speech_sentence session  year country
0  Let me begin by congratulating Ms. María Ferna...      73  2018     BRB
1  However, I would like to pause at this stage, ...      73  2018     BRB
2  Those events include the transit of a tropical...      73  2018     BRB
3  Those events are of great concern because the ...      73  2018     BRB
4           I ask myself, what does all that matter?      73  2018     BRB

pip install -U spacy
python3 -m spacy download en_core_web_sm

Setting up and implementing the sentiment prediction pipeline

Now we have some neatly processed and organized speeches and want to continue with the exciting part: extracting sentiment using BERT and explaining its decisions with a framework. This leads us to an essential question: what is the right model for the data and task?

A good starting point can be the number of downloads of a model and also a well-documented model card. However, there are a few very decisive properties to narrow down the search:

• Tasks – What is the underlying task the model is trained on (e.g., text classification or automatic speech recognition)?
• Language(s) – Do the pre-trained models speak the same (or a related) language as the data in my dataset?
• Size – Larger models tend to require more computational power, but also also trained on more data, which is beneficial in some cases.
• Source – Who trained the initial model? Given that these models come in with “prior knowledge”, you usually want to reduce the risk of biases as much as you can. While the selection of the initial training data can give a (first) good hint, using models that are well documented and come from reliable sources (although it can be up for debate how to define this) can be additional selection criteria.

BERT is among the most popular transformer-based model. Its architecture is based on twelve of the so-called encoder blocks (as described in the initial BERT paper by Devlin et al. (2018)). As we would like to reduce the computational workload to make this tutorial accessible for everyone, we decided to further work with a distilBERT model: distilbert-base-uncased-finetuned-sst-2-english. These models are smaller and faster than BERT-based models. The first distilBERT model was 40% smaller and 60% faster than a base BERT model while retaining about 97% of its functionality (Sanh et al. 2019).⁶

Using this model, we first start with our sentiment prediction pipeline. This is usually done by loading an appropriate model and then initializing the tokenizer:

from transformers import AutoModelForSequenceClassification, AutoTokenizer
from transformers_interpret import SequenceClassificationExplainer

model_name = 'distilbert-base-uncased-finetuned-sst-2-english'
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

These are also typical steps you need to do when fine-tuning a BERT model. We skip the fine-tuning step here because we do not have pre-labeled training and test data at hand. This also means that we cannot calculate the common metrics to understand the quality of our fine-tuned models. But with our pre-trained model at hand (we use the distilbert-base-uncased-finetuned-sst-2-english), we already have a model that has a good understanding of sentiment in the English language.

For this tutorial, we directly apply the existing knowledge of the model to our text corpus. In the following code chunk, we iterate over all sentences using the Python function apply() to call the custom function predict_sentiment() for each sentence. predict_sentiment() is a function that generalizes the pipeline we already implemented for our toy example in the first part of this blog post. The function takes the sentence and applies tokenization to convert it into a machine-readable format. The so-called inputs are then fed into our pre-trained BERT model to predict. argmax() helps us to find the sentiment class which is assigned with the highest probability according to the prediction. After converting the class ID to its label (either “positive”, “neutral”, or “negative”), we write its result back to a new column called sentiment:

def predict_sentiment(sequence):
  # Apply tokenization on our input sentence
  inputs = tokenizer(sequence, return_tensors='pt')
  # Do the prediction and save the logits
  logits = model(**inputs).logits
  # Find the class (Positive, Neutral, or Negative) which has the highest probability
  predicted_class_id = logits.argmax().item()
  # Return the predicted class
  return(model.config.id2label[predicted_class_id])

# Call predict_sentiment for all sentences
speeches_df['sentiment'] = speeches_df['speech_sentence'].apply(lambda speech_sentence: predict_sentiment(speech_sentence))
# The lambda function used here is a Pythonic approach to write an anonymous function 
# and it can take any number of arguments.

We now have the sentiment on the sentence level and we want to visualize some aspects of it. Since we use {reticulate}, it is fairly easy to switch between the worlds of Python and R. In our case, R seems to be a better candidate for visualization (if you love the logic of {ggplot2}. If you want to stay in Python, you can also give {plotnine} a try).

# Install pacman (package manager) if not done already
# install.packages(pacman)

# Load the tidyverse package via the pacman package manager
pacman::p_load(tidyverse,
               reticulate,
               countrycode)

# Get speeches from Python and transfer them into the R Environment
speeches_df <- py$speeches_df %>%
  # Convert categorical names to sentiment numbers
  dplyr::mutate(
    sentiment = dplyr::case_when(
      sentiment == "NEGATIVE" ~ -1,
      sentiment == "NEUTRAL" ~ 0,
      sentiment == "POSITIVE" ~ 1
    )
  ) %>%
  # group by country and year and calculate net sentiment values by summing them
  dplyr::group_by(country, year) %>%
  dplyr::summarise(net_perc = sum(sentiment))

speeches_df %>%
  # Generate the country name for each country using the
  # `countrycode()` command
  dplyr::mutate(countryname = countrycode(country, "iso3c", "country.name")) %>%
  # Filter and only select specific countries that we want to compare
  dplyr::filter(
    countryname %in% c(
      "North Korea",
      "Germany",
      "United Kingdom",
      "United States",
      "Pakistan",
      "France"
    )
  ) %>%
  # Now comes the plotting part :-)
  ggplot() +
  # We do a bar plot that has the years on the x-axis and the level of the
  # net-sentiment on the y-axis
  # We also color it so that all the net sentiments greater than 0 get a
  # different color
  geom_col(aes(
    x = year,
    y = net_perc,
    fill = (net_perc > 0)
  )) +
  # Here we define the colors as well as the labels and title of the legend
  scale_fill_manual(
    name = "Sentiment",
    labels = c("Negative", "Positive"),
    values = c("#C93312", "#446455")
  ) +
  # Now we add the axes labels
  xlab("Time") +
  ylab("Net sentiment") +
  # And do a facet_wrap by country to get a more meaningful visualization
  facet_wrap( ~ countryname) +
  # And make the theme a bit more beautiful
  theme_minimal() + theme(
    strip.background = element_blank(),
    panel.grid.major = element_blank(),
    panel.grid.minor = element_blank()
  )

Having an understanding of the net sentiment of speeches is a good start. But what we really want is to understand why the model classifies the sentences this way. That is where explainable AI comes in.

Using Transformers Interpret on a single sentence

These sentences sound quite diverse – some more positive, some more negative. Let’s check if that is also what the model classified. Using the method cls_explainer, we also generate an object word_attributions that will be helpful later. We use the first sentence as a working example: Around the world our message is clear — America's goal is lasting harmony, and not to go on with these endless wars.

word_attributions = cls_explainer(random_sentences[0])

With our cls_explainer object, we can look at the predicted class:

cls_explainer.predicted_class_name

'POSITIVE'

And it tells us that the classification is “positive”! But what we are really interested in is which words were important for this classification. This is what you get by calling visualize():

cls_explainer.visualize()

What we see in this visualization are the predicted label (1), the attribution label (“POSITIVE”), the overall attribution score (0.82, where the magnitude indicates the strength of the contribution), and, most importantly, a visual component on the right-hand side. Red highlighted parts contribute negatively to the classification while green highlighted parts contribute positively and white is considered neutral (or no contribution). The saturation shows the magnitude of the contribution. The visualization also matches with the numeric attributions and shows how positively, negatively, or neutrally a word (feature) contributes to the classification. As mentioned earlier, we can also get them by printing our previously generated word_attributions.

print(word_attributions)

[('[CLS]', 0.0), 
('around', 0.027562093028343622), 
('the', 0.026963912669903427), 
('world', 0.007187272390974884), 
('our', 0.04927162987468778), 
('message', 0.06561321917173639), 
('is', 0.02585681936629481), 
('clear', -0.03615942088697932), 
('—', 0.2694262447994862), 
('america', -0.07749574631815151), 
("'", 0.01128585937076096), 
('s', -0.006703021887516693), 
('goal', -0.07731179666725874), 
('is', 0.15398317629444674), 
('lasting', 0.13905982954437562), 
('harmony', 0.051255688029244316), 
(',', -0.00956995279920919), 
('and', -0.007379420575587129), 
('not', 0.7587632160635064), 
('to', 0.20532838096186265), 
('go', -0.06103999232141547), 
('on', -0.4228297838978442), 
('with', -0.11266722350080983), 
('these', -0.20836102947117963), 
('endless', 0.00042060694659563615), 
('wars', 0.05171845681735308), 
('.', -0.0031844890266940536), 
('[SEP]', 0.0)]

We can also use the output to generate another visualization that gives us an easily accessible overview of the attribution scores. We again use the same color codes as in the previous visualizations. Red means again that the tokens contribute negatively to the classification while the token in green contributes positively. Those in the middle ([SEP] and [CLS] for instance) are categorized as “neutral” (or no contribution) but don’t have any bars because their value is zero.

# Install pacman (package manager) if not done already
# install.packages(pacman)

# Load the tidyverse package via the pacman package manager
pacman::p_load(tidyverse,
               reticulate,
               dplyr,
               data.table)

# Get word_attributes from Python and transfer them into the R Environment
word_attributions <- py$word_attributions

# Now we need to do some pre-processing and extract relevant elements from the list
word_attributions_df <- as.data.frame(rbindlist(word_attributions))

# Visualize the attribution scores for each token using a barplot
word_attributions_df %>%
  # We do some housekeeping first
  rename(token = V1, attribution_scores = V2) %>%
  arrange(token, attribution_scores) %>% 
  ggplot() +
  # We do a bar plot that has the years on the x-axis and the level of the
  # net-sentiment on the y-axis
  # We also color it so that all the attribution scores greater than 0 get a
  # different color
  geom_col(aes(
    x = reorder(token, attribution_scores, sum),
    y = attribution_scores,
    fill = (attribution_scores > 0)
  )) +
  # Reverse the axes
  coord_flip() +
  # Here we define the colors as well as the labels and title of the legend
  scale_fill_manual(
    name = "Attribution Scores",
    labels = c("Negative", "Positive"),
    values = c("#C93312", "#446455")
  ) +
  # Now we add the axes labels
  xlab("") +
  ylab("Attribution Scores") +
  # And make the theme a bit more beautiful
  theme_minimal() + theme(
      strip.background = element_blank(),
      panel.grid.major = element_blank(),
      panel.grid.minor = element_blank()
  )

This allows us to also have some numerical output next to the visual output. Positive numbers indicate that the feature contributed positively to the classification and negative numbers indicate the opposite. Here we can see from both the numbers and the visualization, that it is mainly the word not that seems to drive the positive classification of the sentence. This is interesting given that a human reader might probably rather go for other words such as harmony or lasting. This can be a first indicator that we need to fine-tune the model. Another interesting pick is the effect of punctuation. The full stop at the end of the sentence seems to negatively contribute to the classification. Here, it would be interesting to see how the model behaves with another punctuation. Understanding how changes in the features can change the model behavior is what explainable AI is for!

Using Transformers Interpret on multiple sentences

Repeating the steps above for all selected sentences shows us what Transformers Interpret can tell us about the remaining sentences. We can also see the variance of positively and negatively labeled attributes.

# Define a custom function that generates the word_attributions
# and returns the visualization
def interpret_sentence(sentence):
  word_attributions = cls_explainer(sentence)
  return cls_explainer.visualize()

# Here we iterate over the random_sentences and return the 
# visualizations for each sentence
for sentence in range(len(random_sentences)):
    interpret_sentence(random_sentences[sentence])

We see that – based on the overall attribution score – in particular sentences 2 and 5 are labeled as positive, while sentences 3 and 4 are leaning more towards a negative sentiment. Two interesting takeaways here: First, we see that the magnitude of the attribution score varies across sentences and that the attribution score of the first sentence is rather low in comparison to, for instance, sentence 2. Second, when looking closer at sentence 3, we see that peace and wealth (for instance) are highlighted in red. This informs us as to how these words contributed to the prediction. Since the prediction is “negative” it makes sense that the words peace and wealth do not contribute to the prediction but that it is instead a word like undermine that pushes the model to classify the sentence as “negative”.

Using explainable AI for better model understanding and fine-tuning

Looking at a few examples (as we just did) is great. But looking at more is even better! This way, we can develop a good understanding of why the model classifies data the way it does and give us an idea of where to improve the training data. Although we can theoretically compute the attribution scores for each input, going through the output requires manual work. In practice, we will be inclined to limit the checks to a sample. In this case, we risk cherry-picking. To avoid this, we should start thinking of a framework for how to best integrate a consistent and systematic explainable AI check in our model evaluation. Depending on the data size, it might be impossible to check all input but we could strategically focus on specific cases. We could, for instance, check a certain percentage of cases falling into a clear positive or negative classification (based on the probabilities) and then, to a larger share, a percentage of those cases falling into a probability area around the threshold. These cases are likely to be somewhat of an “either/or”-decision where it is a great way to learn from explainable AI why the model opted for either a positive or negative classification. These insights will also help us further fine-tune the models (if needed).

Detecting biases and building fair(er) models

But this is not the only use case of explainable AI. A better model understanding also allows us to detect biases and build fair(er) models. Hovy and Prabhumoye (2021) identify five sources of bias in NLP:

1. the data,
2. the annotation process,
3. the input representations,
4. the models, and finally,
5. the research design

Using explainable AI can help us to understand how the model behaves and where potential biases may have been injected. One example was revealed by Arvind Narayanan and Aylin Caliskan who showed that Google Translate tended to associate job professions in a gender-biased manner. Using a gender-neutral Turkish pronoun, they showed that professions such as nurse and teacher were associated with women (e.g., “she is a nurse/teacher”) whereas translations of sentences involving professions such as professor or doctor were more likely to be associated with men (e.g., “he is a doctor/professor”). Thus, when training data are biased, the model is likely to reflect gender-specific biases and stereotypes (Caliskan, Bryson, and Narayanan 2017). Using model explainability can therefore not only benefit practitioners but also general audiences by reducing the reinforcement of gender-specific stereotypes.

But explainable AI is also beneficial beyond text-based models. A prominent example that was portrayed in the Netflix documentary “Coded Bias” highlights the discriminating effects of biased training data in facial recognition. The MIT researcher Joy Buolamwini discovered that dark-skinned faces are not detected accurately but that the program only worked when she wore a white mask. Becoming aware of these effects and biases is a crucial step in making the world a fair(er) place for everyone.

Another example – while less about fairness – is a situation that you are likely to experience when using reverse vending machines at the supermarket. Why do these machines fail to detect your water bottles correctly and, more importantly, what can you, as a user, do about it? Envisioning some improvements, explainable AI could help and visually tell which parts of the bottle were accurately detected and which were not. This could then help us to place the bottles in the correct position when inserting them into the reverse vending machine.

Using explainable AI as a standard for rigorous model checks

As a last potential for explainable AI, we want to draw attention to technical reporting. When we turn to academic papers, we typically report common model metrics such as precision, recall, or F1 scores¹⁴ to show how well our models perform. Digging deeper and doing a more qualitative evaluation of what the models do is needed. With the methods provided (and based on) frameworks such as Captum, this is luckily not too difficult to implement. It helps us to better understand what the models do and why they do it. In scientific papers, we could use this approach, for instance, as part of rigorous model checks, and, if the models are deployed somewhere, having an explainable AI dashboard would be a great asset to monitor and evaluate the model performance beyond the typical model metrics.