Behavioral Testing of NLP models with CheckList

When developing an NLP model, it's a standard practice to test how well a model generalizes to unseen examples by evaluating it on a held-out dataset. Suppose we reach our target performance metric of 95% on a held-out dataset and thus deploy the model to production based on this single metric.

But, when real users start using it, the story could be completely different than what our 95% performance metric was saying. Our model might perform poorly even on simple variations of the training text.

In contrast, the field of software engineering uses a suite of unit tests, integration tests, and end-to-end tests to evaluate all aspects of the product for failures. An application is deployed to production only after passing these rigorous tests.

Ribeiro et al. noticed this gap and took inspiration from software engineering to propose an evaluation methodology for NLP called "CheckList". Their paper won the best overall paper award at ACL 2020.

In this post, I will explain the overall concept of CheckList and the various components that it proposes for evaluating NLP models.

Behavioral Testing

To understand CheckList, let's first understand behavioral testing in the context of software engineering.

Behavioral testing, also known as black-box testing, is a method where we test a piece of software based on its expected input and output. We don't need access to the actual implementation details.

For example, let's say you have a function that adds two numbers together.

def add(a, b):
    return a + b

We can evaluate this function by writing tests to compare it's output to the expected answer. We are not concerned with how this function was implemented internally.

def test_add():
    assert add(1, 2) == 3
    assert add(1, 0) == 1
    assert add(-1, 1) == 0
    assert add(-1, -1) == -2

Even for a simple function such as addition, there are capabilities that it should satisfy. For example, the addition of a number with zero should yield the original number itself.

CheckList Framework

CheckList proposes a general framework for writing behavioral tests for any NLP model and task.

The core idea is based on a conceptual matrix that is composed of linguistic capabilities as rows and test types as columns. The intersecting cells contain multiple test examples generated from templates that we run and calculate the failure rate for.

By calculating the failure rates for various test types and capabilities, we can know exactly where our model is weak.

Let's understand each part of this conceptual matrix in detail now.

1. Test Types

These are the columns in the previous matrix. There are 3 types of tests proposed in the CheckList framework:

a. Minimum Functionality Test(MFT)

This test is similar to unit tests in software engineering. We build a collection of (text, expected label) pairs from scratch and test the model on this collection.

For example, we are testing the negation capability of the model using an MFT test below.

Template: I {NEGATION} {POS_VERB} the {THING}
{: .text-center}

The goal of this test is to make sure the model is not taking any shortcuts and possesses linguistic capabilities.

b. Invariance Test(INV)

In this test, we perturb our existing training examples in a way that the label should not change. Then, the model is tested on this perturbed example and the model passes the test only if its prediction remains the same (i.e invariant).

For example, changing the location from Chicago to Dallas should not change the original sentiment of a text.

We can use different perturbation functions to test different capabilities. The paper mentions two examples:

c. Directional Expectation Test(DIR)

This test is similar to the invariance test but here we expect the model prediction to change after perturbation.

For example, if we add a text "You are lame" to the end of a text, the expectation is that sentiment of the original text will not move towards a positive direction.

We can also write tests where we expect the target label to change. For example, consider the QQP task where we need to detect if two questions are duplicates or not.

If we have a pair of duplicate questions and we change the location in one of the questions, then we expect the model to predict that they are not duplicates.

|Capability|Question 1| Question 2|Expected|Predicted|Passed|
|---|---|---|---|---|
|NER|How many people are there in England?|What is the population of England?|Duplicate|Duplicate|✔|
|NER|How many people are there in England?|What is the population of Turkey?|Not Duplicate|Duplicate|X|

2. Linguistic Capabilities

These are the rows in the CheckList matrix. Each row contains a specific linguistic capability that applies to most NLP tasks.

Let's understand examples of capabilities given in the original paper. The authors provide a lot of examples to help us build a mental model of how to test new capabilities relevant to our task and domain.

a. Vocabulary and POS

We want to ensure the model has enough vocabulary knowledge and can differentiate words with a different part of speech and how it impacts the task at hand.

For example, the paper shows the 3 test types for a sentiment analysis task.

This can also be applied for the QQP task as shown below.

b. Named Entity Recognition(NER)

It tests the capability of the model to understand named entities and whether it is important for the current task or not.

We have examples of NER capability tests for sentiment analysis given below.

We can also apply this to the QQP task.

c. Temporal

Here we want to test if the model understands the order of events in the text.

Below are examples of tests we can devise to evaluate this capability for a sentiment model.

Similarly, we can devise temporal capability tests for QQP data as well.

d. Negation

This ensures the model understands negation and its impact on the output.

Below are examples of tests we can devise to evaluate negation capabilities for a sentiment model.

Similarly, we can devise negation capability tests for QQP data as well.

e. Semantic Role Labeling(SRL)

This ensures the model understands the agent and the object in the text.

Below are examples of tests we can devise to evaluate SRL capabilities for a sentiment model.

Similarly, we can devise SRL capability tests for QQP data as well.

f. Robustness

This ensures that the model can handle small variations or perturbations to the input text such as typos and irrelevant changes.

Below are examples of tests we can devise to evaluate robustness capabilities for a sentiment model.

Similarly, we can devise robustness capability tests for QQP data as well.

g. Taxonomy

This ensures that the model has an understanding of synonyms and antonyms and how they affect the task at hand.

Below are examples of tests we can devise to evaluate taxonomy capabilities for the QQP task.

h. Coreference Resolution

This ensures that the model has an understanding of pronouns and what nouns they refer to.

Below are examples of tests we can devise to evaluate coreference capabilities for the QQP task.

i. Logic

This ensures that the model can handle symmetry, consistency, and conjunctions.

For example, in the QQP task, the order of the question shouldn't matter. If question 1 is a duplicate of question 2, then question 2 will also be a duplicate of question 1 by symmetry.

j. Fairness

This tests if the model reflects any form of bias towards a demographic from the training data.

Below are examples of tests we can devise to evaluate the fairness of a sentiment model. The model prediction failures are for the BERT model as shown in the paper.

3. Test Generation

The paper's authors have open-sourced a software tool that can generate test cases at scale based on the ideas above.

The tool provides three approaches to write test cases:

To generate templates, you can either brainstorm them from scratch or generalize patterns from your existing data.

Manually Generated Templates

For example, if we had a text such as "I didn't love the food" in our training data, we can generalize it as:

Now, you can brainstorm possible fillers for the various template parts.

By taking the cartesian products of all these possibilities, we can generate a lot of test cases.

Masked Language Model Template

Instead of manually specifying fill-ins for the template, we can also use MLM models like ROBERTA and use masking to generate variants.

For example, here we are using ROBERTA to suggest words for the mask and then we manually filter them into positive/negative/neutral.

These fill-ins can be reused across multiple tests. The paper also suggests using WordNet to select only context-appropriate synonyms from ROBERTA.

Built-in Fill-ins

CheckList also provides out-of-box support for lexicons such as:

• NER: common first/last names, cities and countries
• Protected Group Adjectives: Nationalities, Religions, Gender, Sexuality

Built-in Perturbations

CheckList also provides perturbation functions such as character swaps, contractions, name and location changes, and neutral word replacement.

Conclusion

Thus, CheckList proposes a general framework to perform a comprehensive and fine-grained evaluation of NLP models. This can help us better understand the state of NLP models beyond the leaderboard.

References

• Marco Tulio Ribeiro et al., "Beyond Accuracy: Behavioral Testing of NLP models with CheckList"

Connect

If you enjoyed this blog post, feel free to connect with me on Twitter where I share new blog posts every week.