Zero-shot learning has emerged as a promising approach to perform modeling with minimal labeled data. This method offers a solution to the challenging task of data labeling. In the context of text classification, the application of zero-shot learning gives rise to what is known as zero-shot text classification. In this article, we will delve into the concept of zero-shot text classification and explore various ways to implement it effectively.
What is Zero-Shot Learning?
Zero-shot learning refers to a modeling technique that does not heavily rely on a large volume of labeled data. Instead, it leverages existing knowledge and semantic relationships between seen and unseen classes to perform recognition tasks. Human beings are capable of zero-shot learning by establishing connections between known and unknown classes based on their existing knowledge. This approach finds extensive usage in recognition modeling, enabling models to learn about unseen classes that were not labeled during training. It operates by predicting new classes through the understanding of intermediate semantic layers and their attributes.
Understanding Zero-Shot Text Classification
Text classification, a fundamental task in natural language processing, involves predicting the classes of textual documents. Traditionally, text classification models require a significant amount of labeled data for training and are limited in their ability to predict using unseen data. However, the integration of zero-shot learning with text classification has revolutionized natural language processing.
The primary objective of zero-shot text classification is to classify text documents without relying on any labeled data or prior exposure to labeled text. This technique finds wide application in transformer models. The Hugging Face transformers library, for instance, offers over 60 transformers that facilitate zero-shot classification.
It’s worth mentioning that zero-shot text classification is closely related to few-shot classification, which employs a small number of labeled samples during training. GPT-3 by OpenAI is a well-known few-shot classifier.
Flair, another powerful library, provides a TARSclassifier specifically designed for zero-shot classification. It encompasses various transformers for performing NLP tasks like named entity recognition, text tagging, and text embedding.
Implementing Zero-Shot Text Classification with Hugging Face Transformers and TARSclassifier
To perform zero-shot text classification using Hugging Face transformers and TARSclassifier in Python, we will focus on implementing some of the most popular transformers. Let’s explore the step-by-step process.
Implementation using Transformers
Before diving into transformer implementation, make sure to install the transformers library in your environment using the following code:
!pip install transformers
Once the installation is complete, you’re ready to leverage transformers.
BART-large-mnli
Developed by Facebook researchers, BART-large-mnli is an upgraded model based on the Bart-large architecture, trained using the MNLI dataset. The Hugging Face transformers library provides a pipeline module for zero-shot classification.
Let’s try out this transformer:
import transformers
classifier = transformers.pipeline("zero-shot-classification",
model="facebook/bart-large-mnli")
Output:
...
To perform an operation, use the following code:
sequence = "I can perform article"
labels = ['writing', 'management', 'checking']
classifier(sequence, labels)
Output:
...
This transformer enables multiclass labeling and can be used in the PyTorch environment. For detailed information about the transformer, refer to the documentation.
Cross-Encoder
The Cross-Encoder transformer, also part of the Hugging Face transformer family, is trained on the SNLI and MNLI datasets. It is suitable for cross-encoding and zero-shot text classification.
Let’s explore this transformer:
classifier1 = transformers.pipeline("zero-shot-classification",
model='cross-encoder/nli-distilroberta-base')
Output:
...
To perform zero-shot text classification using the Cross-Encoder, use the following code:
classifier1(sequence, labels)
Output:
...
For more details on this transformer, consult the documentation.
Bart-large-nli
The Bart-large-nli model is specifically designed for zero-shot text classification, leveraging the NLI dataset. This model is built upon the Bart-large architecture. To use this model, follow these steps:
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline
# Load model & tokenizer
bart_model = AutoModelForSequenceClassification.from_pretrained('navteca/bart-large-mnli')
bart_tokenizer = AutoTokenizer.from_pretrained('navteca/bart-large-mnli')
# Get predictions
nlp = pipeline('zero-shot-classification', model=bart_model, tokenizer=bart_tokenizer)
Output:
...
To perform zero-shot text classification with Bart-large-nli, use the code below:
sequence = "I can perform article"
labels = ['writing', 'management', 'checking']
nlp(sequence, labels)
Output:
...
These are three popular transformers commonly used for zero-shot text classification.
Implementation using Flair
In addition to Hugging Face, we can also employ the Flair library for zero-shot text classification. Flair offers the TARSclassifier model, which is specifically designed for this purpose.
To install Flair, use the following code:
!pip install Flair
After successful installation, you can perform zero-shot text classification as follows:
Importing the Model:
from flair.models import TARSClassifier
classifier2 = TARSClassifier.load('tars-base')
Output:
...
Defining the Sentence:
from flair.data import Sentence
sentence = Sentence("I am so glad to use Flair")
Defining the Classes:
classes = ["happy", "sad"]
Generating Predictions:
classifier2.predict_zero_shot(sentence, classes)
print(sentence)
Output:
...
As evident from the results, this model predicts accurately. Flair offers additional functionality for various NLP tasks. For further information on Flair, refer to the documentation.
Conclusion
In this comprehensive guide, we explored the concept of zero-shot learning and its application in zero-shot text classification. We also discussed the effective implementation of zero-shot text classification using Hugging Face transformers and Flair. These frameworks have consistently demonstrated excellent performance in various applications.