Basic Usage

This example shows a basic workflow of OpenAttack which consists of four main steps:

• Initializing classifier

• Loading dataset

• Initializing attacker

• Evaluation

Initializing Classifier

victim = OpenAttack.loadVictim("BERT.SST")

Victim.BERT.SST is a pytorch model which is trained on SST dataset.

The load operation returns a Classifier that can be further used for Attacker and AttackEval.

Loading dataset

def dataset_mapping(x):
    return {
        "x": x["sentence"],
        "y": 1 if x["label"] > 0.5 else 0,
    }
dataset = datasets.load_dataset("sst").map(function=dataset_mapping)

We use the datasets package to manage our data, through which we load the data and map the fields to their corresponding places.

For each data instance, x means the attacked text and y means the true label of the data.

Initializing Attacker

attacker = OpenAttack.attackers.GeneticAttacker()

After this step, we’ve initialized a GeneticAttacker and uses the default configuration during attack process.

To use a custom configuration, you can pass in some parameters manually.

Evaluation

attack_eval = OpenAttack.AttackEval(attacker, victim)
attack_eval.eval(dataset, visualize=True)

AttackEval is the class used for evaluation. It has many options however we will not go into details here.

Using visualize=True in attack_eval.eval can make it displays a visualized result. This function is really useful for analyzing small datasets.

Complete Code

import OpenAttack
import datasets
def main():
    victim = OpenAttack.loadVictim("BERT.SST")
    def dataset_mapping(x):
        return {
            "x": x["sentence"],
            "y": 1 if x["label"] > 0.5 else 0,
        }
    dataset = datasets.load_dataset("sst", split="train[:20]").map(function=dataset_mapping)
    attacker = OpenAttack.attackers.GeneticAttacker()
    attack_eval = OpenAttack.AttackEval(attacker, victim)
    attack_eval.eval(dataset, visualize=True)

Run python examples/workflow.py to see visualized results.