Skip to content

Automated tool to evaluate Twitter saliency filter algorithmic bias

License

Notifications You must be signed in to change notification settings

MoadComputer/twitter_saliency_bias_analysis

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 

Repository files navigation

Automated algorithmic bias analysis of Twitter saliency filter

Open In Colab

This repository hosts a Jupyter notebook that introduces a few broad concepts, to help further develop automated testing tools for detecting algorithmic bias in machine vision tools, such as saliency filters.

The tool evaluated here is the Twitter saliency filter.

FairFace: the face attribute dataset that is balanced for gender, race and age; is used here to generate the random image pairs for performing the saliency filter tests.

Quantification of the statisitcal significance in differences between the carefully manipulated saliency filter outputs and the baseline saliency filter outputs, is performed using the Wilcoxon signed rank test.

Additional requirements

  • Valid Google account
  • This notebook by default assumes that the user is working inside the original Google Colab environment. To run locally or in other cloud environments, please make sure that the data dependencies are satisfied.
  • Google Drive access to save the FairFace dataset and the experiment history
Parts of the code used in this notebook are copyright protected.
Copyright 2021 Twitter, Inc.
SPDX-License-Identifier: Apache-2.0

Data download

Download the FairFace dataset fairface-img-margin125-trainval.zip file and the labels fairface_label_train.csv file from the official FairFace GitHub repo. The maintainers of the FairFace GitHub repository have published the links to download the data in their GitHub repo README file.

About

Automated tool to evaluate Twitter saliency filter algorithmic bias

Topics

Resources

License

Stars

Watchers

Forks