Image Crop Analysis
This is a repo for the code used for reproducing our Image Crop Analysis paper as shared on our blog post.
If you plan to use this code please cite our paper as follows:
@article{TwitterImageCrop2021,
author = {Yee, Kyra and Tantipongpipat, Uthaipon and Mishra, Shubhanshu},
title = {Image Cropping on Twitter: Fairness Metrics, Their Limitations, and the Importance of Representation, Design, and Agency},
year = {2021},
issue_date = {October 2021},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {5},
number = {CSCW2},
url = {https://doi.org/10.1145/3479594},
doi = {10.1145/3479594},
journal = {Proceedings of the ACM on Human-Computer Interaction},
month = oct,
articleno = {450},
numpages = {24},
keywords = {image cropping, ethical HCI, fairness in machine learning, demographic parity, representational harm}
}
@article{TwitterImageCrop2021ArXiv,
author = {{Yee}, Kyra and {Tantipongpipat}, Uthaipon and {Mishra}, Shubhanshu},
title = "{Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency}",
journal = {arXiv e-prints},
keywords = {Computer Science - Computers and Society, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Human-Computer Interaction, Computer Science - Machine Learning},
year = 2021,
month = may,
eid = {arXiv:2105.08667},
pages = {arXiv:2105.08667},
archivePrefix = {arXiv},
eprint = {2105.08667},
primaryClass = {cs.CY},
}
Instructions
- Install Miniconda or Anaconda and then follow these steps:
- create a conda environment using
conda env create -f environment.yml - activate the environment using
conda activate image-crop-analysis
- create a conda environment using
- Put a dummy jpeg image at
data/dummy.jpeg - Put any additional images with
*.jpegextension in DATA_DIR, which is./data - If you just want to investigate how the model predicts the saliency map then you can use the notebook notebooks/Image Annotation Dash.ipynb
- To reproduce the analysis first prepare the data using notebooks/Data Preparation.ipynb and then run notebooks/Demographic Bias Analysis.ipynb
- To reproduce the plots first run notebooks/Demographic Bias Plots.ipynb
- If you want to explore how the library behind the dashboard works see notebooks/Image Crop Analysis.ipynb
- If you have the dataset prepared from the above steps then you can create the gender gaze dataset by running notebooks/Gender Gaze Analysis.ipynb
Docker Run
- Install docker
- Run the following commands in this root directory of this project:
docker build -t "image_crop" -f docker/Dockerfile .
docker run -p 9000:9000 -p 8900:8900 -it image_crop- Open the jupyter lab URL shown in terminal.
Run on Google Colab
- Open a google colab notebook
- Run the following code in the cell where
HOME_DIRvariable is set:
try:
import google.colab
! pip install pandas scikit-learn scikit-image statsmodels requests dash
! [[ -d image-crop-analysis ]] || git clone https://github.com/twitter-research/image-crop-analysis.git
HOME_DIR = Path("./image_crop_analysis").expanduser()
IN_COLAB = True
except:
IN_COLAB = False
- Try the notebooks/Image Crop Analysis.ipynb notebook for example.
Security Issues?
Please report sensitive security issues via Twitter's bug-bounty program (https://hackerone.com/twitter) rather than GitHub.