Data Cards Playbook

The Data Cards Playbook helps dataset producers and publishers adopt a people-centered approach to transparency in dataset documentation. Using the Playbook activities and resources on our website, you can create transparency-focused metadata schema for datasets across domains, organizational structures, and audience groups

In this repository, you can:

• Explore templates of Transparency Artifacts (Data Cards, Model Cards, Healthsheets)
• See and contribute examples of Data Cards in this repository

Data Cards

Data Cards are structured summaries of essential facts about various aspects of ML datasets needed by stakeholders across a dataset's lifecycle for responsible AI development. These summaries provide explanations of processes and rationales that shape the data and consequently the models, such as upstream sources, data collection and annotation methods; training and evaluation methods, intended use; or decisions affecting model performance.

Read our paper on Data Cards

Watch the paper video from FAccT 2022

Hands-on Data Card creation

Our Data Card template is available in .docx format. It contains numerous sections, questions and guidelines for responses that are designed to comprehensively document any possible dataset.

Along with Data Cards, we've also made Healthsheets(Research Paper) and Model Card (Research Paper) templates available to document healthcare-specific datasets and general purpose models, respectively.

Examples of Data Cards

• GEM Benchmark Data Cards
• FIT400m Data Card
• WikiDialog-OQ
• Open Images Extended - Crowdsourced
• Relative Movie Attributes
• More Inclusive Annotated People
• Translated Wikipedia Biographies
• Crowdsourced high-quality multi-speaker speech datasets
  • Argentinian Spanish
  • Chilean Spanish
  • Colombian Spanish
  • Peruvian Spanish
  • Venezulean Spanish
• Ivy Lee's collection of ML model cards and datasheets

Want to add your Data Card to this list? Open an issue!

Frequently Asked Questions (FAQs)

Coming Soon

Note

The Data Cards Playbook is being actively developed and documentation is likely to change as we improve our methodologies. We want to hear from you! Leave notes, feedback, or suggestions on our GitHub. Use #datacardsplaybook.

Citation

M. Pushkarna, A. Zaldivar, D. Nanas, et al. Data Cards Playbook. Published March 5, 2021.

License

The Data Cards Playbook is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.

Credits

Core Team

This work was co-created by Mahima Pushkarna and Andrew Zaldivar and done in collaboration with Reena Jana, Vivian Tsai, and Oddur Kjartansson. We want to thank Donald Gonzalez, Dan Nanas, Parker Barnes, Laura Rosenstein, Diana Akrong, Monica Caraway, Ding Wang, Danielle Smalls, Aybuke Turker, Emily Brouillet, Andrew Fuchs, Sebastian Gehrmann, Cassie Kozyrkov, Alex Siegman, and Anthony Keene for their immense contributions; and Meg Mitchell and Timnit Gebru for championing this work.

We also want to thank Adam Boulanger, Lauren Wilcox, Parker Barnes, Roxanne Pinto and Ayça Çakmakli for their feedback; Tulsee Doshi, Dan Liebling, Meredith Morris, Lucas Dixon, Fernanda Viegas, Jen Gennai, and Marian Croak for their support. This work would not have been possible without our workshop and study participants, and numerous partners, whose insights and experiences have shaped this Playbook.

Special Thanks

This work would not have been possible without our workshop participants, supporters and champions, whose insights and experiences have shaped this Playbook: Lucas Ackerknecht, Hartwig Adam, Seiji Armstrong, Lora Aroyo, Sebastian Assaf, Anurag Batra, Samy Bengio, Louisa Bostrom, Thomas Cadwalader, Michelle Carney, Will Carter, Amanda Casari, Di Dang, Alex David Norton, Tiffany Deng, Emily Denton, Tulsee Doshi, Madeleine Elish, Patrick Gage Kelley, Timnit Gebru, Sara Goetz, Robbie Gonzalez, Alex Hanna, Jing Hua, Ben Hutchinson, Nathan Ie, Robyn Im, Orion Jankowski, Ellen Jiang, Shivani Kapania, David Karam, Daniel Kim, Leslie Lai, Eryka Lehr, Elijah Logan, Daphne Luong, Nicole Maffeo, Meg Mitchell, Maysam Moussalem, Unni Nair, Ricardo Olenewa, Kristen Olson, Praveen Paritosh, Adam Pearce, Angie Peng, Ludovic Peran, Roxanne Pinto, Vinodkumar Prabhakaran, Rida Qadri, Ravi Rajakumar, Hima Rajana, Susanna Ricco, Kevin Robinson, Taylor Roper, Negar Rostamzadeh, Mo Shomrat, Andrew Smart, Jamila Smith-Loud, Nithum Thain, Janel Thamkul, Aybuke Turker, Joseph Thomas, Bobby Tran, James Wang, Martin Wattenberg, James Wexler, Catherine Williams, Catherina Xu, Tabitha Yong, and Ben Zevenbergen.