Discover dannguyen/excsv Open Source project

Computer-Assisted Reporting and Data Journalism Syllabuses, compiled by Dan Nguyen

309

journalism-syllabi

Evaluating the performance and accuracy of ABBYY FineReader's OCR on Senate Financial Disclosure scanned forms

165

abbyy-finereader-ocr-senate

CSS

125

github-for-portfolios

A layperson's step-by-step guide to building webpages with Github

CSS

python-notebooks-data-wrangling

Python 3.x notebooks about real-world data cleaning and visualization

Python code to scrape and collect data from the RSS feeds Facebook uses to augment its Trending Section

facebook-trending-rss-fetcher

An online reference for data journalism

smalldata_journalism

A work-in-progress guide showing how and why you should learn command-line tools (xsv, csvkit) to work with data

learn-data-csv-cli

My personally curated list of bash/command-line commands and snippets that are very useful yet I keep on forgetting

bashfoo

a general list of resources and articles for people interested in getting into data journalism

datajournalism-primer

What fancy schools do U.S. legislators go to?

congress-colleges

Tracking my progress in doing GIS/Geospatial work in Python 3.x

gis-geospatial-fun-python3x

NICAR 2019 workshop on using Python and PDFplumber to extract text from PDFs

nicar-2019-pdfplumbing

A tutorial on using Face.com's and NYT Congress's API + Sunlight data

Congressmiles

I'm making a Github Pages repo!

dannguyen.github.io

looking at U.S. Senators' disclosures, including how to parse and track them

scrape-senate-financial-disclosures

how hard is it to get a list of all local news sites in the United States (LOL)

local-news-data

Python Courses at Stanford

python-at-stanford

NICAR-Google-Refine

The lesson and source files for Dan Nguyen's NICAR 2012 lesson on Google Refine

pdftotablestable

Comparing the programs that extract tabular data from PDFs, e.g. ABBYY FineReader, Tabula, CometDocs

house-financial-disclosures

Scraping House representative financial disclosures

OCR copy of the 2015-2016 FBI Investigation into Hillary Clinton's emails

clinton-hillary-email-fbi-investigation-docs

pydataproject-template

dan's personal reference for properly creating an empty/fresh python-based data wrangling project

PADJO 2017 SQL Exam - Now with extra election and disbursement data!

padjo-2017-sql-exam

Testing the new AWS Textract when it comes to extracting data tables from PDFs (pdf-to-csv) and whether it can deliver us from our endless torments

aws-textract-pdf-to-csv-demo

nhtsa-complaint-data

Some scripts/data description for NHTSA complaint data

quickdataproject-template

a template I use for quick data project examples where collection, wrangling, and exploration can be done by standalone shell/python scripts

A command-line tool for making it easier to create and save screenshots as a blogger

screencappy

A repo of collected data and records from U.S. state DMVs regarding rejected vanity license plates

dmv-vanity-plate-rejections

csvkitcat has been archived (Oct. 2020), and is being carted over to csvmedkit

csvkitcat

A "frozen" version of https://analytics.usa.gov to practice network traffic inspection and web scraping

frozen.analytics.usa.gov

CSS

writhub

A simple Python-based static post generator, because I just need to post, not make an entire website

My personal repo for doing quick journaling on Github with Markdown, plus some helper TOC scripts

journaling-on-github

"How to Find Stories in Data" for the Associated Collegiate Press 2017 San Francisco Midwinter Convention

acp-2017-finding-stories-in-data

kfc-scrape

chicken

til

A simple static Jekyll blog of things I've learned, day-to-day, particularly in programming and data journalism

Visualization in Python with the Altair library. Done in Jupyter Notebooks.

altair-dataviz

For the OCRing of scanned, murky documents where privacy, speed, accuracy, and cost are all priorities

mechanical-unmurk-ocr

seeing-is-beliebing

Instagram util for finding photos taken shorty before and after near where another photo was taken

A data/lesson repo teaching SQL syntax and concepts with a very simple SQLite database

simplestuff-sqlite

A list of small datasets for examples of exploration in spreadsheets

smalldata

Data notebook for CMS Medicare fee data

cms_medicare_fee_data

marktoc

A Python library for generating a table of contents and anchor markup for a Markdown file

A compilation of daily snapshots of San Francisco's emergency shelter reservation wait-list during the COVID-19 pandemic

sf-shelter-waitlist-daily-snapshots

seshkit is a command-line tool for creating transcripts from audio files

seshkit

A command-line tool for getting meta information from a URL

merle

Build out a gal using RMagick

DepGal

please i would like someday a tool that is like csvkit but for making charts from the command line

csvviz

supcli: my personal guide to modern CLI, including third-party replacement for classic Nix tools

supcli

xkcd-on-reactjs

Just playing around with React.js to make a searchable xkcd archive

yearbook

Examples of GIS with New York data and CartoDB

ny-gis-cartodb-fun

sf-ethics-lobbyist-sql

A repo of San Francisco lobbyist data compiled into SQLite form, including data-handling scripts

Machine-readable emotions in machine-readable CSV

emojicsv

command line lessons for 2022 quickie repo

command-line-basics-mz2022

SCOTUS-Transcript-Viewer

A Backbone.js viewer of SCOTUS transcripts

Analyzing the Bard's work with Ruby!

Shakyspeare

death-data

bts-transstats-t100-domestic-demo

Demo of data processing for BTS transtats

middleman-meta-tags

Meta and SEO tag helpers for Middleman

collecting crime report data from cities that have it in a granular format

city_crime_data

Makefile

bashappy_helpers

A bunch of helper functions I wrote to use for my own macOS terminal convenience

air_skift

Air rails

fetching and exploring SEC structured data for fun

secdataexploring

A repo for collecting data/records regarding the Defense Logistics

dod-leso-1033-data

A quick iPython notebook showing how to create and style Matplotlib charts with roughly same flexibility as ggplot2

matplotlib-styling-tutorial

playing around with texas state salary data courtesy of the Texas Tribune

texas-state-salaries

A copy of healthcare.gov when it was built on Jekyll, before they removed the source code

healthcare.gov

Trying to make a template that scaffolds a basic jekyll site with bootstrap and vendor d3v5

jekyll-datasite-template

pgark (page archiver): Python library and CLI for archiving URLs on popular services like Wayback Machine [alpha, just spitballing]

pgark

Going through Jason Brownlee's "Clever Algorithms: Nature-Inspired Programming Recipes" http://cleveralgorithms.com/nature-inspired/stochastic/random_search.html

nature-inspired-algorithms-in-python

Lookup tables and data references

lookups-of-note

censusscout

making my own lightweight version of Census Explorer because y not

A demonstration of scraping health inspection websites and doing statistical analysis

motherfuckingwebdesignguide

just do it

foodscrape

nicar-2019-github-intro

Intro to git and github for journalists

Makefile

Sinatra-Fun

Testing out sinatra

a basic jekyll theme that sits atop of Bootstrap 4.x. For my convenience only

jekyll-bootstrap-starter

The Little Data Wrangling Fakebook

data-wrangling-fakebook

a curated list of interesting foia/foil requests

foiastories

astronautdata

A repo of astronaut data

This is a bunch of examples of things I forget how to do in Sphinx and reST

danssphinx-template

A bash script for converting SQLite query into Markdown-ready-pastable results

sql2md

Poynter Census Data Workshop 2019, using Sphinx-hieroglyph slidemaker

poynter-census-data-2019

just collecting san francisco evictions data

stanford-public-affairs-data-journalism

sf-evictions

d3choro-template

merde

digital-jo-2017

Quickie repo for digital journalism notes for stanford journalism 2017

twitkit

yet another attempt at making a personal twitter data exploration command-line tool

wire-glossary

high-charty

wikipedia-trends

revelecture

A command-line tool to turn Markdown files into Reveal.js powered slideshows

need to practice this javascript thing

hello-svelte

Using R's ggplot2 and rgdal to examine earthquake activity in Oklahoma

ok-earthquakes-RNotebook

fatal-encounters-and-census-sql

SQLite database exercises for analyzing Fatal Encounters (police officer involved homicides) and Census data