• Stars
    star
    526
  • Rank 84,247 (Top 2 %)
  • Language
    Jupyter Notebook
  • Created over 5 years ago
  • Updated over 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Data Science Best Practices with pandas

Data Science Best Practices with pandas

This tutorial was presented by Kevin Markham at PyCon on May 2, 2019. Watch the complete tutorial video on YouTube.

Data Science Best Practices with pandas

Jupyter Notebook

The tutorial code is available as a Jupyter notebook. You can run this notebook in the cloud (no installation required) by clicking the "launch binder" button:

Binder

What is the tutorial about?

The pandas library is a powerful tool for multiple phases of the data science workflow, including data cleaning, visualization, and exploratory data analysis. However, the size and complexity of the pandas library makes it challenging to discover the best way to accomplish any given task.

In this tutorial, you'll use pandas to answer questions about a real-world dataset. Through each exercise, you'll learn important data science skills as well as "best practices" for using pandas. By the end of the tutorial, you'll be more fluent at using pandas to correctly and efficiently answer your own data science questions.

How well do I need to know pandas to participate?

You will get the most out of this tutorial if you are an intermediate pandas user, since the tutorial does not cover pandas basics.

  • If you are new to pandas, I recommend watching some videos from my free pandas course before the tutorial.
  • If you just need a pandas refresher, I recommend reviewing this Jupyter notebook, which includes all of the code from my pandas course.

What dataset are we using?

ted.csv is the TED Talks dataset from Kaggle Datasets, made available under the CC BY-NC-SA 4.0 license.

How do I download the CSV file from GitHub?

Here are three options that will work equally well:

  • If you want to directly download only the CSV file, right click on the following link and select "Save As": ted.csv.
  • If you know how to use git, you can click the green button above and clone the entire repository.
  • If you know how to open a ZIP file, you can click the green button above and download the entire repository.

What do I need to do before the tutorial?

  1. Make sure that pandas and matplotlib are installed on your computer. (The easiest way to install pandas and matplotlib is by downloading the Anaconda distribution.)
  2. Download the CSV file from this repository.
  3. Read the file into pandas using the read_csv() function to make sure everything is working.

How can I check that pandas and matplotlib are properly installed?

  1. Move the CSV file into your working directory. (This is usually the directory where you create Python scripts or notebooks.)

  2. Open the Python environment of your choice.

  3. If you're using the Jupyter notebook, run the following code:

    import pandas as pd
    import matplotlib.pyplot as plt
    %matplotlib inline
    ted = pd.read_csv('ted.csv')
    ted.comments.plot()
  4. If you're using any other Python environment, run the following code:

    import pandas as pd
    import matplotlib.pyplot as plt
    ted = pd.read_csv('ted.csv')
    ted.comments.plot()
    plt.show()

If you don't get any error messages, and a plot appears on your screen, then it's very likely that pandas and matplotlib are installed correctly.

Who is the instructor?

Kevin Markham is the founder of Data School, an online school for learning data science with Python. He is passionate about teaching data science to people who are new to the field, regardless of their educational and professional backgrounds. Previously, Kevin was the lead data science instructor for General Assembly in Washington, DC. Currently, he teaches machine learning and data analysis to over 10,000 students each month through the Data School YouTube channel. He has a degree in Computer Engineering from Vanderbilt University and lives in Asheville, North Carolina with his wife and son.

More Repositories

1

scikit-learn-videos

Jupyter notebooks from the scikit-learn video series
Jupyter Notebook
3,663
star
2

pandas-videos

Jupyter notebook and datasets from the pandas video series
Jupyter Notebook
2,143
star
3

scikit-learn-tips

🤖⚡ 50 scikit-learn tips
Jupyter Notebook
1,714
star
4

DAT8

General Assembly's 2015 Data Science course in Washington, DC
Jupyter Notebook
1,602
star
5

DAT4

General Assembly's Data Science course in Washington, DC
Jupyter Notebook
794
star
6

python-reference

Python Quick Reference
Jupyter Notebook
669
star
7

DAT3

General Assembly's Data Science course in Washington, DC
Roff
660
star
8

pycon-2016-tutorial

Machine Learning with Text in scikit-learn
Jupyter Notebook
441
star
9

pycon-2018-tutorial

Using pandas for Better (and Worse) Data Science
Jupyter Notebook
321
star
10

trump-lies

Tutorial: Web scraping in Python with Beautiful Soup
Jupyter Notebook
241
star
11

DAT7

General Assembly's Data Science course in Washington, DC
Jupyter Notebook
230
star
12

DAT5

General Assembly's Data Science course in Washington, DC
Jupyter Notebook
185
star
13

dplyr-tutorial

Tutorials for the dplyr package in R
159
star
14

pydata-dc-2016-tutorial

Tutorial: Machine Learning with Text in scikit-learn
Jupyter Notebook
74
star
15

python-data-analysis-workshop

Workshop: Intro to Python for Data Analysis
Python
71
star
16

python-data-science-workshop

Workshop: Python for Data Science
Python
61
star
17

kaggle-allstate

Allstate Purchase Prediction Challenge on Kaggle
R
58
star
18

kaggle-pycon-2015

Solution code from my winning submission to Kaggle's PyCon 2015 competition
Python
55
star
19

tidy-data

Commented R code from Hadley Wickham's "tidy data" presentation
R
29
star
20

PracticalMachineLearning

Course project for Practical Machine Learning: https://www.coursera.org/course/predmachlearn
13
star
21

coursera-getting-data

Class project for Coursera's "Getting and Cleaning Data" class
R
10
star
22

babynames

Baby Names by Birth Year
R
5
star
23

justmarkham

1
star