• Stars
    star
    118
  • Rank 299,923 (Top 6 %)
  • Language
    Jupyter Notebook
  • Created about 7 years ago
  • Updated over 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

This repository provides everything to get started with Python for Text Mining / Natural Language Processing (NLP)

Get started with Python for Text Mining (NLP)

Want to learn how to use Python for Text Mining / Natural Language Processing (NLP)?
This repository has everything that you need to get started!

Author: Ties de Kok (Personal Page)

These materials accompany a PhD session on NLP for Accounting Research: slides

Quick link to the notebook: open notebook

Table of contents

Introduction

The goal of this GitHub page is to provide you with everything you need to get started with Python and Natural Language Processing (NLP)

The following topics are discussed:

(Note: the neural network part is only a reference to the Stanford course CS224n)

Who is this repository for?

The topics and techniques demonstrated in this repository are primarily oriented towards empirical research projects in fields such as Accounting, Finance, Marketing, Political Science, and other Social Sciences.

However, many of the basics are also perfectly applicable if you are looking to use Python for any other type of Data Science!

How to use this repository?

This repository is written to facilitate learning by doing.

All the material is written up in a Jupyter Notebook. See: NLP_notebook.ipynb.
The topics are split up by task description.

It is best to view the notebook locally or on nbviewer using this link: click here

An environment.yml file is provided that you can install using conda, this will automatically install all the packages used in the notebook.

Instructions on how to install the environment are provided here: Install environment

Not yet familiar with the basic Python syntax?

Please check out my "Getting started with Python for Research" repository: click here

Using Jupyter

To run the provided notebook file you need to use Jupyter Lab or Jupyter Notebook.

Jupyter comes pre-installed with the Anaconda distribution so you should have everything already installed and ready to go. The environment.yml will also install Jupyter Lab if you prefer to use that.

What is the Jupyter Notebook?

From the Jupyter website:

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text.

In other words, the Jupyter Notebook allows you to program Python code straight from your browser!

How does the Jupyter Notebook work in the background?

The diagram below sums up the basics components of Jupyter:

At the heart there is the Jupyter Server that handles everything, the Jupyter Notebook which is accessed and used through your browser, and the kernel that executes the code. We will be focusing on the natively included Python Kernel but Jupyter is language agnostic so you can also use it with other languages/software such as 'R'.

It is worth noting that in most cases you will be running the Jupyter Server on your own computer and will connect to it locally in your browser (i.e. you don't need to be connected to the internet). However, it is also possible to run the Jupyter Server on a different computer, for example a high performance computation server in the cloud, and connect to it over the internet.

How to start a Jupyter Notebook?

The primary method that I would recommend to start a Jupyter Notebook is to use the command line (terminal) directly:

  1. Open your command prompt / terminal (on Windows I recommend the Anaconda Prompt)
  2. Activate the environment conda activate PythonNLPTutorial
  3. cd (i.e. Change) to the desired starting directory
    for example: cd "C:\Files\Work\Project_1"
    Note: if you are changing do folder on another drive you might have to also switch drives by typing, for example, E:
  4. Start the Jupyter Notebook server by typing: jupyter notebook or jupyter lab

This should automatically open up the corresponding Jupyter Notebook/Lab in your default browser. You can also manually go to the Jupyter Notebook/Lab by going to localhost:8888 with your browser.

How to close a Jupyter Notebook/Lab server?

If you want to close down the Jupyter Server: open up the command prompt window that runs the server and press CTRL + C twice.
Make sure that you have saved any open Jupyter Notebooks!

How to use the Jupyter Notebook?

Some shortcuts are worth mentioning for reference purposes:

command mode --> enable by pressing esc
edit mode --> enable by pressing enter

command mode edit mode both modes
Y : cell to code Tab : code completion or indent Shift-Enter : run cell, select below
M : cell to markdown Shift-Tab : tooltip Ctrl-Enter : run cell
A : insert cell above Ctrl-A : select all
B : insert cell below Ctrl-Z : undo
X: cut selected cell

Code along!

Option 1: clone repository

You can essentially "download" the contents of this repository by cloning the repository.

You can do this by clicking "Clone or download" button and then "Download ZIP":

If you extract the downloaded ZIP to a folder you can start the Jupyter Notebook/Lab in that folder and access the notebook.

Environment

You can install the environment by following these steps:

  1. Make sure you have Anaconda installed (link)
  2. Open your command prompt / terminal (on Windows I recommend the Anaconda Prompt)
  3. cd (i.e. Change) to the folder where you extracted the ZIP file
    for example: cd "C:\Files\Work\Project_1"
    Note: if you are changing do folder on another drive you might have to also switch drives by typing, for example, E:
  4. Run the following command conda env create -f environment.yml
  5. Activate the environment with: conda activate PythonNLPTutorial

A full list of all the packages used is provided in the environment.yml file.

Option 2: use Binder

Binder

Note: some functionality might not work on Binder.

Questions?

If you have questions or experience problems please use the issues tab of this repository.

License

MIT - Ties de Kok - 2020

Special Thanks

https://github.com/teles/array-mixer for having an awesome readme that I used as a template.

More Repositories

1

chat-gpt-jupyter-extension

A browser extension to provide various AI helper functions in Jupyter Notebooks, powered by ChatGPT.
TypeScript
299
star
2

LearnPythonforResearch

This repository provides everything you need to get started with Python for (social science) research.
Jupyter Notebook
247
star
3

ipystata

Enables the use of Stata together with Python via Jupyter (IPython) notebooks.
Jupyter Notebook
192
star
4

limperg_python

Repository with material for the Limperg Python course by Ties de Kok.
Jupyter Notebook
148
star
5

fast_xbrl_parser

An XBRL parser built in Rust that provides a fast, easy, and lightweight way to convert XBRL XML files into JSON or CSV.
Rust
33
star
6

chatgpt_paper

Code examples to accompany the paper: "ChatGPT for Textual Analysis? How to use Generative LLMs in Accounting Research"
Jupyter Notebook
24
star
7

UW_Python_Camp

Jupyter Notebook
19
star
8

fWHR_calculator

Facial Width Height Ratio (fWHR) calculator using Python
Jupyter Notebook
19
star
9

pydata_2023

Companion materials for my PyData 2023 talk on generative LLMs
Jupyter Notebook
17
star
10

acctg-579B-python-for-business-research

This is a standalone version of my former ACCTG 579B phd class on Python programming for business research.
Jupyter Notebook
16
star
11

NotebookOpener

Open and manage Jupyter (IPython) notebooks using different Python environments and working directories.
Python
9
star
12

gpt-prompt-example

This is a companion repository to my blog post contain Python code to show how to do zero shot, few shot, and fine-tuning using the OpenAI API.
Jupyter Notebook
8
star
13

xbrl-api-playground

Jupyter Notebook
8
star
14

jupyterlab_code_bookmarks

JupyterLab extension to add code bookmarking functionality
TypeScript
6
star
15

AccountingNLP_Slides

Repository containing the slides for a PhD course lecture on using Python and NLP for Accounting research
HTML
5
star
16

mnsc.2023.4670

Code repository for "How Resilient Are Firms’ Financial Reporting Processes?"
Jupyter Notebook
3
star
17

BristolPythonWorkshop2018

Jupyter Notebook
3
star
18

eaa_2023_nlp_workshop

Jupyter Notebook
3
star
19

blockchain_with_python

This Jupyter Notebook contains the Python code equivalent of the R code provided in the paper "Introduction to Blockchain With R" written by Theophanis Stratopoulos and Jesus Calderon.
Jupyter Notebook
2
star
20

TiesdeKok.github.io

HTML
1
star
21

foundry-vtt-macros

A personal collection of foundry vtt macros
JavaScript
1
star
22

limperg_python_2019

Jupyter Notebook
1
star
23

zoom-question-manager

This extension provides a dashboard to manage questions asked in the Zoom chat.
HTML
1
star
24

MAS_2022_textual_analysis

Jupyter Notebook
1
star