Python-Custom-Digit-Recognition

You can apply a simple OCR on your own handrwitten digits using this python script. I have used OpenCV to pre-process the image and to extract the digits from the picture. Using K-Nearest Neighbours (or SVM) as my model - I trained it using my own handwritten data set. I have also included the freely available MNIST data set so you can experiment on how different datasets work with different handwritings.

Analysis

I tried using just extracted the pixels as data to train and to predict the digits, but the accuracy was too low even on popular classification algorithms like SVM, KNN and Neural Netoworks. I did improve the accuracy a little bit after trying some custom threshold values. The best accuracy I could achieve only using pixel values was close to 55-60% that was after converting all the images to Black OR White from Black AND White.

After searching and reading about feature extraction from images for OCR - I stumbled HOG (Histogram of Gradients). Basically, it tries to capture the shape of structures in the region by capturing information about gradients. Image gradient are simply intensity changes across pixels in an image.

It works by dividing the image into small (usually 8x8 pixels) cells and blocks of 4x4 cells. Each cell has a fixed number of gradient orientation bins. Each pixel in the cell votes for a gradient orientation bin with a vote proportional to the gradient magnitude at that pixel or simple put, the "histogram" counts how many pixels have an edge with a specific orientation. More more info please refer this blog post

Using just only HOG histogram vectors as features drastically improved the accuracy of the prediction. Currently, I have used KNN from OpenCV as my model - I tried using SVM from the same module, but its accuracy was not as good as KNN. The best accuracy I have achieved on a sample image of about 100 digits is 80%. In the future, I might add more features after looking into SIFT, SURF or even try to get a better accuracy using just plain pixels as data

Usage

digit_recog.py is deprecated - may not work with newer versions of libraries

UPDATED CODE: NEW_digit_recog.py

To run code, download/clone repo and execute: python NEW_digit_recog.py

This code uses my own handwritten digits (custom_train_digits.jpg) as training data. You can also use your own but keep the positioning of the digits similar to whats in custom_train_digits.jpg file. If you make modifications in the format of the custom training data (your handwritten digits) make sure to edit load_digits_custom function in NEW_digit_recog.py as per the changes

Executing the program will generate 2 output files

This is the original image with digit boxes and the numbers on the top.
This is a plain image with just the recognized numbers printed.

Note:

• User image should be a scanned (atleast 300dpi) image.
• Image can be any format supported by OpenCV.

In NEW_digit_recog.py, use either
digits, labels = load_digits(TRAIN_DATA_IMG) #original MNIST data
For MNIST dataset OR
digits, labels = load_digits_custom('custom_train_digits.jpg') #my handwritten dataset For your own custom dataset.

Edit TRAIN_DATA_IMG and USER_IMG At line 190 and 191 if you want to use your own images for testing and training.

Libraries and Environement:

NOTE: To run this code without errors, you need a virtualenv with the correct libraries because the code is outdated (it was written over 5 years ago...)

Recommended py version: 3.6+

sudo apt-get install python3-venv 
sudo apt-get install libgtk2.0-dev pkg-config

python3 -m venv github-test
source github-test/bin/activate

pip3 install numpy==1.18
pip3 install scipy==1.1.0
pip3 install scikit-learn==0.21.3
pip3 install opencv-python==3.2.0.8
pip3 install scikit-image==0.12.1
pip3 install Pillow==2.2.2

git clone https://github.com/pavitrakumar78/Python-Custom-Digit-Recognition.git

python NEW_digit_recog.py

If you don't want to manually work with the versions, I've also added a tested requirements.txt file. Just intall whatever is in there and this script should run without any issues.

The accuarcy may be lower. You will need to tune the hyperparams in the model and try modifying the image processing piepline.

[PROBABLY DOES NOT WORK:] ~~Tested on:
Windows 10
Python 3.5

~~Dependencies:
numpy 1.13.1
SciPy 0.19.0
OpenCv (cv2) 3.2.0 ~~

Similar Project

I recently did a project where I use 2 CNNs to do both bounding box regression for detection and classification for digits on the street view house numbers dataset (SVHN). You can view the project here:
https://github.com/pavitrakumar78/Street-View-House-Numbers-SVHN-Detection-and-Classification-using-CNN