Project Sophia: Make Neural Networks from Scratch

Plagiarized from Inspired by Andrej Karparthy's cs231n and Colah's Blog.

This notebook recreates neural network algorithms using only the linear algebra library numpy (uses other libraries for performance benchmark and visualization). The sample data is a 2-dimensional data with three classes in a form of spirals.

The content consists of the following aspects of neural networks:

• Weight initialization
• Loss functions
  • Cross-entropy loss
  • Hinge loss
  • Regularization loss
• Layers
  • Linear
  • Sigmoid
  • Tanh
  • ReLu
  • Swish
  • Softmax
  • SVM
  • Dropout
  • Batch Normalization
  • Convolution (TBD)
  • LSTM (TBD)
  • Embeddings (TBD)
• Training
  • Linear > Softmax
  • Linear > SVM
  • [Linear > Activation]++ > [Linear > Output]
• Optimization (TBD)
• Validation (TBD)