Learn Node.js by building a backend framework - Velocy

You can access the current version of the book in the chapters directory or in PDF format (both Light and Dark modes are available) by clicking here. Note that this version includes the current release of the content, and is not the final version.

This book is still in a very early stage. It contains an insignificant portion of the total content that the book is supposed to cover.

There’s going to be 0 dependencies for our backend framework, as well as our logging library. Everything will be done using vanilla Node.js, the hard-way (the best way to learn).

I've found that one of the best ways to get a handle on a new concept is to start from scratch. Begin with nothing, and build it up yourself. This approach lets you not only learn how it works, but also understand why it works that way.

This isn't any normal Node.js tutorial or guide. It's a detailed walkthrough, and a code along experience that shows you how to create a backend framework from the ground up while getting a solid grasp of Node.js's inner workings and it’s standard library.

In this guide, we will not only build a web framework, but also focus on designing a powerful and optimized end product that is ready for use in production applications (somewhat). Our goal is to create a modular backend framework that can be easily extended with new features as needed.

We will cover topics like error handling, security, and testing to ensure that our framework is reliable and secure. We will explore different approaches to modularity and demonstrate how to create reusable components that can be shared across multiple projects. By the end of this guide, you will have a solid understanding of Node.js and it’s tough parts.

I highly recommend coding along with this guide rather than just reading it.

The repo for our backend framework- Velocy

Table of contents

• (optional) Node.js is way faster than you think
  • Contenders for the test
    • Elysia - Bun
    • Axum - Rust
    • Express - Node.js
    • Velocy - Node.js
  • The benchmark
• What the hell is a web server any way?
  • Parts of a Web Server
  • Navigating the World of Protocols: A Quick Overview
  • The Relationship Between HTTP and TCP
    • Data Integrity and Order
    • Acknowledgment Mechanism
    • Complex Interactions
    • Transmission Overhead
  • How Web Servers Respond to Your Requests
    • The Request
      • Your Request
      • Finding the Address
      • Resolving the Address
    • The Response
      • Return Address
      • Sending the Request
      • Preparing the Content
      • Sending the Response
      • Enjoying the Content
• Your first web server with node.js
  • What exactly is node or nodejs?
  • Your first node.js program
  • How does console.log() work in Node.js?
  • The process Object
  • The stdout property of the process object
• Working with files
  • What will the logging library do
  • How do you work with files anyway?
  • Let's get back to files
  • A little more about file descriptors
  • Creating our first file
    • path argument
    • flag argument
    • mode argument
  • Reading from a file
  • A small primer to for..of and for await..of in javascript
    • for..of
    • for await..of
  • Reading the json file
  • Buffers
  • Parsing the json file
• logtar - Our Own logging library
  • Initializing a new project
  • A little about SemVer
  • Creating a LogLevel class
  • The Logger class
  • Encapsulation with private fields
  • The LogConfig class
  • Design Patterns
    • The Builder pattern
    • Using the Builder pattern with the LogConfig class
  • jsdoc comments
  • The RollingConfig class
    • The RollingSizeOptions class
    • The RollingTimeOptions class
  • Finishing up the RollingConfig class
  • Let's recap
  • Adding more useful methods in the LogConfig class
  • Why readFileSync?
• Refactoring the code
  • The need for refactoring
  • Creating Separate Files
    • Explanation
    • index.js file
    • lib/logtar.js file
    • lib/logger.js file
    • lib/config/log-config.js file
    • lib/config/rolling-config.js file
    • lib/utils/log-level.js file
    • lib/utils/rolling-options.js class
• Writing Logs
  • Re-using the file handle
  • Log rotation
  • Asynchronous logging
  • Getting caller information
  • Testing our current API
  • Implementing logging methods
    • DRY (Don't Repeat Yourself)
    • The log method
    • Considering the log_level member variable
  • Writing to a file
    • A small primer on Regular Expressions
    • Testing the log file creation
    • Another gotcha
    • Logs directory configuration
    • Our first script
  • The require object
  • Adding a new helper to create log directory
  • Updating the init method
  • Completing the log method
• Capturing metadata
  • What is a stack
  • Examples of stacks
  • The call stack
  • Getting the stack info
  • Getting the callee name and the line number
  • A more ergonomic way
  • Using the get_caller_info function
• A small intro to async vs sync
  • The Balance between Opposites
  • Mixing Asynchronous and Synchronous Code
  • Faster I/O out of the box
  • Blocking Code
  • Concurrency
• Adding Rolling File Support
  • Rolling features
    • #time_threshold
    • #size_threshold
  • The rolling_check() method
  • file_handle.stat()
  • Calling the rolling_check method
  • A big gotcha!
  • Stack traces across await points
    • The culprit
  • Testing the new Log file creation
• HTTP Deep Dive
  • A small web server
    • Starting our web server
    • Testing our web server
    • Testing with cURL
• HTTP Verbs, Versioning and the benefits of HTTP/1.1
  • GET - Retrieve data
  • POST - Create something
  • PUT - Replace or create
  • HEAD - Retrieve metadata
  • DELETE - Remove from existence
  • PATCH - Partial updates
  • A small recap
  • The / path
  • HTTP/0.9
  • HTTP/1.0
    • Introduction to the HTTP Header
    • Versioning
    • Status Codes
    • Content-Type Header
    • New Methods
  • HTTP/1.1
    • Persistent Connections
    • Pipelining
    • Chunked Transfer Encoding
    • The Host header
    • Caching improvements
    • Range Requests
    • New Methods: PUT, DELETE, CONNECT, OPTIONS, TRACE
• User Agents
  • User-Agent can be weird
• MIME Type and Content-Type
  • Understanding the Accept Header
    • Breaking Down the Line
    • Why the Wildcard?
    • Server Response
  • Mime Type
  • Anatomy of a MIME type
  • But why the wildcard */*?
  • The Content-Type header
    • Content-Type on request header
    • Content-Type on response header
  • The charset=UTF-8: character encoding
    • Universal Character Encoding
• Headers
  • Header Name
  • Colon (:)
  • Header Value
  • Whitespace
  • Custom X- based headers
• Request headers
  • Accept
  • Referer
  • Authorization
  • Cookie
  • Host
  • Content-Type
• Response Headers
  • Content-Type (response)
  • Cache-Control
    • How Caches Work
    • Cache-Control Directives
      • Always Cache (infrequent updates)
      • Always Cache (private only)
      • Never Cache (realtime data)
  • Set-Cookie
• Response and Status Codes
  • Connection: close in action
  • Status Codes
• velocy our backend framework
  • Why velocy?
  • What is a backend framework/library anyway?
  • Core features of our backend framework
    • Routing and URL Handling
    • Middlewares
    • Building our own database
      • Data Storage and Retrieval
      • Indexing
      • Querying
    • Caching
    • Rate limiting
    • Other features
      • Shared state
      • File uploads
      • Static file serving
      • Multi-part data
      • Websockets
      • Logging
      • Monitoring
• A basic Router implementation
  • A Toy Router
  • Chunks, oh no!
  • Specifying Content-Length
  • Code reusability
  • Separation of Concerns
• The Router class
  • Using Router with an HTTP server
  • this is not good
    • Using .bind()
    • Using Arrow function
  • Lexical context
  • Arrow functions are not free
  • Why should we care about memory?
  • Testing the updated code
• Improving the Router API
• The need for a trie
  • What is a Trie anyway?
• Exercise 1 - Implementing a Trie
  • Root Node
  • End of the word
  • Challenge 1: Basic Trie with insert Method
    • Requirements
    • More details
    • Solution
  • Challenge 2: Implement search method
    • Requirements
    • More details
    • Hints
    • Solution
• Exercise 2 - Implementing our Trie based Router
  • Challenge 1: Implementing the addRoute method
    • Requirements
    • More details
    • Hints
    • Solution
    • Explanation
  • Challenge 2: Implementing the findRoute method
    • Requirements
    • More details
    • Starting Boilerplate
    • Hints
    • Solution
    • Explanation