• Stars
    star
    372
  • Rank 114,858 (Top 3 %)
  • Language
    C
  • License
    Other
  • Created almost 5 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A simple native file system for Linux kernel

simplefs - a simple file system for Linux

The file system "simplefs" is helpful to understand Linux VFS and file system basics. The Linux VFS supports multiple file systems. The kernel does most of the work while the file system specific tasks are delegated to the individual file systems through the handlers. Instead of calling the functions directly the kernel uses various Operation Tables, which are a collection of handlers for each operation (these are actually structures of function pointers for each handlers/callbacks).

The super block operations are set at the time of mounting. The operation tables for inodes and files are set when the inode is opened. The first step before opening an inode is lookup. The inode of a file is looked up by calling the lookup handler of the parent inode.

Current features

  • Directories: create, remove, list, rename;
  • Regular files: create, remove, read/write (through page cache), rename;
  • Hard/Symbolic links (also symlink or soft link): create, remove, rename;
  • No extended attribute support

Prerequisite

Install linux kernel header in advance.

$ sudo apt install linux-headers-$(uname -r)

Build and Run

You can build the kernel module and tool with make. Generate test image via make test.img, which creates a zeroed file of 50 MiB.

You can then mount this image on a system with the simplefs kernel module installed. Let's test kernel module:

$ sudo insmod simplefs.ko

Corresponding kernel message:

simplefs: module loaded

Generate test image by creating a zeroed file of 50 MiB. We can then mount this image on a system with the simplefs kernel module installed.

$ mkdir -p test
$ dd if=/dev/zero of=test.img bs=1M count=50
$ ./mkfs.simplefs test.img
$ sudo mount -o loop -t simplefs test.img test

You shall get the following kernel messages:

simplefs: '/dev/loop?' mount success

Here /dev/loop? might be loop1, loop2, loop3, etc.

Perform regular file system operations: (as root)

$ echo "Hello World" > test/hello
$ cat test/hello
$ ls -lR

Remove kernel mount point and module:

$ sudo umount test
$ sudo rmmod simplefs

Design

At present, simplefs only provides straightforward features.

Partition layout

+------------+-------------+-------------------+-------------------+-------------+
| superblock | inode store | inode free bitmap | block free bitmap | data blocks |
+------------+-------------+-------------------+-------------------+-------------+

Each block is 4 KiB large.

Superblock

The superblock is the first block of the partition (block 0). It contains the partition's metadata, such as the number of blocks, number of inodes, number of free inodes/blocks, ...

Inode store

Contains all the inodes of the partition. The maximum number of inodes is equal to the number of blocks of the partition. Each inode contains 72 B of data: standard data such as file size and number of used blocks, as well as a simplefs-specific field ei_block. This block contains:

  • for a directory: the list of files in this directory. A directory can contain at most 40920 files, and filenames are limited to 255 characters to fit in a single block.
inode
+-----------------------+
| i_mode = IFDIR | 0755 |            block 123
| ei_block = 123    ----|-------->  +----------------+
| i_size = 4 KiB        |         0 | ee_block  = 0  |
| i_blocks = 1          |           | ee_len    = 8  |      block 84
+-----------------------+           | ee_start  = 84 |--->  +-----------+
                                    |----------------|    0 | 24 (foo)  |
                                  1 | ee_block  = 8  |      |-----------|
                                    | ee_len    = 8  |    1 | 45 (bar)  |
                                    | ee_start  = 16 |      |-----------|
                                    |----------------|      | ...       |
                                    | ...            |      |-----------|
                                    |----------------|   14 | 0         |
                                341 | ee_block  = 0  |      +-----------+
                                    | ee_len    = 0  |
                                    | ee_start  = 0  |
                                    +----------------+

  • for a file: the list of extents containing the actual data of this file. Since block IDs are stored as sizeof(struct simplefs_extent) bytes values, at most 341 links fit in a single block, limiting the size of a file to around 10.65 MiB (10912 KiB).
inode                                                
+-----------------------+                           
| i_mode = IFDIR | 0644 |          block 93       
| ei_block = 93     ----|------>  +----------------+      
| i_size = 10 KiB       |       0 | ee_block  = 0  |     
| i_blocks = 25         |         | ee_len    = 8  |      extent 94 
+-----------------------+         | ee_start  = 94 |---> +--------+
                                  |----------------|     |        |     
                                1 | ee_block  = 8  |     +--------+
                                  | ee_len    = 8  |      extent 99
                                  | ee_start  = 99 |---> +--------+ 
                                  |----------------|     |        |
                                2 | ee_block  = 16 |     +--------+
                                  | ee_len    = 8  |      extent 66 
                                  | ee_start  = 66 |---> +--------+
                                  |----------------|     |        |
                                  | ...            |     +--------+
                                  |----------------|  
                              341 | ee_block  = 0  | 
                                  | ee_len    = 0  |
                                  | ee_start  = 0  |
                                  +----------------+

Extent support

The extent covers consecutive blocks, we allocate consecutive disk blocks for it at a single time. It is described by struct simplefs_extent which contains three members:

  • ee_block: first logical block extent covers.
  • ee_len: number of blocks covered by extent.
  • ee_start: first physical block extent covers.
struct simplefs_extent
  +----------------+                           
  | ee_block =  0  |    
  | ee_len   =  200|              extent
  | ee_start =  12 |-----------> +---------+
  +----------------+    block 12 |         |
                                 +---------+
                              13 |         |
                                 +---------+
                                 | ...     |
                                 +---------+
                             211 |         |
                                 +---------+

TODO

  • journalling support

License

simplefs is released under the BSD 2 clause license. Use of this source code is governed by a BSD-style license that can be found in the LICENSE file.

More Repositories

1

lkmpg

The Linux Kernel Module Programming Guide (updated for 5.0+ kernels)
TeX
7,496
star
2

shecc

A self-hosting and educational C optimizing compiler
C
1,119
star
3

lab0-c

C Programming Lab: Assessing Your C Programming Skills
C
407
star
4

rv32emu

Compact and Efficient RISC-V RV32I[MAFC] emulator
C
393
star
5

concurrent-programs

Complementary Concurrency Programs for course "Linux Kernel Internals"
C
301
star
6

jitboy

A Game Boy emulator with dynamic recompilation (JIT)
C
299
star
7

semu

A minimalist RISC-V system emulator capable of running Linux kernel
C
252
star
8

cpumemory-zhtw

Traditional Chinese translation of "What Every Programmer Should Know About Memory"
CSS
216
star
9

vwifi

A virtual wireless device driver for Linux
C
201
star
10

kvm-host

A minimalist type 2 hypervisor using Linux Kernel Virtual Machine (KVM)
C
152
star
11

pitifulvm

A shabby implementation of Java virtual machine in C
C
138
star
12

vcam

Virtual camera device driver for Linux
C
89
star
13

sehttpd

A small and efficient web server with 1K lines of C code
C
82
star
14

concurrency-primer

Concurrency Primer
TeX
75
star
15

cserv

An event-driven and non-blocking web server
C
70
star
16

concurrent-ll

concurrent linked list implementation
C
68
star
17

khttpd

An experimental HTTP server implemented as Linux kernel module
C
60
star
18

rv32emu-legacy

RISC-V RV32I[MA] emulator with ELF support
C
47
star
19

raycaster

Wolfenstein 3D-style raycasting implementation
C
44
star
20

linux-list

Linux-like doubly-linked list
C
39
star
21

fibdrv

Linux kernel module that calculates Fibonacci numbers
Shell
37
star
22

gameboy-emu

An efficient and portable Game Boy emulator
C
36
star
23

mado

A window system for resource-constrained devices
C
34
star
24

lkm-hidden

A Linux kernel module which hides itself
C
29
star
25

kecho

A lightweight echo server implementation in Linux kernel mode
C
27
star
26

rubi

Ruby-like high-performance script programming language with JIT compilation
C
25
star
27

threaded-logger

Threaded Logger
C
21
star
28

threadkit

A collection of lightweight threading utilities
C
20
star
29

vinput

A collection of virtual input device drivers for Linux
C
20
star
30

linux-cfs-sim

Simulate Linux Completely Fair Scheduler (CFS) using POSIX Threads
C
18
star
31

rnnoise

A noise suppression library based on a recurrent neural network
C
17
star
32

kcalc

Math expression evaluation as Linux kernel module
C
17
star
33

dict

Ternary Search Tree + Bloom filter
C
15
star
34

jitcalc

A simple integer calculator using JIT compilation
C
15
star
35

y86_64-tools

Y86-64 Tools: assembler, simulator, Verilog designs
C
14
star
36

fastcat

A faster "cat" implementation using splice and sendfile system calls
C
13
star
37

neocon

A simple serial console utility
C
13
star
38

bignum

An incomplete arbitrary-precision integer arithmetic library
C
13
star
39

fiber

A User Space Threading Library
C
13
star
40

compute-pi

Leibniz formula for π
C
12
star
41

ca2023-lab3

Lab3: Construct a single-cycle CPU with Chisel
Scala
12
star
42

mapreduce

A simple C Thread pool implementation
C
12
star
43

prefix-search

Implement prefix search using ternary search tree
C
12
star
44

jit-construct

JIT compiler from scratch, derived from Nick Desaulniers' great work
Lua
11
star
45

datalab

Improved CS:APP Data Lab
C
9
star
46

buddy

Buddy Memory Allocator
C
8
star
47

moxiebox

A secure, sandboxed execution mechanism that enables deterministic input, processing and output
C
8
star
48

phonebook

sample phonebook program to illustrate the impact of cache miss
Shell
8
star
49

raytracing

Small ray tracing program for performance evaluation
C
8
star
50

intrusive-ds

A collection of intrusive data-structures for C
C
8
star
51

kilo

A text editor in less than 1000 LoC with syntax highlight and search
C
8
star
52

gecos

GECOS: A lock-free synchronization mechanism
C
7
star
53

nyancat

Nyancat rendered in your terminal
C
6
star
54

matrix_oo

Sample matrix implementation illustrating object-oriented techniques in C99
Shell
6
star
55

dont-trace

A simple Linux kernel module that kills ptrace tracer and its tracees
C
6
star
56

kfifo-examples

Linux kernel module examples about kfifo
C
5
star
57

mergesort-concurrent

merge sort on singly-linked list utilzing POSIX Thread
C
5
star
58

tinymembench

Measure peak bandwidth of sequential memory accesses and the latency of random memory accesses
C
5
star
59

cirbuf

Circular Buffer implementation with mmap(2) *incomplete*
C
4
star
60

align-bench

Microbenchmark for unaligned memory access
C
4
star
61

kcalc-fixed

Math expression evaluation as Linux kernel module, fixed-point implementation
C
4
star
62

malloc-test-concurrent

concurrent malloc benchmark
C
3
star
63

prefetcher

Evaluate the effects of prefetching
Shell
3
star
64

sched-plugin

A Linux kernel module to allow user processes being handed out with LKM based scheduler
C
3
star
65

phonebook-concurrent

build a phonebook program by concurrent linked list
C
3
star
66

simrupt

A Linux device driver that simulates interrupts
C
3
star
67

tco-test

Test the ability of C compilers performing Tail Call Optimization
C
2
star
68

vsnd

Virtual Linux soundcard driver
C
2
star
69

rv32emu-demo

HTML
2
star
70

balanced-ternary

Ilustrate how balanced ternary works
Shell
2
star
71

bf-runtime

Brainf*ck runtime engine
C
1
star
72

quotient-filter

(Incomplete) in-memory quotient filter
C
1
star
73

ksort

Linux kernel module that implements and validates sorting algorithms
C
1
star
74

arm-assembler-latex-listings

Arm Assembler language definition for the LaTeX listings package
TeX
1
star
75

clz-tests

Evaluate implementations of count leading zero
C
1
star