• Stars
    star
    243
  • Rank 166,489 (Top 4 %)
  • Language
    Go
  • License
    MIT License
  • Created over 12 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Detect silent data corruption under Linux using sha256 stored in extended attributes

CI Go Report Card β€’ View Changelog β€’ Download Binary Releases

cshatag is a tool to detect silent data corruption. It is meant to run periodically and stores the SHA256 of each file as an extended attribute. The project started as a minimal and fast reimplementation of shatag, written in Python by Maxime Augier.

See the Man Page further down this page for details.

Similar Tools

Checksums stored in extended attributes for each file

Checksums stored in single central database

Checksums stored in one database per directory

Compile Yourself

$ git clone https://github.com/rfjakob/cshatag.git
$ cd cshatag
$ make

Man Page

CSHATAG(1)                       User Manuals                       CSHATAG(1)

NAME
       cshatag - compiled shatag

SYNOPSIS
       cshatag [OPTIONS] FILE [FILE...]

DESCRIPTION
       cshatag is a minimal and fast re-implementation of shatag
       (  https://github.com/maugier/shatag  ,  written  in  Python  by Maxime
       Augier )
       in a compiled language (since v2.0: Go, earlier versions: C).

       cshatag is a tool to detect silent data corruption. It writes the mtime
       and  the sha256 checksum of a file into the file's extended attributes.
       The filesystem needs to be mounted with user_xattr enabled for this  to
       work.   When  run  again,  it compares stored mtime and checksum. If it
       finds that the mtime is unchanged but  the  checksum  has  changed,  it
       warns  on  stderr.   In  any case, the status of the file is printed to
       stdout and the stored checksum is updated.

       File statuses that appear on stdout are:
            <new>         file is missing both attributes
            <outdated>    both mtime and checksum have changed
            <ok>          both checksum and mtime stayed the same
            <timechange>  only mtime has changed, checksum stayed the same
            <corrupt>     mtime stayed the same but checksum changed

       cshatag aims to be format-compatible with shatag and uses the same  ex‐
       tended attributes (see the COMPATIBILITY section).

       cshatag was written in C in 2012 and has been rewritten in Go in 2019.

OPTIONS
       -dry-run    don't make any changes
       -recursive  recursively process the contents of directories
       -remove     remove cshatag's xattrs from FILE
       -q          quiet mode - don't report <ok> files
       -qq         quiet2 mode - only report <corrupt> files and errors

EXAMPLES
       Check  all regular files in the file tree below the current working di‐
       rectory:
       # cshatag -qq -recursive .
       Errors like corrupt files will be printed to stderr.  Run without "-qq"
       to see progress output.

       To remove the extended attributes from all files:
       # cshatag -recursive -remove .

RETURN VALUE
       0 Success
       1 Wrong number of arguments
       2 One or more files could not be opened
       3 One or more files is not a regular file
       4 Extended attributes could not be written to one or more files
       5 At least one file was found to be corrupt
       6 More than one type of error occurred

COMPATIBILITY
       cshatag  writes  the  user.shatag.ts field with full integer nanosecond
       precision, while python uses a double for the whole mtime and loses the
       last few digits.

AUTHOR
       Jakob   Unterwurzacher   <[email protected]>,   https://github.com/rf‐
       jakob/cshatag

COPYRIGHT
       Copyright 2012 Jakob Unterwurzacher. MIT License.

SEE ALSO
       shatag(1), sha256sum(1), getfattr(1), setfattr(1)

Linux                              MAY 2012                         CSHATAG(1)

Changelog

Short changelog - for all the details look at the git log.

vNEXT, in progress

  • Linux: use 100ns resolution when comparing timestamps instead of 1ns to match SMB protocol restrictions (#21 commit)
  • MacOS: use 1s resolution when comparing timestamps to match MacOS SMB client restrictions (#21)

v2.1.0, 2022-10-22

  • Add -dry-run #22
  • This version is called v2.1.0 as opposed to v2.1 to conform to go.mod versioning rules (three-digit semver).

v2.0, 2020-11-15

  • Rewrite cshatag in Go
  • add MacOS support
  • Add -remove flag
  • Add -q and -qq flags
  • Accept multiple files per invocation to improve performance
  • Work around problems on MacOS SMB mounts (#11)

v1.1, 2019-06-09

  • Add test suite (make test) (commit)
  • Add Travis CI
  • Drop useless trailing null byte from user.shatag.sha256

v1.0, 2019-01-02

  • Add make format target

2019-02-01

  • Fix missing null termination in ts buffer that could lead to false positives (commit)

2017-05-04

  • Respect PREFIX for make install (commit)

2016-09-17

  • Check for malloc returning NULL (commit)

2012-12-05

  • C source code & man page published on Github (commit)

More Repositories

1

gocryptfs

Encrypted overlay filesystem written in Go
Go
3,529
star
2

earlyoom

earlyoom - Early OOM Daemon for Linux
C
2,937
star
3

systembus-notify

systembus-notify - system bus notification daemon
C
57
star
4

eme

EME (Encrypt-Mix-Encrypt) wide-block encryption for Go.
Go
39
star
5

gocryptfs-website

https://nuetzlich.net/gocryptfs
HTML
16
star
6

fuse-xfstests

xfstests adapted for gocryptfs, from https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
Shell
11
star
7

the-fuse-wire-protocol

Document describing the filesystem in userspace wire protocol
Makefile
9
star
8

barwin-arduino

Arduino code for the Barwin cocktail robot
C++
4
star
9

save-a-file

Toy program for benchmarking safe and unsafe ways of saving a file
Go
3
star
10

fusecompress0

Git mirror of the fusecompress 0.9 tree by Ulrich Hecht, https://code.google.com/archive/p/fusecompress/
C
3
star
11

mediashrink

Collection of bash scripts to resize and transcode image and video files to save space
Shell
3
star
12

HelmholtzM

Code for the 1300mm setup. Matlab app that controls a three-axes Helmholtz coil setup to generate rotating magnetic fields
C
3
star
13

unhandled_exit

Python package that exits the whole process if any thread throws an unhandled exception
Python
2
star
14

umemtester

umemtester - Micro-Memtester
C
2
star
15

barwin-pc

PC software for the Barwin cocktail robot
Java
2
star
16

github-actions-3729

https://github.com/actions/virtual-environments/issues/3729
Shell
1
star
17

fallocate_write

Minimal application to reproduce the "no space left on device" errors that are seen when running gocryptfs on top of Btrfs
C
1
star
18

test-build-linux

Tries to build the linux kernel using github actions
1
star
19

access

Trivial access(1) implementations in Go and C
Go
1
star
20

fuse

Mirror of the FUSE (Filesystem in Userspace) project. Cloned from http://sourceforge.net/p/fuse/fuse/ci/master/tree/ . Homepage: http://fuse.sourceforge.net/
C
1
star
21

errno.html

https://nuetzlich.net/errno.html
HTML
1
star
22

mmapwrite

Simple test tool that mmaps a file and writes to it in a loop. Created to track down a FUSE performance regression in the Linux kernel.
C
1
star
23

rust-pretty-version

Returns the a version string that looks like `0.9.4.234-07f1468-dirty`
Rust
1
star