• Stars
    star
    2,076
  • Rank 22,254 (Top 0.5 %)
  • Language
    Go
  • License
    MIT License
  • Created about 9 years ago
  • Updated 9 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Fast, dependency-free Go package to infer binary file types based on the magic numbers header signature

filetype GoDoc Go Version

Small and dependency free Go package to infer file and MIME type checking the magic numbers signature.

For SVG file type checking, see go-is-svg package. Python port: filetype.py.

Features

  • Supports a wide range of file types
  • Provides file extension and proper MIME type
  • File discovery by extension or MIME type
  • File discovery by class (image, video, audio...)
  • Provides a bunch of helpers and file matching shortcuts
  • Pluggable: add custom new types and matchers
  • Simple and semantic API
  • Blazing fast, even processing large files
  • Only first 262 bytes representing the max file header is required, so you can just pass a slice
  • Dependency free (just Go code, no C compilation needed)
  • Cross-platform file recognition

Installation

go get github.com/h2non/filetype

API

See Godoc reference.

Subpackages

Examples

Simple file type checking

package main

import (
  "fmt"
  "io/ioutil"

  "github.com/h2non/filetype"
)

func main() {
  buf, _ := ioutil.ReadFile("sample.jpg")

  kind, _ := filetype.Match(buf)
  if kind == filetype.Unknown {
    fmt.Println("Unknown file type")
    return
  }

  fmt.Printf("File type: %s. MIME: %s\n", kind.Extension, kind.MIME.Value)
}

Check type class

package main

import (
  "fmt"
  "io/ioutil"

  "github.com/h2non/filetype"
)

func main() {
  buf, _ := ioutil.ReadFile("sample.jpg")

  if filetype.IsImage(buf) {
    fmt.Println("File is an image")
  } else {
    fmt.Println("Not an image")
  }
}

Supported type

package main

import (
  "fmt"

  "github.com/h2non/filetype"
)

func main() {
  // Check if file is supported by extension
  if filetype.IsSupported("jpg") {
    fmt.Println("Extension supported")
  } else {
    fmt.Println("Extension not supported")
  }

  // Check if file is supported by extension
  if filetype.IsMIMESupported("image/jpeg") {
    fmt.Println("MIME type supported")
  } else {
    fmt.Println("MIME type not supported")
  }
}

File header

package main

import (
  "fmt"
  "os"

  "github.com/h2non/filetype"
)

func main() {
  // Open a file descriptor
  file, _ := os.Open("movie.mp4")

  // We only have to pass the file header = first 261 bytes
  head := make([]byte, 261)
  file.Read(head)

  if filetype.IsImage(head) {
    fmt.Println("File is an image")
  } else {
    fmt.Println("Not an image")
  }
}

Add additional file type matchers

package main

import (
  "fmt"

  "github.com/h2non/filetype"
)

var fooType = filetype.NewType("foo", "foo/foo")

func fooMatcher(buf []byte) bool {
  return len(buf) > 1 && buf[0] == 0x01 && buf[1] == 0x02
}

func main() {
  // Register the new matcher and its type
  filetype.AddMatcher(fooType, fooMatcher)

  // Check if the new type is supported by extension
  if filetype.IsSupported("foo") {
    fmt.Println("New supported type: foo")
  }

  // Check if the new type is supported by MIME
  if filetype.IsMIMESupported("foo/foo") {
    fmt.Println("New supported MIME type: foo/foo")
  }

  // Try to match the file
  fooFile := []byte{0x01, 0x02}
  kind, _ := filetype.Match(fooFile)
  if kind == filetype.Unknown {
    fmt.Println("Unknown file type")
  } else {
    fmt.Printf("File type matched: %s\n", kind.Extension)
  }
}

Supported types

Image

  • jpg - image/jpeg
  • png - image/png
  • gif - image/gif
  • webp - image/webp
  • cr2 - image/x-canon-cr2
  • tif - image/tiff
  • bmp - image/bmp
  • heif - image/heif
  • jxr - image/vnd.ms-photo
  • psd - image/vnd.adobe.photoshop
  • ico - image/vnd.microsoft.icon
  • dwg - image/vnd.dwg
  • avif - image/avif

Video

  • mp4 - video/mp4
  • m4v - video/x-m4v
  • mkv - video/x-matroska
  • webm - video/webm
  • mov - video/quicktime
  • avi - video/x-msvideo
  • wmv - video/x-ms-wmv
  • mpg - video/mpeg
  • flv - video/x-flv
  • 3gp - video/3gpp

Audio

  • mid - audio/midi
  • mp3 - audio/mpeg
  • m4a - audio/mp4
  • ogg - audio/ogg
  • flac - audio/x-flac
  • wav - audio/x-wav
  • amr - audio/amr
  • aac - audio/aac
  • aiff - audio/x-aiff

Archive

  • epub - application/epub+zip
  • zip - application/zip
  • tar - application/x-tar
  • rar - application/vnd.rar
  • gz - application/gzip
  • bz2 - application/x-bzip2
  • 7z - application/x-7z-compressed
  • xz - application/x-xz
  • zstd - application/zstd
  • pdf - application/pdf
  • exe - application/vnd.microsoft.portable-executable
  • swf - application/x-shockwave-flash
  • rtf - application/rtf
  • iso - application/x-iso9660-image
  • eot - application/octet-stream
  • ps - application/postscript
  • sqlite - application/vnd.sqlite3
  • nes - application/x-nintendo-nes-rom
  • crx - application/x-google-chrome-extension
  • cab - application/vnd.ms-cab-compressed
  • deb - application/vnd.debian.binary-package
  • ar - application/x-unix-archive
  • Z - application/x-compress
  • lz - application/x-lzip
  • rpm - application/x-rpm
  • elf - application/x-executable
  • dcm - application/dicom

Documents

  • doc - application/msword
  • docx - application/vnd.openxmlformats-officedocument.wordprocessingml.document
  • xls - application/vnd.ms-excel
  • xlsx - application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
  • ppt - application/vnd.ms-powerpoint
  • pptx - application/vnd.openxmlformats-officedocument.presentationml.presentation

Font

  • woff - application/font-woff
  • woff2 - application/font-woff
  • ttf - application/font-sfnt
  • otf - application/font-sfnt

Application

  • wasm - application/wasm
  • dex - application/vnd.android.dex
  • dey - application/vnd.android.dey

Benchmarks

Measured using real files.

Environment: OSX x64 i7 2.7 Ghz

BenchmarkMatchTar-8    1000000        1083 ns/op
BenchmarkMatchZip-8    1000000        1162 ns/op
BenchmarkMatchJpeg-8   1000000        1280 ns/op
BenchmarkMatchGif-8    1000000        1315 ns/op
BenchmarkMatchPng-8    1000000        1121 ns/op

License

MIT - Tomas Aparicio

More Repositories

1

imaginary

Fast, simple, scalable, Docker-ready HTTP microservice for high-level image processing
Go
5,543
star
2

toxy

Hackable HTTP proxy for resiliency testing and simulated network conditions
JavaScript
2,733
star
3

bimg

Go package for fast high-level image processing powered by libvips C library
Go
2,683
star
4

gock

HTTP traffic mocking and testing made easy in Go ΰΌΌΚ˜ΜšΩ„ΝœΚ˜ΜšΰΌ½
Go
2,067
star
5

gentleman

Plugin-driven, extensible HTTP client toolkit for Go
Go
1,072
star
6

videoshow

Simple node.js utility to create video slideshows from images with optional audio and visual effects using ffmpeg
JavaScript
873
star
7

baloo

Expressive end-to-end HTTP API testing made easy in Go
Go
776
star
8

jshashes

Fast and dependency-free cryptographic hashing library for node.js and browsers (supports MD5, SHA1, SHA256, SHA512, RIPEMD, HMAC)
JavaScript
698
star
9

filetype.py

Small, dependency-free, fast Python package to infer binary file types checking the magic numbers signature
Python
642
star
10

nar

node.js application archive - create self-contained binary like executable applications that are ready to ship and run
LiveScript
428
star
11

rocky

Full-featured, middleware-oriented, programmatic HTTP and WebSocket proxy for node.js (deprecated)
JavaScript
371
star
12

pook

HTTP traffic mocking and testing made easy in Python
Python
329
star
13

paco

Small utility library for coroutine-driven asynchronous generic programming in Python
Python
202
star
14

semver.c

Semantic version in ANSI C
C
185
star
15

riprova

Versatile async-friendly retry package with multiple backoff strategies
Python
116
star
16

node-imaginary

Minimalist node.js command-line & programmatic API client for imaginary
JavaScript
106
star
17

youtube-video-api

Simplified programmatic and command-line interface for YouTube Video API. Built for node.js
JavaScript
67
star
18

siringa

Minimalist dependency injection library for Python that embraces type annotations syntax
Hy
54
star
19

audioconcat

Tiny node.js module to concat multiple audio files using ffmpeg
JavaScript
45
star
20

requireg

Resolve and require local & global modules in node.js like a boss
JavaScript
45
star
21

thread.js

Frictionless library to deal with multithread programming in the browser
JavaScript
41
star
22

grunt-stubby

Grunt task to setup a stub/mock server based on JSON/YAML/JS configuration files
JavaScript
26
star
23

hu

Small, generic functional helper library for node.js and browsers
wisp
20
star
24

gentleman-consul

Gentleman's plugin for Consul server discovery and optional configurable retry/backoff policies
Go
19
star
25

nightmare-google-oauth2

Nightmare plugin to retrieve a Google APIs OAuth2 token. Useful for server-to-server automation
JavaScript
19
star
26

balboa

Simple HTTP forward proxy
JavaScript
16
star
27

domlight

Spotlight DOM elements easily
JavaScript
16
star
28

resizr

Dead simple HTTP service written in Go for fast and easy image resizing from remote URLs
Go
16
star
29

google-oauth2-token

No headaches. Automation wins. Get a fresh OAuth2 token for Google APIs in just one command
JavaScript
16
star
30

angular-thread

AngularJS primitive bindings for thread.js
JavaScript
15
star
31

resilient-consul

Resilient.js middleware to use Consul for service discovery and balancing
JavaScript
14
star
32

gentleman-mock

HTTP mocking for gentleman's clients made easy. Uses gock under the hood
Go
14
star
33

rocky-consul

Rocky middleware for service discovery and dynamic traffic balancing using Consul
JavaScript
14
star
34

angular-resilient

Use $http as a resilient, failover and client-side balanced HTTP client
JavaScript
13
star
35

go-is-svg

Check if a given buffer is a valid SVG image in Go (golang)
Go
12
star
36

node-os-shim

Native OS module API shim for older node.js versions
JavaScript
12
star
37

stream-interceptor

Intercept, modify and/or ignore chunks data and events in any readable stream
JavaScript
11
star
38

gentleman-retry

gentleman's plugin providing retry policy capabilities in your HTTP clients
Go
11
star
39

bbscraper

Simple phpBB forum thread web scraper written in Python
Python
11
star
40

Sencha-WebSocket

WebSocket client abstraction library for Sencha JavaScript frameworks (DEPRECATED)
JavaScript
10
star
41

grunt-beautiful-docs

Generate beautiful markdown-based documentation using Grunt
CoffeeScript
9
star
42

findup

Find the first file matching in a current working directory or the nearest ancestor directory up to root using Go
Go
9
star
43

resolve-tree

Recursively resolve node.js modules and its dependencies looking into node_modules trees.
JavaScript
8
star
44

http-forward

Simple one-line proxy forward for incoming HTTP requests. Built for node.js/io.js
JavaScript
7
star
45

injecty

Micro library for dependency injection and inversion of control containers in node and browsers
wisp
6
star
46

sharedworkers-angular-poc

A simple proof of concept using Shared WebWorkers + Two-way data binding in AngularJS
CSS
6
star
47

toxy-ip

toxy rule to easily filter by IP addresses (supports CIDR, subnet, IP ranges...)
JavaScript
6
star
48

fw

Tiny library for asynchronous control-flow in node and browsers
wisp
6
star
49

pipefy

Transform a function into a pipeable writable stream in node/io.js
JavaScript
5
star
50

node-bintray

CLI and programmatic interface for Bintray built for node.js
CoffeeScript
5
star
51

generator-api-service

A Yeoman generator to build an opinionated but community-driven convention-focused, general purpose, resource-oriented HTTP service in node.js
JavaScript
4
star
52

htgen

Tiny hypertext markup code generator for node and browsers
LiveScript
4
star
53

node-authrc

authrc implementation for node
CoffeeScript
4
star
54

gulp-nar

Create and extract nar archives from Gulp
JavaScript
4
star
55

ng-exam

Are you a great AngularJS developer? (wip)
JavaScript
4
star
56

oml

Minimalist template engine built-on-top of the Oli language for node and browsers
JavaScript
4
star
57

buildspec

Declarative, featured, unopiniotated, CI-agnostic build configuration file
4
star
58

OPEW

[NOT MAINTAINED] OPEW is a powerful, complete, independent and extensible open source distribution stack for GNU/Linux (64 bits) based OS. Its goal is to provides a rich portable ready-to-run development environment focused on modern and robust programming languages in oder to satisfy the common needs for the web development.
Shell
4
star
59

promitto

Tiny promise library mostly compatible with Promise/A+ spec and ES6
wisp
3
star
60

heroku-buildpack-binary-download

Download and execute POSIX compatible binaries in Heroku apps
Shell
3
star
61

grunt-nar

Create and extract nar archives from Grunt
CoffeeScript
2
star
62

ASIR

ASIR course exercises for fun and profit
Shell
2
star
63

consul-cluster-test

Consul cluster test suite
Python
2
star
64

butler

A small, elegant and friendly library to deal elegantly with Service Workers (experimental)
JavaScript
2
star
65

node-cloudimage

Minimalist node/io.js CLI & programmatic stream-based interface for Cloudimage.io
JavaScript
2
star
66

mimeware

Simple node.js/io.js HTTP middleware to infer the proper MIME content type response header
JavaScript
2
star
67

bock

Browser next-generation HTTP traffic mock and proxy interceptor using Service Worker (experimental)
JavaScript
2
star
68

OPEW-bash-installer

Bash-based installer builder utility for the OPEW project
Shell
2
star
69

findup.rs

Find the first file matching in a current working directory or the nearest ancestor directory up to root
Rust
2
star
70

http-version

HTTP API version matching strategies as middleware for connect/express/rocky
JavaScript
2
star
71

nar-installer

Simple Bash script to easily install nar executable packages. Analog to npm install --global
Shell
2
star
72

http-api-versioning

Analysis of multiple HTTP API versioning design strategies
1
star
73

midware-pool

Tiny module to create a pool of connect-style domain-agnostic middleware layers in browsers and node.js
JavaScript
1
star
74

apachelog

Simple Go net/http compatible middleware for Apache style logging
Go
1
star
75

rocky-cli

Command-line interface for rocky. Supports TOML configuration file to easily set up your proxy
JavaScript
1
star
76

carcasse

Build structured, modular and object-oriented JavaScript projects
JavaScript
1
star
77

go-memstats

Human-friendly debugger to print in stdout the memory and GC stats
Go
1
star