• Stars
    star
    205
  • Rank 190,665 (Top 4 %)
  • Language
    Python
  • License
    MIT License
  • Created almost 4 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Automatically create YARA rules from malicious documents.

Halogen


Halogen is a tool to automate the creation of yara rules based on the image files embedded within a malicious document. This can assist cyber security professionals in writing detection rules for malicious threats as well as help responders in identifying with particular threat they are dealing with. Currently, Halogen is able to create rules based on JPG and PNG files.


Halogen Walkthrough

Halogen help

python3 halogen.py -h
usage: halogen.py [-h] [-f FILE] [-d DIR] [-n NAME] [--png-idat] [--jpg-sos] [--jpg-sof2sos] [--jpg-jump] [-c CONTAINER] [--clam] [--rprefix RPREFIX]

Halogen: Automatically create yara rules based on images embedded in office documents.

optional arguments:
  -h, --help            show this help message and exit
  -f FILE, --file FILE  File to parse
  -d DIR, --directory DIR
                        directory to scan for image files.
  -n NAME, --rule-name NAME
                        specify a custom name for the rule file
  --png-idat            For PNG matches, instead of starting with the PNG file header, start with the IDAT chunk.
  --jpg-sos             For JPG matches, skip over the header and look for the Start of Scan marker, and begin the match there.
  --jpg-sof2sos         for JPG matches, skip over the header and match the SOF all the way to the SOS + 45 bytes of the data within the SOS.
  --jpg-jump            for JPG matches, skip over the header and identify the sof, the sos and then read the actual image data take that data and look for repeated bytes. Skip those bytes and then
                        create 45 bytes of raw image data.
  -c CONTAINER, --container CONTAINER
                        specify a clamav container type defaults to CL_TYPE_MSOLE2, CL_TYPE_OOXML_WORD, CL_TYPE_OOXML_XL, CL_TYPE_OOXML_PPT
  --clam                generate a clam rule instead of a yara rule
  --rprefix RPREFIX     specify a clamav ruleset prefix


Testing it out

We've included some test document files with embedded images for you to test this out with. Running python3 halogen/halogen.py -d tests/ > /tmp/halogen_test.yara will produce the test yara file containing all images found within the files inside the tests/ directory.
From here you can run yara -d /tmp/halogen_test.yara tests/ and observe which images match which files.

Notes

  1. We use two patterns for JPG matching. One is less strict than the typical JPG file header, and we use this because we've seen some malicious files match this pattern. If Halogen finds both, it'll default to writing out the more strict match. Typically, these have the same matching content, so no detection really gets missed.
  2. For PNG files you can choose to start by default at the file header, or with --png-idat you can start at the IDAT chunk found within a PNG file. We also reduced the bytes returned when matching on the IDAT chunk.
  3. Similar to the above, you can start JPG matches at the Start of Scan marker by using the --jpg-sos flag.
  4. Because of how the SOS section of the JPG file works, we've also included an optional --jpg-sof2sos flag, which reads the Start of Frame (SOF) marker until the SOS is found, and then reads an additional 45 bytes. This is useful if the the stardard --jpg-sos is giving you false positives.
  5. In an effort to reduce false positives, we've added in the --jpg-jump flag which reads the compressed image data and creates a hex jump in the yara output if it finds repeated image bytes. This allows us to match on the SOF and SOS of the file, as well as some of the more unique data in the image.

Contributing

Please contribute pull requests in python3, and submit any bugs you find as issues.

More Repositories

1

goalert

Open source on-call scheduling, automated escalations, and notifications so you never miss a critical alert
Go
2,215
star
2

lorri

Your project's nix-env
Rust
990
star
3

strelka

Real-time, container-based file scanning at enterprise scale
Python
859
star
4

matrixprofile-ts

A Python library for detecting patterns and anomalies in massive datasets using the Matrix Profile
Python
734
star
5

flottbot

A chatbot framework written in Go. All configurations are made in YAML files, or inside scripts written in your favorite language.
Go
333
star
6

pod-reaper

Rule based pod killing kubernetes controller
Go
197
star
7

portauthority

API that leverages Clair to scan Docker Registries and Kubernetes Clusters for vulnerabilities
Go
151
star
8

huntlib

A Python library to help with some common threat hunting data analysis operations
Python
131
star
9

mmk-ui-api

UI, API, and Scanner (Rules Engine) services for Merry Maker
TypeScript
119
star
10

row-types

A Haskell library for open records and variants using closed type families and type literals
Haskell
112
star
11

data-validator

A tool to validate data, built around Apache Spark.
Scala
100
star
12

libdart

A High Performance, Network Optimized, JSON Library
C++
80
star
13

XCBBuildServiceProxy

A framework to create proxies for XCBBuildService, which allows for custom Xcode build integrations.
Swift
74
star
14

strelka-ui

Strelka Web UI for File Submission and Analysis
JavaScript
51
star
15

graphql-liftoff

Generate GraphQL schema language from API specifications and more
TypeScript
44
star
16

nix-fetchers

A set of morally pure fetching builtins for Nix.
Python
43
star
17

grease

Automated Scripting Engine For the Modern Age
Python
42
star
18

react-native-svg-parser

Parses SVG files and converts to 'react-native-svg' format objects. NOTE: This project has been archived.
JavaScript
42
star
19

theta-idl

Define communication protocols between applications using algebraic data types.
Haskell
41
star
20

Threat-Hunting

Detection of obfuscated Powershell commands
Jupyter Notebook
41
star
21

f5-bigip-cookbook

Chef cookbook for F5 Big IP
Ruby
37
star
22

winnaker

An audit tool that tests the whole system functionality of Spinnaker
Python
33
star
23

go-arty

Go client library for Artifactory and Xray
Go
31
star
24

captains-log

A continuous integration plugin that helps organize release information in slack
JavaScript
31
star
25

attack-navigator-docker

A simple Docker container that serves the MITRE ATT&CK Navigator web app
Makefile
26
star
26

POSSUM

Java
24
star
27

reuse

A simple Golang app to test TCP and SSL/TLS session reuse.
Go
22
star
28

karmabot

A karmabot for Slack
Python
21
star
29

edge-mac-integrations

A collection scripts and API interactions used by Target to simplify the user experience and make Jamf Pro Self Service the one stop shop for access, peripherals, and software.
Shell
19
star
30

statsd-kafka-backend

A Statsd backend for sending metrics to Kafka
JavaScript
18
star
31

network_interfaces_v2-cookbook

Chef cookbook for managing network interfaces on Ubuntu, RHEL and Windows
Ruby
17
star
32

impeller

Manage Helm charts in Kubernetes clusters.
Go
15
star
33

table-model

Supercharge your datagrid with TableModel
JavaScript
15
star
34

REDstack

REDstack - Hadoop as a service on OpenStack
Python
15
star
35

lite-for-jdbc

Lightweight library to simplify JDBC database access
Kotlin
15
star
36

secured-yarn-cluster-ansible

Ansible playbook for provisioning secured yarn cluster
Ruby
14
star
37

casper-auto-provisioning

Shell
13
star
38

jenkins-docker-master

Dockerfile for Jenkins master
Shell
12
star
39

sccmosd-refresh-multitool

A method to migrate from Windows 7 (w/ BIOS) to Windows 10 (w/ UEFI) In A Single Task Sequence
PowerShell
9
star
40

markdown-inject

Add file or command output to markdown documents.
TypeScript
9
star
41

jenkins-docker-nginx

Dockerfile for NGINX frontend to Jenkins
Shell
9
star
42

intellidiff

Kotlin
9
star
43

emoji_manager

Custom emoji management for Enterprise Slack users
Kotlin
8
star
44

cloudpunch

A framework to performance test OpenStack at scale
Python
8
star
45

native_memory_allocator

A Kotlin library providing a simple, high-performance way to use off-heap native memory in JVM applications.
Kotlin
8
star
46

boots_of_haste

This script parses through an Nmap XML file and sends requests through Burp for every open port.
Python
7
star
47

gelvedere

Cli to deploy a Jenkins master
Go
7
star
48

consensource-api

API for sending batches to the process, and reading from pg
Rust
6
star
49

jenkins-docker-api

An API to manage containerized Jenkins masters
Go
6
star
50

sensu-go-goalert

Sensu Go GoAlert Handler
Go
6
star
51

cartster

Target Partner's Commerce Basket Transfer Example App
JavaScript
6
star
52

consensource-compose

INACTIVE REPO! Please visit github.com/target/consensource
Shell
5
star
53

consensource-database

Reporting database
Rust
5
star
54

consensource-processor

Transaction processor, smart contracts
Rust
5
star
55

pacemaker-cookbook

Chef cookbook for managing pacemaker on RHEL
Ruby
5
star
56

consensource-common

Data models, addressing, and dockerfiles
Rust
4
star
57

consensource-ui

Web client to interface with the ConsenSource blockchain
JavaScript
4
star
58

coldsalt

(THIS REPO HAS BEEN ARCHIVED) API test automation
Python
4
star
59

jupyter-git-extension

Extension that adds basic git functionality to the Jupyter Notebook UI
JavaScript
4
star
60

DataStoreExplorer

Kotlin
3
star
61

plugin-for-rundeck-to-execute-sap-modules

Rundeck plugin for connecting to SAP systems for triggering ABAP programs and process chains
Java
3
star
62

mmk-js-scope

Puppeteer worker for Merry Maker
TypeScript
3
star
63

consensource-docs

WIP: Docsite
CSS
3
star
64

consensource-sds

An event subscriber for publishing blockchain events to an off-chain reporting database
Rust
3
star
65

osx-edgelab

Python
2
star
66

pull-request-code-coverage

A continuous integration plugin to allow detecting code coverage for only the lines changed in a PR.
Go
2
star
67

mmk-types

JavaScript
2
star
68

woozie

An Emacs package for creating and validating Apache Oozie workflows
Emacs Lisp
2
star
69

consensource-cli

CLI for testing transactions and genesis
Rust
2
star
70

flottbot-docs

Documentation for flottbot
JavaScript
2
star
71

concatenated-barcode-parser

This library has logic to parse GS1-128 (Global Standard 1) concatenated barcode and return a list of parsed objects
Kotlin
2
star
72

hdp-cloud

Ruby
1
star
73

burndown-for-github-projects

TypeScript
1
star
74

setupcfg2nix

Generate nix expressions from setup.cfg for a python package.
Python
1
star
75

k-sim

A simple simulator trying to work through bottleneck/constraints theory as applied to a few simple Kafka topologies.
JavaScript
1
star
76

compiler-of-android-for-lona

The Android Compiler for Lona Design Systems
FreeMarker
1
star
77

token-manager-for-salesforce

Spring Boot library to make Salesforce API calls easy
Java
1
star
78

chatops-docs

Terms & Conditions for using Slack at Target
HTML
1
star
79

Schema-Check-filter-for-Logstash

(This repo is archived) Schema Check filter for Logstash
Ruby
1
star