• Stars
    star
    413
  • Rank 104,801 (Top 3 %)
  • Language
    Python
  • License
    GNU Affero Genera...
  • Created over 6 years ago
  • Updated 26 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

YARA malware query accelerator (web frontend)

mquery: Blazingly fast Yara queries for malware analysts

Ever had trouble searching for malware samples? Mquery is an analyst-friendly web GUI to look through your digital warehouse.

It can be used to search through terabytes of malware in a blink of an eye:

mquery web GUI

Under the hood we use our UrsaDB, to accelerate yara queries with ngrams.

Demo

Public instance will be created soon, stay tuned...

Quickstart

1. Install and start

The easiest way to do this is with docker-compose:

git clone --recurse-submodules https://github.com/CERT-Polska/mquery.git
cd mquery
vim .env  # optional - change samples and index directory locations
docker-compose up --scale daemon=3  # building the images will take a while

The web interface should be available at http://localhost.

(For more installation options see the installation manual ).

2. Add the files

Put some files in the SAMPLES_DIR (by default ./samples in the repository, configurable with variable in the .env file).

3. Index your collection

Launch ursacli in docker:

sudo docker-compose exec ursadb ursacli
[2023-06-14 17:20:24.940] [info] Connecting to tcp://localhost:9281
[2023-06-14 17:20:24.942] [info] Connected to UrsaDB v1.5.1+98421d7 (connection id: 006B8B46B6)
ursadb>

Index the samples with n-grams of your choosing (this may take a while!)

ursadb> index "/mnt/samples" with [gram3, text4, wide8, hash4];
[2023-06-14 17:29:27.672] [info] Working... 1% (109 / 8218)
[2023-06-14 17:29:28.674] [info] Working... 1% (125 / 8218)
...
[2023-06-14 17:37:40.265] [info] Working... 99% (8217 / 8218)
[2023-06-14 17:37:41.266] [info] Working... 99% (8217 / 8218)
{
    "result": {
        "status": "ok"
    },
    "type": "ok"
}

This will scan samples directory for all new files and index them. You can monitor the progress in the tasks window on the left:

You have to repeat this process every time you want to add new files!

After indexing is over, you will notice new datasets:

This is a good and easy way to start, but if you have a big collection you are strongly encouraged to read indexing page in the manual.

4. Test it

Now your files should be searchable - insert any Yara rule into the search window and click Query. Just for demonstration, I've indexed the source code of this application and tested this Yara rule:

rule mquery_exceptions {
    strings: $a = "Exception"
    condition: all of them
}

Learn more

See the documentation to learn more. Probably a good idea if you plan a bigger deployment.

You can also read the hosted version here: cert-polska.github.io/mquery/docs.

Installation

See the installation instruction.

Contributing

If you want to contribute, see our dedicated documentation for contributors.

Changelog

Learn how the project has changed by reading our release log.

Contact

If you have any problems, bugs or feature requests related to mquery, you're encouraged to create a GitHub issue.

If you have questions unsuitable for github, you can email CERT.PL ([email protected]) directly.

More Repositories

1

drakvuf-sandbox

DRAKVUF Sandbox - automated hypervisor-level malware analysis system
Python
1,043
star
2

Artemis

A modular vulnerability scanner with automatic report generation capabilities.
Python
548
star
3

karton

Distributed malware processing framework based on Python, Redis and S3.
Python
389
star
4

mwdb-core

Malware repository component for samples & static configuration with REST API interface.
Python
323
star
5

malduck

🦆 Malduck is your ducky companion in malware analysis journeys
Python
315
star
6

mailgoose

A web application that allows the users to check whether their SPF, DMARC and DKIM configuration is set up correctly.
Python
132
star
7

hfinger

Hfinger - fingerprinting HTTP requests
Python
131
star
8

ursadb

Trigram database written in C++, suited for malware indexing
C++
123
star
9

n6

Automated handling of data feeds for security teams
Python
121
star
10

phobos-cuda-decryptor-poc

C++
68
star
11

training-materials

PHP
50
star
12

mwdblib

Client library for the mwdb service by CERT Polska.
Python
40
star
13

hsn2-bundle

Shell
29
star
14

nymaim-tools

C
26
star
15

Artemis-modules-extra

Python
24
star
16

dbglib

C#
22
star
17

training-mwdb

MWDB exercises
Python
19
star
18

phishing-api

Python
17
star
19

karton-misp-pusher

Python
16
star
20

n6sdk

Server-side SDK to provide a simple REST API for distribution of security information.
Python
16
star
21

karton-playground

Python
15
star
22

anti-modlishka

Python
12
star
23

mtracker

Python
12
star
24

HSN-Capture-HPC-NG

HoneySpider Network version of Capture-HPC
C++
11
star
25

drakpdb

Convert Windows PDB into JSON profile supported by DRAKVUF/LibVMI
Python
10
star
26

mwdb_iocextract

Python
10
star
27

karton-config-extractor

Static configuration extractor for the Karton framework
Python
8
star
28

ursadb-cli

Lightweight Python client for ursadb
Python
8
star
29

karton-pcap-miner

Python
8
star
30

proactive-detection-survey

Reference material for the proactive detection of incidents survey
7
star
31

warning-list-tools

A scripts, utilities, and documentation intended to help with integrating CERT.PL's Dangerous websites Warning List.
PowerShell
6
star
32

karton-classifier

File type classifier for the Karton framework.
Python
6
star
33

mwdb-plugin-drakvuf

DRAKVUF Sandbox simple integration plugin for mwdb-core.
Python
6
star
34

karton-dashboard

A small Flask application that allows for Karton task and queue introspection.
HTML
6
star
35

karton-autoit-ripper

AutoIt script ripper for Karton framework
Python
6
star
36

swf-tools

Java
5
star
37

karton-yaramatcher

File and analysis artifacts yara matcher for Karton framework
Python
5
star
38

snort3-x509-reputation-plugin

Snort3 x509 reputation plugin
C++
5
star
39

karton-archive-extractor

Extractor of various archive formats for Karton framework
Python
5
star
40

hsn2-yara

Python
4
star
41

hsn2-cuckoo

Python
3
star
42

hsn2-pcap-extract

Python
3
star
43

hsn2-shell-scdbg

Java
3
star
44

karton-asciimagic

Various decoders for ascii-encoded executables for Karton framework
Python
3
star
45

malduck-modules

YARA
3
star
46

karton-mwdb-reporter

Karton service that uploads analyzed artifacts and metadata to MWDB Core
Python
2
star
47

hsn2-pcap-analyze

Python
2
star
48

hsn2-malicious-domains

Python
2
star
49

hsn2-unicorn

Java
2
star
50

hsn2-razorback

C
2
star
51

hsn2-framework

Java
2
star
52

hsn2-commons-python-lib

Python
2
star
53

python-deploy

Build, push and deploy k8s services with single deploy.json file to provide common convention for multiple production services.
Python
2
star
54

hsn2-js-sta

C++
1
star
55

hsn2-dnsinfo

Java
1
star
56

hsn2-object-store-mongodb

Java
1
star
57

hsn2-swf-cve

Java
1
star
58

hsn2-url-feeder

Python
1
star
59

hsn2-commons-python-protobuf

Python
1
star
60

hsn2-commons-java

Java
1
star
61

hsn2-cuckoo-java

Java
1
star
62

hsn2-norm-url

Java
1
star
63

hsn2-webclient

Java
1
star
64

hsn2-file-feeder

Java
1
star
65

hsn2-commons-debian

Shell
1
star
66

hsn2-console

Python
1
star
67

hsn2-capture-hpc

Java
1
star
68

hsn2-md5-to-ssdeep

Java
1
star
69

hsn2-thug

Python
1
star
70

hsn2-commons-protobuf-definitions

C++
1
star
71

hsn2-reporter

Java
1
star
72

hsn2-data-store

Java
1
star
73

lint-python-action

Python
1
star