• Stars
    star
    1,795
  • Rank 25,884 (Top 0.6 %)
  • Language
    C++
  • License
    GNU General Publi...
  • Created over 6 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A userspace out-of-memory killer

oomd

Support Ukraine Build Status

oomd is userspace Out-Of-Memory (OOM) killer for linux systems.

Background

Out of memory killing has historically happened inside kernel space. On a memory overcommitted linux system, malloc(2) and friends usually never fail. However, if an application dereferences the returned pointer and the system has run out of physical memory, the linux kernel is forced to take extreme measures, up to and including killing processes. This is sometimes a slow and painful process because the kernel can spend an unbounded amount of time swapping in and out pages and evicting the page cache. Furthermore, configuring policy is not very flexible while being somewhat complicated.

oomd aims to solve this problem in userspace. oomd leverages PSI and cgroupv2 to monitor a system holistically. oomd then takes corrective action in userspace before an OOM occurs in kernel space. Corrective action is configured via a flexible plugin system, in which custom code can be written. By default, this involves killing offending processes. This enables an unparalleled level of flexibility where each workload can have custom protection rules. Furthermore, time spent livelocked in kernelspace is minimized. In practice at Facebook, we've regularly seen 30 minute host lockups go away entirely.

Installing on Debian 11+ or Ubuntu 20.04+

# apt install oomd

Installing from RPMs on Fedora

oomd is packaged in Fedora as of Fedora 32 and can be installed with:

$ sudo dnf install oomd

Finally, enable and start it with:

$ sudo systemctl enable --now oomd.service

Building from source

Note that oomd requires PSI to function. This kernel feature has been merged into the 4.20 release.

oomd currently depends on meson and jsoncpp. libsystemd is an optional dependency.

oomd also requires GCC 8+ or clang 6+. Other compilers have not been tested.

$ git clone https://github.com/facebookincubator/oomd
$ cd oomd/oomd
$ meson build && ninja -C build
$ cd build && sudo ninja install

Configuration

See docs/configuration.md for a high level overview and some examples.

See docs/core_plugins.md for a quick reference on core plugin capabilities.

See docs/production_setup.md for guidelines on how oomd should be set up in a production environment.

Running tests

oomd depends on gtest/gmock to run tests. Installing gtest/gmock from master is preferred.

If meson detects gtest/gmock is installed, meson will generate build rules for tests.

$ cd oomd
$ rm -rf build
$ meson build && ninja test -C build

Writing custom plugins

It is both possible and encouraged to write custom plugins. The codebase is designed to make writing plugins as easy as possible.

See docs/writing_a_plugin.md for a tutorial.

Help / Discussion / Support

Join our #oomd channel on irc.freenode.net!

License

oomd is GPL 2 licensed, as found in the LICENSE file.

More Repositories

1

SocketRocket

A conforming Objective-C WebSocket client library.
Objective-C
9,534
star
2

katran

A high performance layer 4 load balancer
C
4,674
star
3

AITemplate

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.
Python
4,515
star
4

velox

A C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.
C++
3,474
star
5

cinder

Cinder is Meta's internal performance-oriented production version of CPython.
Python
3,458
star
6

FBX2glTF

A command-line tool for the conversion of 3D model assets on the FBX file format to the glTF file format.
C++
2,060
star
7

spectrum

A client-side image transcoding library.
C++
1,987
star
8

fastmod

A fast partial replacement for the codemod tool
Rust
1,648
star
9

xar

executable archive format
Python
1,571
star
10

Bowler

Safe code refactoring for modern Python.
Python
1,532
star
11

submitit

Python 3.8+ toolbox for submitting jobs to Slurm
Python
1,245
star
12

gloo

Collective communications library with various primitives for multi-machine training.
C++
1,181
star
13

fizz

C++14 implementation of the TLS-1.3 standard
C++
1,128
star
14

dhcplb

dhcplb is Facebook's implementation of a load balancer for DHCP.
Go
1,046
star
15

below

A time traveling resource monitor for modern Linux systems
Rust
1,029
star
16

OnlineSchemaChange

A tool for performing online schema changes on MySQL.
Python
965
star
17

Glean

System for collecting, deriving and working with facts about source code.
Hack
923
star
18

Battery-Metrics

Library that helps in instrumenting battery related system metrics.
Java
736
star
19

retrie

Retrie is a powerful, easy-to-use codemodding tool for Haskell.
Haskell
500
star
20

superconsole

The superconsole crate provides a handler and building blocks for powerful, yet minimally intrusive TUIs. It is cross platform, supporting Windows 7+, Linux, and MacOS. Rustaceans who want to create non-interactive TUIs can use the component composition building block system to quickly deploy their code.
Rust
477
star
21

nvdtools

A set of tools to work with the feeds (vulnerabilities, CPE dictionary etc.) distributed by National Vulnerability Database (NVD)
Go
446
star
22

nimble

New file format for storage of large columnar datasets.
C++
419
star
23

infima

A UI framework that provides websites with the minimal CSS and JS needed to get started with building a modern responsive beautiful website
HTML
405
star
24

CG-SQL

CG/SQL is a compiler that converts a SQL Stored Procedure like language into C for SQLite. SQLite has no stored procedures of its own. CG/CQL can also generate other useful artifacts for testing and schema maintenance.
HTML
391
star
25

TTPForge

The TTPForge is a Cybersecurity Framework for developing, automating, and executing attacker Tactics, Techniques, and Procedures (TTPs).
Go
320
star
26

flowtorch

This library would form a permanent home for reusable components for deep probabilistic programming. The library would form and harness a community of users and contributors by focusing initially on complete infra and documentation for how to use and create components.
Jupyter Notebook
300
star
27

ptr

Python Test Runner.
Python
284
star
28

senpai

Senpai is an automated memory sizing tool for container applications.
Python
270
star
29

fbjni

A library designed to simplify the usage of the Java Native Interface
C++
260
star
30

dynolog

Dynolog is a telemetry daemon for performance monitoring and tracing. It exports metrics from different components in the system like the linux kernel, CPU, disks, Intel PT, GPUs etc. Dynolog also integrates with pytorch and can trigger traces for distributed training applications.
C++
251
star
31

gazebo

A Rust library containing a collection of small well-tested primitives.
Rust
235
star
32

reindeer

Reindeer is a tool to transform Rust Cargo dependencies into generated Buck build rules
Rust
177
star
33

dispenso

The project provides high-performance concurrency, enabling highly parallel computation.
C++
174
star
34

GeoLift

GeoLift is an end-to-end geo-experimental methodology based on Synthetic Control Methods used to measure the true incremental effect (Lift) of ad campaign.
R
171
star
35

oculus-linux-kernel

The Linux kernel code for Oculus devices
C
156
star
36

FCR

FBNet-Command-Runner: A thrift service to run commands on heterogeneous Network devices with configurable parameters.
154
star
37

dataclassgenerate

DataClassGenerate (or simply DCG) is a Kotlin compiler plugin that addresses an Android APK size overhead from Kotlin data classes.
Kotlin
154
star
38

hsthrift

The Haskell Thrift Compiler. This is an implementation of the Thrift spec that generates code in Haskell. It depends on the fbthrift project for the implementation of the underlying transport.
Haskell
148
star
39

FioSynth

Tool which enables the creation of synthetic storage workloads, automates the execution and results collection of synthetic storage benchmarks.
Python
140
star
40

meta-code-verify

Code Verify is an open source web browser extension that confirms that your Facebook, Messenger, Instagram, and WhatsApp Web code hasn’t been tampered with or altered, and that the Web experience you’re getting is the same as everyone else’s.
TypeScript
137
star
41

tacquito

Tacquito is an open source TACACs+ server written in Go that implements RFC8907
Go
93
star
42

go-qfext

a fast counting quotient filter implementation in golang
Go
91
star
43

momentum

A library for human kinematic motion and numerical optimization solvers to apply human motion
C++
89
star
44

ForgeArmory

ForgeArmory provides TTPs that can be used with the TTPForge (https://github.com/facebookincubator/ttpforge).
Swift
80
star
45

antlir

ANoTher Linux Image buildeR
Rust
76
star
46

sks

Secure Key Storage (SKS) is a library for Go that abstracts Security Hardware on laptops.
Go
72
star
47

dcrpm

A tool to detect and correct common issues around RPM database corruption.
Python
72
star
48

ConversionsAPI-Tag-for-GoogleTagManager

This repository will contain the artifacts needed for setting up Conversions API implementation on Google Tag Manager's serverside. Please follow the instructions https://www.facebook.com/business/help/702509907046774
Smarty
64
star
49

InjKit

Injection Kit. It is a java bytecode processing library for bytecode injection and transformation.
Java
55
star
50

obs-plugins

OBS Plugins
C++
54
star
51

glTFVariantMeld

An application that accepts files on the glTF format, interprets them as variants of an over-arching whole, and melds them together.
Rust
51
star
52

haberdashery

A collection of high-performance crypto implementations.
Rust
44
star
53

later

A framework for python asyncio with batteries included for people writing services in python asyncio
Python
39
star
54

go2chef

A Golang tool to bootstrap a system from zero so that it's able to run Chef to be managed
Go
39
star
55

CommutingZones

Commuting zones are geographic areas where people live and work and are useful for understanding local economies, as well as how they differ from traditional boundaries. These zones are a set of boundary shapes built using aggregated estimates of home and work locations. Data used to build commuting zones is aggregated and de-identified.
JavaScript
39
star
56

ConversionsAPI-Client-for-GoogleTagManager

This repository will contain the artifacts needed for setting up Conversions API implementation on Google Tag Manager's serverside. Primarily we will be hosting, - ConversionsAPI(Facebook) Client - listens on the events fired to GTM Server and maps them to common GTM schema. - ConversionsAPI(Facebook) Tag - server tag that fires events to CAPI.For more details on Design here https//fburl.com/uae68vlr
37
star
57

Facebook-Pixel-for-Wordpress

A plugin for advertisers who use Wordpress to enable them easily setup the facebook pixel.
JavaScript
37
star
58

strobelight

Meta's fleetwide profiler framework
C++
28
star
59

buck2-change-detector

Given a Buck2 built project and a set of changes (e.g. from source control) compute the targets that may have changed. Sometimes known as a target determinator, useful for optimizing a CI system.
Rust
28
star
60

wordpress-messenger-customer-chat-plugin

Messenger Customer Chat Plugin for WordPress
PHP
27
star
61

CP4M

CP4M is a conversational marketing platform which enables advertisers to integrate their customer-facing chatbots with FB Messenger/WhatsApp, in order to meet customers where they are and drive native conversations on the advertiser's owned infra.
Java
27
star
62

rush

RUSH (Reliable - unreliable - Streaming Protocol)
C++
26
star
63

MY_ENUM

Small c++ macro library to add compile-time introspection to c++ enum classes.
C++
17
star
64

SafeC

Library containing safer alternatives/wrappers for insecure C APIs.
C++
16
star
65

go-belt

It is an implementation-agnostic Go(lang) package to generalize observability tooling (logger, metrics, tracer and so on) and provide ability to use any of these tools with a standard context. Essentially it is an attempt to standardize observability API in Go.
Go
16
star
66

spark-ar-core-libs

Core libraries that can be used in Spark AR. You can import each library depends on your requirements.
TypeScript
15
star
67

scrut

Scrut is a testing toolkit for CLI applications. A tool to scrutinize terminal programs without fuss.
Rust
15
star
68

sado

A macOS signed-app shim for running daemons with reliable capabilities.
Swift
12
star
69

Portal-Kernel

Kernel Code for Portal.
C
11
star
70

npe-toolkit

Libraries, guides, blueprints, and sample code, to enable rapidly building 0-1 applications on iOS, Android and web.
TypeScript
10
star
71

Eigen-FBPlugins

This is collection of plugins extending Eigen arrays/matrices with main focus on using them for computer vision. In particular, this project should provide support for multichannel arrays (missing in vanilla Eigen) and seamless integration between Eigen types and OpenCV functions.
C++
9
star
72

isometric_pattern_matcher

A new isometric calibration pattern - which should/might lead to higher accuracy calibrations compared to existing solutions (checkerboards, patterns of circles).
C++
8
star
73

dnf-plugin-cow

Code to enable Copy on Write features being upstreamed in rpm and librepo
Shell
8
star
74

jupyterhub_fb_authenticator

JupyterHub Facebook Authenticator is a Facebook OAuth authenticator built on top of OAuthenticator.
Python
8
star
75

wireguard_py

Cython library for Wireguard
C
7
star
76

kernel-patches-daemon

Sync Patchwork series's with Github pull requests
Python
6
star
77

meta-fbvuln

OpenEmbedded meta-layer that allows producing a vulnerability manifest alongside a Yocto build. The produced manifest is suitable for ongoing vulnerability scanning of fielded software.
5
star
78

gazebo_lint

A Rust linter that provides various suggestions based on the new primitives offered in the `gazebo` library.
Rust
4
star
79

language-capirca

Adds syntax highlighting for Capirca filetypes in Atom. Capirca is an open source standard for writing vendor-neutral firewall policies as originally released by Google: https://github.com/google/capirca
3
star
80

cinderx

cinderx
C++
3
star
81

fbc_owrt_feed

Facebook Connectivity OpenWrt Feed. Package feed for OpenWrt router OS by Facebook Connectivity programme.
Lua
2
star
82

cutlass-fork

A Meta fork of NV CUTLASS repo.
C++
2
star
83

hwbits_lib

Abstraction of hardware register-level protocols in a python semantic names.
Python
1
star