• Stars
    star
    1,648
  • Rank 28,365 (Top 0.6 %)
  • Language
    Rust
  • License
    Apache License 2.0
  • Created over 6 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A fast partial replacement for the codemod tool

fastmod

fastmod is a fast partial replacement for codemod. Like codemod, it is a tool to assist you with large-scale codebase refactors, and it supports most of codemod's options. fastmod's major philosophical difference from codemod is that it is focused on improving the use case "I want to use interactive mode to make sure my regex is correct, and then I want to apply the regex everywhere". For this use case, it offers much better performance than codemod. Accordingly, fastmod does not support codemod's --start, --end, or --count options, nor does it support anything like codemod's Python API.

Examples

Let's say you're deprecating your use of the <font> tag. From the command line, you might make progress by running:

fastmod -m -d /home/jrosenstein/www --extensions php,html \
    '<font *color="?(.*?)"?>(.*?)</font>' \
    '<span style="color: ${1};">${2}</span>'

For each match of the regex, you'll be shown a colored diff and asked if you want to accept the change (the replacement of the <font> tag with a <span> tag), reject it, or edit the line in question in your $EDITOR of choice.

NOTE: Whereas codemod uses Python regexes, fastmod uses the Rust regex crate, which supports a slightly different regex syntax and does not support look around or backreferences. In particular, use ${1} instead of \1 to get the contents of the first capture group, and use $$ to write a literal $ in the replacement string. See the regex crate's documentation for details.

A consequence of this syntax is that the use of single quotes instead of double quotes around the replacement text is important, because the bash shell itself cares about the $ character in double-quoted strings. If you must double-quote your input text, be careful to escape $ characters properly!

fastmod also offers a usability improvement over codemod: it accepts files or directories to process as extra positional arguments after the regex and substitution. For instance, the example above could have been rewritten as

fastmod -m --extensions php,html \
    '<font *color="?(.*?)"?>(.*?)</font>' \
    '<span style="color: ${1};">${2}</span>' \
    /home/jrosenstein/www

This makes it possible to use fastmod to process a list of files from somewhere else if needed. Note, however, that fastmod does its own parallel directory traversal internally, so doing find ... | xargs fastmod ... may be much slower than using fastmod by itself.

Requirements

fastmod is primarily supported on macOS and Linux.

fastmod has also been reported to work reasonably well on Windows. The major portability concerns are 1) the use of $EDITOR with a fallback and 2) the console UI, which is descended from codemod's ncurses-based text coloring & screen clearing code. Windows-specific issues and PRs will be considered as long as they aren't too invasive. For example, if something doesn't work on Windows because a Linux/Mac-specific API was used instead of equivalent POSIX or Rust standard library calls, we would be happy to fix that. On the other hand, we would like to avoid taking a direct winapi dependency or substantially increasing the size of our dependency graph for Windows-only enhancements.

Building fastmod

fastmod is written in (stable) Rust and compiles with Rust's cargo build system. To build:

$ git clone https://github.com/facebookincubator/fastmod.git
$ cd fastmod
$ cargo build --release
$ ./target/release/fastmod --help
...

Installing fastmod

The easiest way to install fastmod is simply cargo install fastmod. If you have built fastmod from source following the directions above, you can install your build with cargo install.

How fastmod works

fastmod uses the ignore crate to walk the given directory hierarchy using multiple threads in parallel while respecting .gitignore. It uses the grep crate to match each file, reads matching files into memory, applies the given regex substitution one match at a time, and uses the diff crate to present the resulting changes as patches for human review.

Full documentation

See fastmod --help.

License

fastmod is Apache-2.0-licensed.

More Repositories

1

SocketRocket

A conforming Objective-C WebSocket client library.
Objective-C
9,534
star
2

katran

A high performance layer 4 load balancer
C
4,674
star
3

AITemplate

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.
Python
4,515
star
4

velox

A C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.
C++
3,474
star
5

cinder

Cinder is Meta's internal performance-oriented production version of CPython.
Python
3,458
star
6

FBX2glTF

A command-line tool for the conversion of 3D model assets on the FBX file format to the glTF file format.
C++
2,060
star
7

spectrum

A client-side image transcoding library.
C++
1,987
star
8

oomd

A userspace out-of-memory killer
C++
1,795
star
9

xar

executable archive format
Python
1,571
star
10

Bowler

Safe code refactoring for modern Python.
Python
1,532
star
11

submitit

Python 3.8+ toolbox for submitting jobs to Slurm
Python
1,245
star
12

gloo

Collective communications library with various primitives for multi-machine training.
C++
1,181
star
13

fizz

C++14 implementation of the TLS-1.3 standard
C++
1,128
star
14

dhcplb

dhcplb is Facebook's implementation of a load balancer for DHCP.
Go
1,046
star
15

below

A time traveling resource monitor for modern Linux systems
Rust
1,029
star
16

OnlineSchemaChange

A tool for performing online schema changes on MySQL.
Python
965
star
17

Glean

System for collecting, deriving and working with facts about source code.
Hack
923
star
18

Battery-Metrics

Library that helps in instrumenting battery related system metrics.
Java
736
star
19

retrie

Retrie is a powerful, easy-to-use codemodding tool for Haskell.
Haskell
500
star
20

superconsole

The superconsole crate provides a handler and building blocks for powerful, yet minimally intrusive TUIs. It is cross platform, supporting Windows 7+, Linux, and MacOS. Rustaceans who want to create non-interactive TUIs can use the component composition building block system to quickly deploy their code.
Rust
477
star
21

nvdtools

A set of tools to work with the feeds (vulnerabilities, CPE dictionary etc.) distributed by National Vulnerability Database (NVD)
Go
446
star
22

nimble

New file format for storage of large columnar datasets.
C++
419
star
23

infima

A UI framework that provides websites with the minimal CSS and JS needed to get started with building a modern responsive beautiful website
HTML
405
star
24

CG-SQL

CG/SQL is a compiler that converts a SQL Stored Procedure like language into C for SQLite. SQLite has no stored procedures of its own. CG/CQL can also generate other useful artifacts for testing and schema maintenance.
HTML
391
star
25

TTPForge

The TTPForge is a Cybersecurity Framework for developing, automating, and executing attacker Tactics, Techniques, and Procedures (TTPs).
Go
320
star
26

flowtorch

This library would form a permanent home for reusable components for deep probabilistic programming. The library would form and harness a community of users and contributors by focusing initially on complete infra and documentation for how to use and create components.
Jupyter Notebook
300
star
27

ptr

Python Test Runner.
Python
284
star
28

senpai

Senpai is an automated memory sizing tool for container applications.
Python
270
star
29

fbjni

A library designed to simplify the usage of the Java Native Interface
C++
260
star
30

dynolog

Dynolog is a telemetry daemon for performance monitoring and tracing. It exports metrics from different components in the system like the linux kernel, CPU, disks, Intel PT, GPUs etc. Dynolog also integrates with pytorch and can trigger traces for distributed training applications.
C++
251
star
31

gazebo

A Rust library containing a collection of small well-tested primitives.
Rust
235
star
32

reindeer

Reindeer is a tool to transform Rust Cargo dependencies into generated Buck build rules
Rust
177
star
33

dispenso

The project provides high-performance concurrency, enabling highly parallel computation.
C++
174
star
34

GeoLift

GeoLift is an end-to-end geo-experimental methodology based on Synthetic Control Methods used to measure the true incremental effect (Lift) of ad campaign.
R
171
star
35

oculus-linux-kernel

The Linux kernel code for Oculus devices
C
156
star
36

FCR

FBNet-Command-Runner: A thrift service to run commands on heterogeneous Network devices with configurable parameters.
154
star
37

dataclassgenerate

DataClassGenerate (or simply DCG) is a Kotlin compiler plugin that addresses an Android APK size overhead from Kotlin data classes.
Kotlin
154
star
38

hsthrift

The Haskell Thrift Compiler. This is an implementation of the Thrift spec that generates code in Haskell. It depends on the fbthrift project for the implementation of the underlying transport.
Haskell
148
star
39

FioSynth

Tool which enables the creation of synthetic storage workloads, automates the execution and results collection of synthetic storage benchmarks.
Python
140
star
40

meta-code-verify

Code Verify is an open source web browser extension that confirms that your Facebook, Messenger, Instagram, and WhatsApp Web code hasn’t been tampered with or altered, and that the Web experience you’re getting is the same as everyone else’s.
TypeScript
137
star
41

tacquito

Tacquito is an open source TACACs+ server written in Go that implements RFC8907
Go
93
star
42

go-qfext

a fast counting quotient filter implementation in golang
Go
91
star
43

momentum

A library for human kinematic motion and numerical optimization solvers to apply human motion
C++
89
star
44

ForgeArmory

ForgeArmory provides TTPs that can be used with the TTPForge (https://github.com/facebookincubator/ttpforge).
Swift
80
star
45

antlir

ANoTher Linux Image buildeR
Rust
76
star
46

sks

Secure Key Storage (SKS) is a library for Go that abstracts Security Hardware on laptops.
Go
72
star
47

dcrpm

A tool to detect and correct common issues around RPM database corruption.
Python
72
star
48

ConversionsAPI-Tag-for-GoogleTagManager

This repository will contain the artifacts needed for setting up Conversions API implementation on Google Tag Manager's serverside. Please follow the instructions https://www.facebook.com/business/help/702509907046774
Smarty
64
star
49

InjKit

Injection Kit. It is a java bytecode processing library for bytecode injection and transformation.
Java
55
star
50

obs-plugins

OBS Plugins
C++
54
star
51

glTFVariantMeld

An application that accepts files on the glTF format, interprets them as variants of an over-arching whole, and melds them together.
Rust
51
star
52

haberdashery

A collection of high-performance crypto implementations.
Rust
44
star
53

later

A framework for python asyncio with batteries included for people writing services in python asyncio
Python
39
star
54

go2chef

A Golang tool to bootstrap a system from zero so that it's able to run Chef to be managed
Go
39
star
55

CommutingZones

Commuting zones are geographic areas where people live and work and are useful for understanding local economies, as well as how they differ from traditional boundaries. These zones are a set of boundary shapes built using aggregated estimates of home and work locations. Data used to build commuting zones is aggregated and de-identified.
JavaScript
39
star
56

ConversionsAPI-Client-for-GoogleTagManager

This repository will contain the artifacts needed for setting up Conversions API implementation on Google Tag Manager's serverside. Primarily we will be hosting, - ConversionsAPI(Facebook) Client - listens on the events fired to GTM Server and maps them to common GTM schema. - ConversionsAPI(Facebook) Tag - server tag that fires events to CAPI.For more details on Design here https//fburl.com/uae68vlr
37
star
57

Facebook-Pixel-for-Wordpress

A plugin for advertisers who use Wordpress to enable them easily setup the facebook pixel.
JavaScript
37
star
58

strobelight

Meta's fleetwide profiler framework
C++
28
star
59

buck2-change-detector

Given a Buck2 built project and a set of changes (e.g. from source control) compute the targets that may have changed. Sometimes known as a target determinator, useful for optimizing a CI system.
Rust
28
star
60

wordpress-messenger-customer-chat-plugin

Messenger Customer Chat Plugin for WordPress
PHP
27
star
61

CP4M

CP4M is a conversational marketing platform which enables advertisers to integrate their customer-facing chatbots with FB Messenger/WhatsApp, in order to meet customers where they are and drive native conversations on the advertiser's owned infra.
Java
27
star
62

rush

RUSH (Reliable - unreliable - Streaming Protocol)
C++
26
star
63

MY_ENUM

Small c++ macro library to add compile-time introspection to c++ enum classes.
C++
17
star
64

SafeC

Library containing safer alternatives/wrappers for insecure C APIs.
C++
16
star
65

go-belt

It is an implementation-agnostic Go(lang) package to generalize observability tooling (logger, metrics, tracer and so on) and provide ability to use any of these tools with a standard context. Essentially it is an attempt to standardize observability API in Go.
Go
16
star
66

spark-ar-core-libs

Core libraries that can be used in Spark AR. You can import each library depends on your requirements.
TypeScript
15
star
67

scrut

Scrut is a testing toolkit for CLI applications. A tool to scrutinize terminal programs without fuss.
Rust
15
star
68

sado

A macOS signed-app shim for running daemons with reliable capabilities.
Swift
12
star
69

Portal-Kernel

Kernel Code for Portal.
C
11
star
70

npe-toolkit

Libraries, guides, blueprints, and sample code, to enable rapidly building 0-1 applications on iOS, Android and web.
TypeScript
10
star
71

Eigen-FBPlugins

This is collection of plugins extending Eigen arrays/matrices with main focus on using them for computer vision. In particular, this project should provide support for multichannel arrays (missing in vanilla Eigen) and seamless integration between Eigen types and OpenCV functions.
C++
9
star
72

isometric_pattern_matcher

A new isometric calibration pattern - which should/might lead to higher accuracy calibrations compared to existing solutions (checkerboards, patterns of circles).
C++
8
star
73

dnf-plugin-cow

Code to enable Copy on Write features being upstreamed in rpm and librepo
Shell
8
star
74

jupyterhub_fb_authenticator

JupyterHub Facebook Authenticator is a Facebook OAuth authenticator built on top of OAuthenticator.
Python
8
star
75

wireguard_py

Cython library for Wireguard
C
7
star
76

kernel-patches-daemon

Sync Patchwork series's with Github pull requests
Python
6
star
77

meta-fbvuln

OpenEmbedded meta-layer that allows producing a vulnerability manifest alongside a Yocto build. The produced manifest is suitable for ongoing vulnerability scanning of fielded software.
5
star
78

gazebo_lint

A Rust linter that provides various suggestions based on the new primitives offered in the `gazebo` library.
Rust
4
star
79

language-capirca

Adds syntax highlighting for Capirca filetypes in Atom. Capirca is an open source standard for writing vendor-neutral firewall policies as originally released by Google: https://github.com/google/capirca
3
star
80

cinderx

cinderx
C++
3
star
81

fbc_owrt_feed

Facebook Connectivity OpenWrt Feed. Package feed for OpenWrt router OS by Facebook Connectivity programme.
Lua
2
star
82

cutlass-fork

A Meta fork of NV CUTLASS repo.
C++
2
star
83

hwbits_lib

Abstraction of hardware register-level protocols in a python semantic names.
Python
1
star