• This repository has been archived on 18/Dec/2019
  • Stars
    star
    115
  • Rank 305,916 (Top 7 %)
  • Language
    Ruby
  • License
    Other
  • Created over 12 years ago
  • Updated over 7 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Etsy's foodcritic rules

#foodcritic-rules

These are the foodcritic rules we've written here at Etsy to enforce rules on our cookbooks. Some of these rules are best practices we've identified for our particular use case, and some were put in place as remediation items after issues we experienced.

Usage

Once you've cloned the repo from github, you can run foodcritic using the following options to test your cookbooks against these rules:

foodcritic -t etsy,FC023 -I <rules.rb> cookbooks

You should make sure that <rules.rb> is replaced with the location of the rules.rb file from this repository.

Note that we're also checking against the built in foodcritic rule FC023 (Prefer conditional attributes). This is to complement the functionality in rule ETSY004 (see description below)

All of the below rules have the "style" tag as they are best practices and recommendations from our experience rather than rules to detect things which might break your cookbooks.

Rules

ETSY001 - Package or yum_package resource used with :upgrade action on non-whitelisted package

We've had several instances of new packages being put into our yum repository and rolled out to servers before we intended due to the use of the :upgrade action in "package" and "yum_package" resources. This rule will detect when :upgrade has been used instead of :install. An optional whitelist is included at the top of the file to allow you to relax this rule manually for specific packages.

The whitelisted package list is defined at the top of rules.rb, and defaults to empty:

@pkgupgrade_whitelist = []

For example, this block would trip this rule:

package "sendmail" do
  action :upgrade
end

ETSY002 - Execute resource used to run git commands

This rule detects when Git commands are run as bash commands inside of an "execute" block, rather than using the "git" resource. We alert on this behaviour as running these commands inside "execute" blocks gives you much less visibility of the commands' output and status.

For example, this block would trip this rule:

execute "checkout" do
  command "git clone git://github.com/foo/bar.git"
  cwd "/home/me"
end

ETSY003 - Execute resource used to run curl or wget commands

This rule detects when wget or curl commands are run as bash commands inside of an "execute" block, rather than using the "remote_file" resource. We alert on this behaviour as running these commands inside "execute" blocks gives you much less visibility of the commands' output and status.

For example, this block would trip this rule:

execute "fetch_file" do
    command "wget -O /tmp/foo.tar.gz http://me.mycorp.com/files/foo.tar.gz"
    not_if "test -f /tmp/foo.tar.gz"
end

ETSY004 - Execute resource defined without conditional or action :nothing

This rule detects when an "execute" block is defined without specifying a conditional (ie not_if or only_if) or action :nothing. This is to avoid the presence of resources which run on every chef run, whether they need to or not. Please note, this rule does not identify "execute" blocks which are themselves wrapped in a conditional, ie an if statement. For this reason, we recommend that you also use the built in foodcritic rule FC023, which detects this case. Rule FC023 recommends that you always use conditionals inside your resources instead of wrapping them, and we'd concur with this.

For example, this block would trip this rule:

execute "fetch_file" do
    command "wget -O /tmp/foo.tar.gz http://me.mycorp.com/files/foo.tar.gz"
end

As would this block, even though there is technically a conditional wrapping the resource.

if !File.exists?("/tmp/foo.tar.gz")
    execute "fetch_file" do
        command "wget -O /tmp/foo.tar.gz http://me.mycorp.com/files/foo.tar.gz"
    end
end

However, the above block would subsequently be flagged by rule FC023.

ETSY005 - Action :restart sent to a core service

This rule identifies when a recipe might send "action :restart" to a service we've identified as a "core" service. We've had instances in the past of chef changes causing hard restarts of services such as memcached or httpd, which can have unexpected side effects. For this reason, we alert if any restart notifications are sent to a defined list of "core" services. Restart notifications sent to any services not in this list will not cause this rule to trip.

The core services list is defined at the top of rules.rb:

@coreservices = ["httpd", "mysql", "memcached", "postgresql-server"]

For example, this block would trip this rule:

cookbook_file "/etc/httpd/conf.d/myvhost.conf" do
  source "myvhost.conf"
  mode 0644
  owner "root"
  group "root"
  notifies :restart, resources(:service => "httpd")
end

Wheras this block would not (as we're using reload instead of restart):

cookbook_file "/etc/httpd/conf.d/myvhost.conf" do
  source "myvhost.conf"
  mode 0644
  owner "root"
  group "root"
  notifies :reload, resources(:service => "httpd")
end

And neither would this one (as sendmail is not a core service):

template "/etc/mail/sendmail.mc" do
  source "sendmail.mc.erb"
  owner "root"
  group "root"
  mode 00644
  notifies :restart, resources(:service => "sendmail")
end

ETSY006 - Execute resource used to run chef-provided command

This rule detects when an "execute" block is using a shell command when Chef already provides a built in resource with equivalent functionality.

The list of commands to check for is defined at the top of rules.rb:

@corecommands = ["yum -y", "yum install", "yum reinstall", "yum remove", "mkdir", "useradd", "usermod", "touch"]

For example, this block would trip this rule:

execute "reinstall_pear" do
  command "yum -y reinstall php-pear"
end

ETSY007 - Package or yum_package resource used to install core package without specific version number

This rule identifies when a recipe is installing a package for something we've identified as a "core" service without specifying a specific version number to install. We've had instances in the past of divergence occurring when action :install is used by itself, and a new package has been added to the repository resulting in new servers installing the new version while old servers are still using the previous version.

The list of package names to check for is defined at the top of rules.rb:

@coreservicepackages = ["httpd", "Percona-Server-server-51", "memcached", "postgresql-server"]

For example, this block would trip this rule:

package "httpd" do
  action :install
end

More Repositories

1

AndroidStaggeredGrid

An Android staggered grid view which supports multiple columns with rows of varying sizes.
Java
4,756
star
2

skyline

It'll detect your anomalies! Part of the Kale stack.
Python
2,135
star
3

logster

Parse log files, generate metrics for Graphite and Ganglia
Python
1,968
star
4

deployinator

Deployinate!
Ruby
1,878
star
5

morgue

post mortem tracker
PHP
1,017
star
6

411

An Alert Management Web Application
PHP
971
star
7

feature

Etsy's Feature flagging API used for operational rampups and A/B testing.
PHP
869
star
8

MIDAS

Mac Intrusion Detection Analysis System
833
star
9

opsweekly

On call alert classification and reporting
JavaScript
761
star
10

oculus

The metric correlation component of Etsy's Kale system
Java
707
star
11

mctop

a top like tool for inspecting memcache key values in realtime
Ruby
507
star
12

supergrep

realtime log streamer
JavaScript
411
star
13

Conjecture

Scalable Machine Learning in Scalding
Java
361
star
14

statsd-jvm-profiler

Simple JVM Profiler Using StatsD and Other Metrics Backends
Java
330
star
15

nagios-herald

Add context to Nagios alerts
Ruby
322
star
16

dashboard

JavaScript
308
star
17

boundary-layer

Builds Airflow DAGs from configuration files. Powers all DAGs on the Etsy Data Platform
Python
262
star
18

Testing101

Etsy's educational materials on testing and design
PHP
262
star
19

DebriefingFacilitationGuide

Leading Groups at Etsy to Learn From Accidents
247
star
20

phpunit-extensions

Etsy PHPUnit Extensions
PHP
228
star
21

nagios_tools

Tools for use with Nagios
Python
173
star
22

open-api

We are working on a new version of Etsy’s Open API and want feedback from developers like you.
166
star
23

TryLib

TryLib is a simple php library that helps you generate a diff of your working copy and send it to Jenkins to run the test suite(s) on the latest code patched with your changes.
PHP
155
star
24

BugHunt-iOS

Objective-C
148
star
25

mod_realdoc

Apache module to support atomic deploys - http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/
C
128
star
26

ab

Etsy's little framework for A/B testing, feature ramp up, and more.
128
star
27

wpt-script

Scripts to generate WebPagetest tests and download results
PHP
121
star
28

applepay-php

A PHP extension that verifies and decrypts Apple Pay payment tokens
C
118
star
29

kevin-middleware

This is an Express middleware that makes developing javascript in a monorepo easier.
JavaScript
110
star
30

mixer

a tool to initiate meetings by randomly pairing individuals
Go
100
star
31

cloud-jewels

Estimate energy consumption using GCP Billing Data
TSQL
96
star
32

jenkins-master-project

Jenkins Plugin: Master Project. Jenkins project type that allows for selection of sub-jobs to execute, watch, and report worst status of all sub-projects.
Java
83
star
33

Sahale

A Cascading Workflow Visualizer
JavaScript
83
star
34

PushBot

An IRC Bot for organizing code pushes
Java
79
star
35

cdncontrol

CLI tool for working with multiple CDNs
Ruby
79
star
36

rules_grafana

Bazel rules for building Grafana dashboards
Starlark
70
star
37

chef-whitelist

Simple library to enable host based rollouts of changes
Ruby
68
star
38

rfid-checkout

Low Frequency RFID check out/in client for Raspberry Pi
Python
64
star
39

Etsy-Engineering-Career-Ladder

Etsy's Engineering Career Ladder
HTML
61
star
40

Evokit

Rust
60
star
41

ELK-utils

Utilities for working with the ELK (Elasticsearch, Logstash, Kibana) stack
Ruby
59
star
42

incpath

PHP extension to support atomic deploys
C
52
star
43

arbiter

A utility for generating Oozie workflows from a YAML definition
Java
48
star
44

VIPERBuilder

Scaffolding for building apps in a clean way with VIPER architecture
Swift
41
star
45

chef-handlers

Chef handlers we use at Etsy
Ruby
40
star
46

sbt-checkstyle-plugin

SBT Plugin for Running Checkstyle on Java Sources
Scala
32
star
47

es-restlog

Plugin for logging Elasticsearch REST requests
Java
29
star
48

yubigpgkeyer

Script to make RSA authentication key generation on Yubikeys differently painful
Python
28
star
49

Apotheosis

Python
28
star
50

jenkins-deployinator

Jenkins Plugin: Deployinator. Links key deployinator information to Jenkins builds via the CLI.
Java
25
star
51

sbt-compile-quick-plugin

SBT Plugin for Compiling a Single File
Scala
25
star
52

geonames

Scripts for using Geonames
PHP
24
star
53

jading

cascading.jruby build and execution tool
16
star
54

etsy.github.com

Etsy! on Github!
HTML
16
star
55

divertsy-client

The Android client for running DIVERTsy, a waste stream recording tool to help track diversion rates.
Java
13
star
56

cdncontrol_ui

A web UI for Etsy's cdncontrol tool
CSS
13
star
57

terraform-demux

A user-friendly launcher (à la bazelisk) for Terraform.
Go
12
star
58

logstash-plugins

Ruby
11
star
59

jenkins-triggering-user

Jenkins Plugin: Triggering User. Populates a $TRIGGERING_USER environment variable from the build cause and other sources, a best guess.
10
star
60

EtsyCompositionalLayoutBridge

iOS framework that allows for simultaneously leveraging flow layout and compositional layout in collection views
Swift
3
star
61

consulkit

Ruby API for interacting with HashiCorp's Consul.
Ruby
1
star
62

soft-circuits-workshop

Etsy Soft Circuits Workshop
Arduino
1
star