• Stars
    star
    186
  • Rank 199,901 (Top 5 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created about 5 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Ops - cli wrapper for Terraform, Ansible, Helmfile and SSH for cloud automation

Ops CLI

Build status Docker image License

From version 2.0 onward, ops-cli requires Python3.
If you're still using Python2, use ops-cli version <2.0

ops-cli is a python wrapper for Terraform, Ansible and SSH for cloud automation.

We use multiple tools to manage our infrastructure at Adobe. The purpose of ops-cli is to gather the common cluster configurations in a single place and, based on these, interact with the above mentioned tools. In this way, we can avoid duplication and can quickly spin up new clusters (either production or development ones). All we need to do is customize the cluster configuration file (example here).

ops-cli integrates with the Azure and AWS cli, in order to provide inventory, ssh, sync, tunnel and the possibility to run ansible playbooks on a fleet of EC2 instances. It can be used to add a layer of templating (using jinja2) on top of Terraform files. This is useful for removing duplicated code when it comes to spinning up infrastructure across multiple environments (stage/sandbox/prod) and across teams. Useful for both AWS and Kubernetes deployments.

Table of Contents

How it works?

You define a cluster configuration, using a yaml file. The yaml file contains different kind of sections, one for each plugin. For instance, you could have a section for Terraform files, a section for AWS instructions, Kubernetes Helm charts and so forth.

Use cases

Manage AWS EC2 instances

Once you define your cluster configuration, you can run ops commands such as seeing the instance inventory.

# fetch instances from AWS and prints them
ops clusters/mycluster.yaml inventory --limit webapp 

This would output something like: ops

Then you can run ssh, play, run, sync etc.

# SSH to one of the nodes (can handle bastion as well)
ops clusters/mycluster.yaml ssh webapp-01

# run a deployment playbook via ansible
ops clusters/mycluster.yaml play ansible/playbooks/task/webapp/deployment.yaml -- -e version=5.36.2 -u ec2-user --limit webapp

# run command on all selected nodes
ops clusters/mycluster.yaml run "sudo yum upgrade myawesomeapp; sudo service myawesomeapp restart" -- -u ec2-user --limit '"aam_app_group=canary;az=us-east-1a"'

# copy file to all servers
ops clusters/mycluster.yaml sync /tmp/myfile webapp: -l ec2-user

# create a tunnel
ops clusters/stage.yaml ssh --tunnel --local 8080 --remote 8080 stage-thanos-1 -l ec2-user

See examples/features/inventory

Terraform

# Performs jinja templating (if any) and runs terraform plan
ops clusters/mycluster.yaml terraform --path-name aws-eks plan

# Run terraform apply, with the possibility to sync the tf state files remotely (currently, AWS S3 bucket is supported + DynamoDB for locking). 
ops clusters/mycluster.yaml terraform --path-name aws-eks apply

ops-terraform

Run terraform by using hierarchical configs

See examples/features/terraform-hierarchical

Create Kubernetes cluster (using AWS EKS)

See examples/aws-kubernetes

Installing

Local

Virtualenv

Here is a link about how to install and use virtualenv: https://virtualenv.pypa.io/en/stable/

Ops tool installation

Python 3

# Make sure pip is up to date
curl https://bootstrap.pypa.io/get-pip.py | python3

# Install virtualenv
pip install --upgrade virtualenv
pip install --upgrade virtualenvwrapper

echo 'export WORKON_HOME=$HOME/.virtualenvs' >> ~/.bash_profile
echo 'source /usr/local/bin/virtualenvwrapper.sh' >> ~/.bash_profile
source ~/.bash_profile

# create virtualenv
mkvirtualenv ops
workon ops

# uninstall previous `ops` version (if you have it)
pip uninstall ops --yes

# install ops-cli v2.2.0 stable release
pip install --upgrade ops-cli

Terraform

Optionally, install terraform to be able to access terraform plugin. See https://www.terraform.io/intro/getting-started/install.html Also for pretty formatting of terraform plan output you can install https://github.com/coinbase/terraform-landscape (use gem install for MacOS)

Using docker image

You can try out ops-cli, by using docker. The docker image has all required prerequisites (python, terraform, helm, git, ops-cli etc).

To start out a container, running the latest ops-cli docker image run:

docker run -it ghcr.io/adobe/ops-cli:2.2.0 bash

After the container has started, you can start using ops-cli:

ops help
# usage: ops [-h] [--root-dir ROOT_DIR] [--verbose] [-e EXTRA_VARS]
#           cluster_config_path
#           {inventory,terraform,packer,ssh,play,run,sync,noop} ...

git clone https://github.com/adobe/ops-cli.git
cd ops-cli
ls examples
# aws-kubernetes
# cassandra-stress
# features

cd examples/aws-kubernetes
ops clusters/my-kubernetes-cluster.yaml terraform --path-name aws-eks plan
# in order to setup aws-kubernetes follow the steps from https://github.com/adobe/ops-cli/blob/master/examples/aws-kubernetes/README.md

Configuring

AWS

If you plan to use ops with AWS, you must configure credentials for each account

$ aws configure --profile aws_account_name

Azure

TBD

Examples

See examples/ folder:

  • cassandra-stress - n-node cassandra cluster used for stress-testing; a basic stress profile is included
  • spin up a Kubernetes cluster
  • distinct ops features

Usage help

To see all commands and a short description run ops --help

usage: ops [-h] [--root-dir ROOT_DIR] [--verbose] [-e EXTRA_VARS]
           cluster_config_path
           {inventory,terraform,packer,ssh,play,run,sync,noop} ...

Run commands against a cluster definition

positional arguments:
  cluster_config_path   The cluster config path cluster.yaml
  {inventory,terraform,packer,ssh,play,run,sync,noop}
    inventory           Show current inventory data
    terraform           Wrap common terraform tasks with full templated
                        configuration support
    packer              Wrap common packer tasks and inject variables from a
                        cluster file
    ssh                 SSH or create an SSH tunnel to a server in the cluster
    play                Run an Ansible playbook
    run                 Runs a command against hosts in the cluster
    sync                Sync files from/to a cluster
    noop                used to initialize the full container for api usage

optional arguments:
  -h, --help            show this help message and exit
  --root-dir ROOT_DIR   The root of the resource tree - it can be an absolute
                        path or relative to the current dir
  --verbose, -v         Get more verbose output from commands
  -e EXTRA_VARS, --extra-vars EXTRA_VARS
                        Extra variables to use. Eg: -e ssh_user=ssh_user

More help

Each sub-command includes additional help information that you can get by running: ops examples/inventory/aam.yaml sync --help

Tool configuration: .opsconfig.yaml

Some tool settings are available via a .opsconfig.yaml configuration file. The file is looked-up in /etc/opswrapper/.opsconfig.yaml, then in ~/.opsconfig.yaml and then in the project folder starting from the current dir and up to the root dir. All the files found this way are merged together so that you can set some global defaults, then project defaults in the root dir of the project and overwrite them for individual envs. Eg: ~/.opsconfig.yaml, /project/.opsconfig.yaml, /project/clusters/dev/.opsconfig.yaml

Inventory

The inventory command will list all the servers in a given cluster and cache the results for further operations on them (for instance, SSHing to a given node or running an ansible playbook).

You can always filter which nodes you want to display or use to run an ansible playbook on, by using the --limit argument (eg. --limit webapp). The extra filter is applied on the instance tags, which includes the instance name.

The way inventory works is by doing a describe command in AWS/Azure. The describe command matches all the nodes that have the tag "cluster" equal to the cluster name you have defined.

In order to configure it, you need to add the inventory section in your cluster configuration file (example here).

AWS example

---
inventory:
  - plugin: cns
    args:
      clusters:
        - region: us-east-1
          boto_profile: aam-npe # make sure you have this profile in your ~/.aws/credentials file
          names: [mycluster1] # this assumes the EC2 nodes have the Tag Name "cluster" with Value "mycluster1"

Azure example

---
inventory:
  - plugin: azr
    args:
      tags: environment=prod
      locations: westeurope,northeurope

Inventory usage

usage: ops cluster_config_path inventory [-h] [-e EXTRA_VARS]
                                         [--refresh-cache] [--limit LIMIT]
                                         [--facts]

optional arguments:
  -h, --help            show this help message and exit
  -e EXTRA_VARS, --extra-vars EXTRA_VARS
                        Extra variables to use. Eg: -e ssh_user=ssh_user
  --refresh-cache       Refresh the cache for the inventory
  --limit LIMIT         Limit run to a specific server subgroup. Eg: --limit
                        newton-dcs
  --facts               Show inventory facts for the given hosts

Terraform

usage: ops cluster_config_path terraform [-h] [--var VAR] [--module MODULE]
                                         [--resource RESOURCE] [--name NAME]
                                         [--plan]
                                         subcommand

positional arguments:
  subcommand           apply | console | destroy | import | output | plan |
                       refresh | show | taint | template | untaint

optional arguments:
  -h, --help            show this help message and exit
  --var VAR             the output var to show
  --module MODULE       for use with "taint", "untaint" and "import". The
                        module to use. e.g.: vpc
  --resource RESOURCE   for use with "taint", "untaint" and "import". The
                        resource to target. e.g.: aws_instance.nat
  --name NAME           for use with "import". The name or ID of the imported
                        resource. e.g.: i-abcd1234
  --plan                for use with "show", show the plan instead of the
                        statefile
  --skip-refresh        for use with "plan". Skip refresh of statefile
  --raw-output          for use with "plan". Show raw plan output without piping through terraform landscape (if terraform landscape is not enabled in opsconfig.yaml this will have no impact)
  --path-name PATH_NAME in case multiple terraform paths are defined, this
                        allows to specify which one to use when running
                        terraform

    Examples:
        # Create a new cluster with Terraform
        ops clusters/qe1.yaml terraform plan
        ops clusters/qe1.yaml terraform apply

        # Update an existing cluster
        ops clusters/qe1.yaml terraform plan
        ops clusters/qe1.yaml terraform apply

        # Get rid of a cluster and all of its components
        ops clusters/qe1.yaml terraform destroy

        # Retrieve all output from a previously created Terraform cluster
        ops clusters/qe1.yaml terraform output

        # Retrieve a specific output from a previously created Terraform cluster
        ops clusters/qe1.yaml terraform output --var nat_public_ip

        # Refresh a statefile (no longer part of plan)
        ops clusters/qe1.yaml terraform refresh

        # Taint a resource- forces a destroy, then recreate on next plan/apply
        ops clusters/qe1.yaml terraform taint --module vpc --resource aws_instance.nat

        # Untaint a resource
        ops clusters/qe1.yaml terraform untaint --module vpc --resource aws_instance.nat

        # Show the statefile in human-readable form
        ops clusters/qe1.yaml terraform show

        # Show the plan in human-readable form
        ops clusters/qe1.yaml terraform show --plan

        # View parsed jinja on the terminal
        ops clusters/qe1.yaml terraform template

        # Import an unmanaged existing resource to a statefile
        ops clusters/qe1.yaml terraform import --module vpc --resource aws_instance.nat --name i-abcd1234

        # Use the Terraform Console on a cluster
        ops clusters/qe1.yaml terraform console

        # Validate the syntax of Terraform files
        ops clusters/qe1.yaml terraform validate

        # Specify which terraform path to use
        ops clusters/qe1.yaml terraform plan --path-name terraformFolder1

Terraform landscape

For pretty formatting of terraform plan output you can install https://github.com/coinbase/terraform-landscape (use gem install for MacOS). To make ops use it you need to add terraform.landscape: True in opsconfig.yaml file.

SSH

usage: ops cluster_config_path ssh [-h] [-e EXTRA_VARS] [-l USER]
                                   [--ssh-config SSH_CONFIG] [--index INDEX]
                                   [--tunnel] [--ipaddress] [--local LOCAL]
                                   [--remote REMOTE] [--proxy] [--nossh]
                                   role [ssh_opts [ssh_opts ...]]

positional arguments:
  role                  Server role to ssh to. Eg: dcs
  ssh_opts              Manual ssh options

optional arguments:
  -h, --help            show this help message and exit
  -e EXTRA_VARS, --extra-vars EXTRA_VARS
                        Extra variables to use. Eg: -e ssh_user=ssh_user
  -l USER, --user USER  SSH User
  --ssh-config SSH_CONFIG
                        Ssh config file name in the ./ansible dir
  --index INDEX         Index of the server from the group
  --tunnel              Use SSH tunnel, must pass --local and --remote
  --ipaddress
  --local LOCAL         local port for ssh proxy or ssh tunnel
  --remote REMOTE       remote port for ssh tunnel
  --proxy               Use SSH proxy, must pass --local
  --nossh               Port tunnel a machine that does not have SSH. Implies
                        --ipaddress, and --tunnel; requires --local and
                        --remote
  --keygen              Create a ssh keys pair to use with this infrastructure
  --noscb               Disable use of Shell Control Box (SCB) even it is
                        enabled in the cluster config
  --auto_scb_port       When using Shell Control Box (SCB) and creating a
                        proxy,a random port is generated, which will be used
                        in the ssh config for all playbook, run and sync
                        operations

    Examples:
        # SSH using current username as remote username
        ops clusters/qe1.yaml ssh nagios

        # SSH using a different username
        ops clusters/qe1.yaml ssh nagios -l ec2-user

        # SSH to the second nagios instance
        ops clusters/qe1.yaml ssh nagios --index 2

        # SSH to a specific hostname, instead of the tagged role
        ops clusters/qe1.yaml ssh full-hostname-here-1

        # Create an SSH tunnel to Nagios forwarding the remote port 80 to local port 8080
        ops clusters/qe1.yaml ssh --tunnel --remote 80 --local 8080 nagios

        # Create an SSH tunnel to a host where the service is NOT listening on `localhost`
        ops clusters/qe1.yaml ssh --tunnel --remote 80 --local 8080 nagios --ipaddress

        # Create an SSH tunnel to a host with an open port which does NOT have SSH itself (Windows)
        # Note that the connection will be made from the Bastion host
        ops clusters/qe1.yaml ssh --tunnel --local 3389 --remote 3389 --nossh windowshost

        # Create a proxy to a remote server that listens on a local port
        ops clusters/qe1.yaml ssh --proxy --local 8080 bastion

        # In case Shell Control Box (SCB) is configured and enabled on the cluster a proxy which
        # will be used by all ops play, run and sync operations, can be created either using
        # either the port configured the cluster config file or an auto generated port.
        # In this case --local param must not be used
        # Example for using the port configured in the cluster config
        ops clusters/qe1.yaml ssh bastion --proxy
        # Example for using the auto generated port
        ops clusters/qe1.yaml ssh bastion --proxy --auto_scb_port


        # Disable use of Shell Control Box (SCB) even it is enabled in the cluster config
        ops clusters/qe1.yaml ssh bastion --noscb

SSHPass

In case you want to use the OSX Keychain to store your password and reuse across multiple nodes (e.g. running a playbook on 300 nodes and not having to enter the password for every node) follow the tutorial below:

  1. Open Keychain Access app on OSX

  2. Create a new keychain (File -> New Keychain), let's say aam

  3. Select the aam keychain and add a new password entry in this (File -> New Password Item): - Name: idm - Kind: application password - Account: your_ldap_account (e.g. johnsmith) - Where: idm

  4. Create $HOME/bin dir - this is where the scripts below are saved

  5. Create ~/bin/askpass script and update the ldap account there:

cat > ~/bin/askpass  <<"EOF"
#!/usr/bin/env bash
/usr/bin/security find-generic-password -a <your_ldap_account> -s idm -w $HOME/Library/Keychains/aam.keychain
EOF
chmod +x ~/bin/askpass
  1. Checkout notty github repo, build and move the binary to $HOME/bin/

  2. Create ~/bin/sshpass script:

cat > $HOME/bin/sshpass <<"EOF"
#!/usr/bin/env bash
export DISPLAY=:99
export SSH_ASKPASS="$HOME/bin/askpass"
[[ $1 == -d* ]] && shift
$HOME/bin/notty $@
EOF

chmod +x $HOME/bin/sshpass
  1. Verify the setup works:
# Connect to bastion
~/bin/sshpass ssh -o StrictHostKeyChecking=no -l <your_ldap_account> <52.5.5.5>
  1. Run ops tool

SCB

Shell Control Box (SCB) is an activity monitoring appliance from Balabit (now One Identity) that controls privileged access to remote servers. ops has support for using SCB as ssh proxy for the following operations: ssh, tunnel, proxy, ansible play, run and sync

In order to use SCB an extra section needs to be added to the cluster config file:

scb:
  enabled: true
  host: "scb.example.com"
  proxy_port: 2222 # optional

Having this config all ssh operations will be done via the scb host, unless the --noscb flag is used.

When using SCB, SSHPass will not be used.

For ansible play, run and sync operations to work via SCB a proxy needs to be created first and then run ops in a different terminal window or tab:

# 1. Create a proxy in a terminal window
# Example for using the port configured in the cluster config
ops clusters/qe1.yaml ssh bastion --proxy
# Example for using the auto generated port
ops clusters/qe1.yaml ssh bastion --proxy --auto_scb_port

# 2. Run the play/run/sync command normally in a different terminal window or tab
# A message will indicate the scb proxy is used
ops clusters/qe1.yaml play ansible/plays/cluster/configure.yaml
...
Connecting via scb proxy at 127.0.0.1:2222.
This proxy should have already been started and running in a different terminal window.
If there are connection issues double check that the proxy is running.
...

Play

Run an ansible playbook.

usage: ops cluster_config_path play [-h] [-e EXTRA_VARS] [--ask-sudo-pass]
                                    [--limit LIMIT] [--noscb]
                                    playbook_path
                                    [ansible_args [ansible_args ...]]

positional arguments:
  playbook_path         The playbook path
  ansible_args          Extra ansible args

optional arguments:
  -h, --help            show this help message and exit
  -e EXTRA_VARS, --extra-vars EXTRA_VARS
                        Extra variables to use. Eg: -e ssh_user=ssh_user
  --ask-sudo-pass       Ask sudo pass for commands that need sudo
  --limit LIMIT         Limit run to a specific server subgroup. Eg: --limit
                        newton-dcs
  --noscb               Disable use of Shell Control Box (SCB) even if it is
                        enabled in the cluster config

    Examples:
        # Run an ansible playbook
        ops clusters/qe1.yaml play ansible/plays/cluster/configure.yaml

        # Limit the run of a playbook to a subgroup
        ops clusters/qe1.yaml play ansible/plays/cluster/configure.yaml -- --limit dcs

        # Overwrite or set a variable
        ops clusters/qe1.yaml play ansible/plays/cluster/configure.yaml -- -e city=paris

        # Filter with tags
        ops clusters/qe1.yaml play ansible/plays/cluster/configure.yaml -- -t common

        # Run a playbook and overwrite the default user
        ops clusters/qe1.yaml play ansible/plays/cluster/configure.yaml -- -u ec2-user

Run command

Run a bash command on the selected nodes.

usage: ops cluster_config_path run [-h] [--ask-sudo-pass] [--limit LIMIT]
                                   [--noscb]
                                   host_pattern shell_command
                                   [extra_args [extra_args ...]]

positional arguments:
  host_pattern     Limit the run to the following hosts
  shell_command    Shell command you want to run
  extra_args       Extra ansible arguments

optional arguments:
  -h, --help       show this help message and exit
  --ask-sudo-pass  Ask sudo pass for commands that need sudo
  --limit LIMIT    Limit run to a specific server subgroup. Eg: --limit
                   newton-dcs
  --noscb          Disable use of Shell Control Box (SCB) even if it is
                   enabled in the cluster config

    Examples:
        # Last 5 installed packages on each host
        ops qe1.yaml run all 'sudo grep Installed /var/log/yum.log | tail -5'

        # See nodetool status on each cassandra node
        ops qe1.yaml run qe1-cassandra 'nodetool status'

        # Complex limits
        ops qe1.yaml run 'qe1-cassandra,!qe1-cassandra-0' 'nodetool status'

        # Show how to pass other args

Sync files

Performs rsync to/from a given set of nodes.

usage: ops cluster_config_path sync [-h] [-l USER] [--noscb]
                                    src dest [opts [opts ...]]

positional arguments:
  src                   Source dir
  dest                  Dest dir
  opts                  Rsync opts

optional arguments:
  -h, --help            show this help message and exit
  -l USER, --user USER  Value for remote user that will be used for ssh
  --noscb               Disable use of Shell Control Box (SCB) even if it is
                        enabled in the cluster config

        rsync wrapper for ops inventory conventions

        Example:

        # rsync from remote dcs role
        ops cluster.yml sync 'dcs[0]:/usr/local/demdex/conf' /tmp/configurator-data --user remote_user

        # extra rsync options
        ops cluster.yml sync 'dcs[0]:/usr/local/demdex/conf' /tmp/configurator-data -l remote_user -- --progress

Noop

usage: ops cluster_config_path noop [-h]

optional arguments:
  -h, --help  show this help message and exit

Packer

Runs packer, for creating images.

usage: ops cluster_config_path packer [-h] subcommand

positional arguments:
  subcommand  build | validate

optional arguments:
  -h, --help  show this help message and exit

    Examples:
        # Validate a packer file
        ops clusters/centos7.yaml packer validate

        # Build a packer file
        ops clusters/centos7.yaml packer build

Secrets Management

There are cases where you need to reference sensitive data in your cluster.yaml file (credentials, passwords, tokens etc). Given that the cluster configuration file can be stored in a version control system (such as Git), the best practice is to not put sensitive data in the file itself. Instead, we can use ops-cli to fetch the desired credentials from a secrets manager such as Vault or Amazon SSM, at runtime.

Vault

Ops can manage the automatic generation of secrets and their push in Vault, without actually persisting the secrets in the cluster file. A cluster file will only need to use a construct like the following:

db_password: "{{'secret/campaign/generated_password'|managed_vault_secret(policy=128)}}"

Which will translate behind the scenes in :

  • look up in vault the secrets at secret/campaign/generated_password in the default key 'value' (Adobe convention that can be overridden with the key parameter)
  • if the value there is missing, generate a new secret using the engine passgen with a policy of length 128 characters
  • return the generated value
  • if the value at that path already exist, just return that value. This allows us to just refer in cluster files a secret that actually exists in vault and make sure we only generate it once - if it was already created by os or any other system, we will just use what is already there. The reference is by means of fixed form jinja call added to the cluster file, which ends up interpreted later during the templating phase.

Amazon Secrets Manager (SSM)

Amazon offers the possibility to use their Secrets Manager in order to manage configuration data such as credentials, passwords and license keys.

We can use ops-cli to fetch the sensitive data from SSM, at runtime. Just define this in your cluster configuration file (eg. mycluster.yaml).

db_password: "{{ '/my/ssm/path' | read_ssm(aws_profile='myprofile') }}"

ops-cli will read the SSM value by running a command similar to: AWS_PROFILE=aam-npe aws ssm get-parameter --name "/my/ssm/path" --region us-east-1 --with-decryption. Note that you can specify the AWS region via read_ssm(aws_profile='myprofile', region_name='us-west-2').

Using jinja2 filters in playbooks and terraform templates

You can register your own jinja2 filters that you can use in the cluster config file, terraform templates and ansible playbooks

All ops commands look for filters in the following locations:

Example simple filter:

# plugins/filter_plugin/myfilters.py

def my_filter(string):
    return 'filtered: ' + string


class FilterModule(object):
    def filters(self):
        return {
            'my_filter': my_filter
        }

# usage in playbook, templates, cluster config
# test_custom_filters: "{{ 'value' | my_filter }}"

SKMS

Create a file in ~/.skms/credentials.yaml which looks like the following:

endpoint: "api.skms.mycompany.com"
username: <username>
password: <password>

Development

Install ops in development mode

git clone https://github.com/adobe/ops-cli.git
cd ops
# Install openssl
brew install openssl libyaml
env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" python setup.py develop

Running tests

  • on your machine: py.test tests

Troubleshooting

  • Permission issues when installing: you should install the tool in a python virtualenv

  • Exception when running: ops pkg_resources._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'!= 2.4'"

    Caused by a broken paramiko version, reinstall paramiko: pip2 uninstall paramiko; pip2 install paramiko

  • Exception when installing ops because the cryptography package fails to install:

Either install the tool in a virtualenv or:

    brew install libffi
    brew link libffi --force
    brew install openssl  
    brew link openssl --force

License

Apache License 2.0

More Repositories

1

brackets

An open source code editor for the web, written in JavaScript, HTML and CSS.
JavaScript
33,300
star
2

react-spectrum

A collection of libraries and tools that help you build adaptive, accessible, and robust user experiences.
TypeScript
11,538
star
3

leonardo

Generate colors based on a desired contrast ratio
JavaScript
1,855
star
4

antialiased-cnns

pip install antialiased-cnns to improve stability and accuracy
Python
1,611
star
5

balance-text

A plugin for implementing balancing of wrapping text in a web page
JavaScript
1,362
star
6

adobe.github.com

Adobe central hub for open source
CSS
1,290
star
7

spectrum-web-components

Spectrum Web Components
TypeScript
1,177
star
8

brackets-shell

CEF3-based application shell for Brackets.
Python
1,176
star
9

spectrum-css

The standard CSS implementation of the Spectrum design language.
CSS
1,154
star
10

aem-core-wcm-components

Standardized components to build websites with AEM.
Java
709
star
11

S3Mock

A simple mock implementation of the AWS S3 API startable as Docker image, TestContainer, JUnit 4 rule, JUnit Jupiter extension or TestNG listener
Java
699
star
12

jsonschema2md

Convert Complex JSON Schemas into Markdown Documentation
JavaScript
563
star
13

NLP-Cube

Natural Language Processing Pipeline - Sentence Splitting, Tokenization, Lemmatization, Part-of-speech Tagging and Dependency Parsing
HTML
549
star
14

aem-project-archetype

Maven template to create best-practice websites on AEM.
JavaScript
519
star
15

ferrum

Features from the rust language in javascript: Provides Traits/Type classes & a hashing infrastructure and an advanced library for working with sequences/iterators in js
JavaScript
496
star
16

brackets-app

Deprecated CEF1-based app shell for Brackets. Use https://github.com/adobe/brackets-shell instead.
C++
490
star
17

cryptr

Cryptr: a GUI for Hashicorp's Vault
HTML
487
star
18

cssfilterlab

CSS FilterLab
JavaScript
348
star
19

hyde

A front-end to Jekyll that parses C++ sources to produce and enforce out-of-line documentation
C++
303
star
20

node-smb-server

A 100% JavaScript implementation of the SMB file sharing protocol.
JavaScript
276
star
21

htl-spec

HTML Template Language Specification
275
star
22

aem-guides-wknd

Tutorial Code companion for Getting Started Developing with AEM Sites WKND Tutorial
JavaScript
261
star
23

lit-mobx

Mixin and base class for using mobx with lit-element
TypeScript
260
star
24

xdm

Experience Data Model
JavaScript
235
star
25

lagrange

A Robust Geometry Processing Library
C++
215
star
26

webkit

Experiments and contributions to WebKit. Tracks git://git.webkit.org/WebKit.git
213
star
27

chromium

Experiments and contributions to Chromium project
C++
207
star
28

elixir-styler

An @elixir-lang code-style enforcer that will just FIFY instead of complaining
Elixir
207
star
29

avmplus

Source code for the Actionscript virtual machine
ActionScript
194
star
30

pdf-embed-api-samples

Samples for Adobe Document Services PDF Embed API
JavaScript
155
star
31

Deep-Audio-Prior

Audio Source Separation Without Any Training Data.
Python
154
star
32

rules_gitops

This repository contains rules for continuous, GitOps driven Kubernetes deployments.
Starlark
151
star
33

aem-htl-repl

Read–Eval–Print Loop environment for HTL.
JavaScript
151
star
34

OSAS

One Stop Anomaly Shop: Anomaly detection using two-phase approach: (a) pre-labeling using statistics, Natural Language Processing and static rules; (b) anomaly scoring using supervised and unsupervised machine learning.
Python
150
star
35

stringlifier

Stringlifier is on Opensource ML Library for detecting random strings in raw text. It can be used in sanitising logs, detecting accidentally exposed credentials and as a pre-processing step in unsupervised ML-based analysis of application text data.
Python
148
star
36

svg-native-viewer

SVG Native viewer is a library that parses and renders SVG Native documents
C++
142
star
37

Spry

Spry is a JavaScript-based framework that enables the rapid development of Ajax-powered web pages.
HTML
140
star
38

XMP-Toolkit-SDK

The XMP Toolkit allows you to integrate XMP functionality into your product or solution
C++
135
star
39

brackets-phonegap

A brackets extension for PhoneGap development.
JavaScript
112
star
40

brackets.io

brackets.io website
HTML
111
star
41

tf-manage

Shell
110
star
42

aem-component-generator

AEM Component Generator is a java project that enables developers to generate the base structure of an AEM component using a JSON configuration file specifying component and dialog properties and other configuration options.
Java
109
star
43

himl

A hierarchical yaml config in Python
Python
107
star
44

adobe-client-data-layer

An event-driven store for all trackable data of your site.
JavaScript
107
star
45

GLS3D

An implementation of OpenGL for Stage3D that can run inside Flash Player 11+
C
105
star
46

coral-spectrum

A JavaScript library of Web Components following Spectrum design patterns.
JavaScript
104
star
47

aem-core-cif-components

A set of configurations and components to get you started with AEM Commerce development
Java
101
star
48

aem-boilerplate

Use this repository template for new AEM projects.
JavaScript
99
star
49

react-webcomponent

This projects automates the wrapping of a React component in a CustomElement.
JavaScript
95
star
50

web-platform

JavaScript
90
star
51

ride

REST API Automation framework for functional, integration, fuzzing, and performance testing
Java
90
star
52

alloy

Alloy is the web SDK for the Adobe Experience Platform.
JavaScript
85
star
53

go-starter

Bootstrap a new project from a template.
Go
83
star
54

asset-share-commons

A modern, open-source asset share reference implementation built on Adobe Experience Manager (AEM)
Java
83
star
55

orc

ORC is a tool for finding violations of C++'s One Definition Rule on the OSX toolchain.
C++
79
star
56

experience-platform-postman-samples

77
star
57

pdfservices-node-sdk-samples

Samples for the Adobe Document Services PDF Tools Node SDK
HTML
77
star
58

sbmc

Sample-based Monte Carlo Denoising using a Kernel-Splatting Network [Siggraph 2019]
Python
76
star
59

git-server

A GitHub Protocol & API emulation
JavaScript
75
star
60

spectrum-tokens

Tokens used by Spectrum, Adobe's design system.
JavaScript
74
star
61

aio-theme

The Adobe I/O theme for building markdown powered sites
JavaScript
70
star
62

aem-sample-we-retail-journal

We.Retail Journal is a sample showcasing SPA Editing capabilities in AEM using React and Angular
CSS
69
star
63

aem-guides-wknd-spa

69
star
64

frontend-regression-validator

Visual regression tool used to compare baseline and updated instances of a website in a deployment pipeline.
Python
67
star
65

blackhole

An HTTP sink (for testing) with optional recording and playback ability
Go
65
star
66

aem-spa-project-archetype

Maven Archetype for creating new AEM SPA projects
CSS
63
star
67

aio-cli

Adobe I/O Extensible CLI
JavaScript
60
star
68

aem-upload

Makes uploading to AEM easier, and can be used as a command line executable or required as a Node.js module.
JavaScript
59
star
69

aem-modernize-tools

A suite of tools to modernize your AEM Sites implementations off legacy features.
Java
58
star
70

dds2atf

Tool for converting DDS files into ATF files suitable for use with the Flash Stage3D API
C++
58
star
71

redux-saga-promise

Create actions that return promises, which are resolved/rejected by a redux saga
JavaScript
58
star
72

aem-react-editable-components

SPA React Editable Components for Adobe Experience Manager
TypeScript
55
star
73

xmp-docs

XMP documentation
52
star
74

adobe-photoshop-api-sdk

Adobe Photoshop API SDK
JavaScript
50
star
75

aem-enablement

Content required for AEM Enablement
Java
50
star
76

brackets-edge-web-fonts

Edge Web Fonts extension for Brackets. Simply unzip and drop into your Brackets extension folder to browse and include Edge Web Fonts.
JavaScript
50
star
77

aem-brackets-extension

Brackets extension for Adobe Experience Manager (AEM) front-end developers with auto-sync and HTL support.
JavaScript
50
star
78

helix-home

The home of Project Helix
HTML
49
star
79

aem-testing-clients

Testing tools for Adobe Experience Manager
Java
49
star
80

aem-guides-wknd-graphql

JavaScript
47
star
81

brackets-registry

A registry system for hosting Brackets extensions powered by node.js
JavaScript
46
star
82

helix-cli

Command-line tools for developing with AEM
JavaScript
46
star
83

htlengine

An HTL (Sightly) Interpreter/Compiler for Node.js
HTML
45
star
84

aem-dispatcher-experiments

Experiments to demonstrate the impact of the Dispatcher and it's configuration parameters.
HTML
44
star
85

pdfservices-python-sdk-samples

Adobe PDFServices python SDK Samples
Python
44
star
86

node-fetch-retry

Node Module for performing retries using node-fetch
JavaScript
42
star
87

commerce-cif-connector

AEM Commerce connector for Magento and GraphQL
Java
42
star
88

aem-react-core-wcm-components

41
star
89

behavior_tree_editor

A visual editor for building behavior trees for the bots
JavaScript
41
star
90

libLOL

Python
40
star
91

starter-repo

Documentation templates for use in open source and open development projects
40
star
92

commerce-cif-magento

Adobe Commerce Integration Framework (CIF) Magento Integration
JavaScript
40
star
93

bin2c

Convert to/Embed binary files in C source files, quickly and efficiently.
C
38
star
94

graphicalweb-keynote

Keynote for Graphical Web Conference
JavaScript
37
star
95

aem-site-template-standard

Basic site template for AEM that allows non-Java experts to create new sites by customizing CSS and JS only.
SCSS
37
star
96

aio-cli-plugin-cloudmanager

Cloud Manager plugin for the Adobe I/O CLI
JavaScript
37
star
97

oss-contributors

How do tech companies rank amongst themselves when it comes to github.com activity?
JavaScript
35
star
98

aem-eclipse-developer-tools

The Eclipse plugin that brings you the full connection to the Adobe Experience Manager, with auto-sync and project creation wizard.
Java
35
star
99

fetch

Simplified HTTP/1(.1) and HTTP/2 requests with Server Push Support
JavaScript
34
star
100

PDFServices.NET.SDK.Samples

This .NET sample solution helps you get started with the Adobe PDF Services SDK.
HTML
33
star