• Stars
    star
    656
  • Rank 68,243 (Top 2 %)
  • Language
    Shell
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated 9 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Library for bash utility methods and tools

bash-lib

Introductory blog post: https://www.conjur.org/blog/stop-bashing-bash/

                   _______________  _______________
                 .'               .'               .|
               .'               .'               .' |
             .'_______________.'______________ .'   |
             | ___ _____ ___ || ___ _____ ___ |     |
             ||_=_|__=__|_=_||||_=_|__=__|_=_||     |
       ______||_____===_____||||_____===_____||     | __________
    .'       ||_____===_____||||_____===_____||    .'          .'|
  .'         ||_____===_____||||_____===_____||  .'          .'  |
.'___________|_______________||_______________|.'__________.'    |
|.----------.|.-----___-----.||.-----___-----.||    |_____.----------.
|]          |||_____________||||_____________|||  .'      [          |
||          ||.-----___-----.||.-----___-----.||.'        |          |
||          |||_____________||||_____________|||==========|          |
||          ||.-----___-----.||.-----___-----.||    |_____|          |
|]         o|||_____________||||_____________|||  .'      [        'o|
||          ||.-----___-----.||.-----___-----.||.'        |          |
||          |||             ||||_____________|||==========|          |
||          |||             |||.-----___-----.||    |_____|          |
|]          |||             ||||             |||  .'      [          |
||__________|||_____________||||_____________|||.'________|__________|
''----------'''------------------------------'''----------''
            (o)LGB                           (o)

The place to store functions that are used in pipelines for multiple repos.

Please add whatever is useful to you, but keep it tidy so its still useful to everyone else :)

Release Status: Alpha

TL;DR: Ready for use, but needs expansion.

The functions in this repo are tested and ready for use, but certain libs are pretty much place holders (eg logging). Those need further contributions before they provide a comprehensive solution.

License: Apache 2.0

See the license file

Usage

Add bash-lib into your project in the way that best fits your workflow. The only requirement is that you pin the version of bash-lib that you use. This is important so that changes to bash-lib do not have the power to break all projects that use bash-lib. Your project can then test updates to bash-lib and roll forward periodically.

Options:

  • Add a submodule: they are an easy way to integrate bash-lib and automatically use a single SHA until manually updated. Submodules add a pointer from a mount point in your repo to the external repo (bash-lib), and require workflow changes to ensure that pointer is derferenced during clone, checkout and some other opertaions.
  • Add a subtree: This repo uses subtrees to pull in test dependencies. Subtrees copy an external repo into a subdirectory of the host repo, no workflow changes are required. Subtrees naturally keep a single version of bash-lib until explicitly updated. Note that subtree merge commits do not rebase well ⚠️, so best to keep subtree updates in separate PRs from normal commits.
  • Clone bash-lib in your deployment process, bash-lib doesn't have to be within your repo, just needs to be somewhere where your scripts can source init. This is where it's most important that you implement a mechanism to always use the same SHA, as a clone will track master by default, which is not an allowed use of bash-lib.

Once you have bash-lib cloned in your project, you source two things:

  1. Source bash-lib/init. This ensures submodules are initalised and sets the BASH_LIB_DIR env var to the absolute path to the bash-lib dir. This makes it easy to source libraries from other scripts.
  2. Source ${BASH_LIB_DIR}/lib-name/lib for any libraries you are interested in.

You are now ready to use bash-lib functions :)

Structure

The /init script sets up everything required to use the library, most importantly the BASH_LIB_DIR variable which gives the absolute path to the root of the library and should be used for sourcing the modules.

The repo is organized into libraries, each library is a directory that has a lib file. Sourcing the lib for a library should expose all the functions that library offers. The lib file may source or reference other supporting files within it's directory.

.
β”œβ”€β”€ libname
β”‚   β”œβ”€β”€ lib
β”‚   └── supporting-file
β”œβ”€β”€ init # init script, source this first
β”œβ”€β”€ run-tests # top level test script, executes all tests
β”œβ”€β”€ secrets.yml # secrets required for executing tests
β”œβ”€β”€ test-utils
β”‚   β”œβ”€β”€ bats # git subtree
β”‚   β”œβ”€β”€ bats-assert-1 # git subtree
β”‚   β”œβ”€β”€ bats-support # git subtree
β”‚   β”œβ”€β”€ lib
β”‚   └── tap2junit
└── tests-for-this-repo
    β”œβ”€β”€ filehandling.bats
    β”œβ”€β”€ fixtures #
    β”‚   └── libname # Dir containing test fixtures for a library
    β”œβ”€β”€ tap2junit
    β”œβ”€β”€ libname.bats # contains tests for libname/lib
    β”œβ”€β”€ python-lint # supporting files for python lint
    β”œβ”€β”€ run-bats-tests # script to run bats tests
    β”œβ”€β”€ run-gitleaks # script to check for leaked secrets
    └── run-python-lint # script to run python lint

Contents

Library Description Functions
filehandling Functions relating to file and path handling
  1. bl_abs_path: Ensure a path is absolute
git Git helpers
  1. bl_git_available: True if git binary or function is available
  2. bl_in_git_repo: True if current directory is a git working directory
  3. bl_github_owner_repo: returns $owner/$repo extracted from the url of the origin remote
  4. bl_repo_root: Find the root of the current git repo.
  5. bl_all_files_in_repo: List files tracked by git, excludes submodules.
  6. bl_remote_latest_tag: Returns the symbolic name of the latest tag from a remote.
  7. bl_remote_latest_tagged_commit: Returns the SHA of the most recently tagged commit in a remote repo (tag^{}).
  8. bl_remote_sha_for_ref: Returns the SHA for a given ref from a named remote.
  9. bl_remote_tag_for_sha: Returns the tag corresponding to a SHA from a named remote - if there is one.
  10. bl_tracked_files_excluding_subtrees: List files tracked by git, but excluding any files that are in paths listed in .gittrees.
  11. bl_gittrees_present: Succeeds if .gittrees is present in the root of the repo, otherwise fails.
  12. bl_cat_gittrees: Returns the contents of .gittrees from the top level of the repo, excluding any comments. Fails if .gittrees is not present.
github Github Related Functions
  1. bl_hub_available: True if hub binary or function is available
  2. bl_hub_creds_available: True if hub creds are available (file or env vars)
  3. bl_hub_check: Preflight check for hub, true if git installed, in git repo, hub installed and hub creds are available
  4. bl_hub_download_latest: Download latest hub binary from github and install to ~/bin or specified path
  5. bl_hub_issue_number_for_title: Find the issue number for an issue from its title, searches open issues in the current repo. (current repo = workding directory, repo is found by origin remote). If multiple issues match the supplied title string, only the first is returned.
  6. bl_hub_add_issue_comment: Add a comment to an issue
  7. bl_hub_comment_or_create_issue: Create issue if an issue matching the title doesn't exist. If a match is found, add a comment to it
helpers Bash scripting helpers
  1. bl_die: print message and exit 1
  2. bl_fail: print message and return 1
  3. bl_spushd/bl_spopd: Safe verisons of pushd & popd that call die if the push/pop fails, they also drop stdout.
  4. bl_is_num: Check if a value is a number via regex
  5. bl_retry: Retry a command until it succeeds up to a user specified maximum number of attempts. Escalating delay between attempts.
  6. bl_retry_constant: Retry a command until it succeeds with a constant delay between attempts
k8s Utils for connecting to K8s
  1. bl_build_gke_image: Build docker image for running kubectl commands against GKE.
  2. bl_delete_gke_image: Delete image from GKE.
  3. bl_run_docker_gke_command: Run command in gke-utils container, already authenticated to k8s cluster.
logging Helpers related to logging.
  1. bl_announce: Echo message in ascii banner to distinguish it from other log messages.
  2. bl_log: Log a message at the specified level. Default log level is info, change level by setting environment variable BASH_LIB_LOG_LEVEL
  3. bl_check_log_level: Check if a value is a valid bash lib log level
  4. bl_debug: Log a message at debug level
  5. bl_info: Log a message at info level
  6. bl_warning: Log a message at warning level
  7. bl_error: Log a message at error level
  8. bl_fatal: Log a message at fatal level
Ruby Helpers related to ruby infrastructure
  1. bl_gem_latest_version: Return the latest version of a gem from rubygems.org
  2. bl_jq_available: Check jq binary is available
  3. bl_curl_available: Check curl binary is available
test-utils Helpers for executing tests
  1. bl_shellcheck_script: Execute shellcheck against a script, uses docker.
  2. bl_find_scripts: Find git tracked files with extension.
  3. bl_tap2junit: Convert a subset of TAP to JUnit XML. Retains logs for errors.
  4. bl_validate_changelog: Check CHANGELOG.md (or a specified file) complies with keepachangelog.com format.

Contributing

For further information on contributing, style & testing, please see CONTRIBUTING.md

Maintainers

More Repositories

1

KubiScan

A tool to scan Kubernetes cluster for risky permissions
Python
1,310
star
2

SkyArk

SkyArk helps to discover, assess and secure the most privileged entities in Azure and AWS
PowerShell
862
star
3

ACLight

A script for advanced discovery of Privileged Accounts - includes Shadow Admins
PowerShell
783
star
4

conjur

CyberArk Conjur automatically secures secrets used by privileged users and machine identities
Ruby
750
star
5

kubeletctl

A client for kubelet
Go
730
star
6

summon

CLI that provides on-demand secrets access for common DevOps tools
Go
687
star
7

PipeViewer

A tool that shows detailed information about named pipes in Windows
C#
551
star
8

DLLSpy

DLL Hijacking Detection Tool
C++
481
star
9

zBang

zBang is a risk assessment tool that detects potential privileged account threats
C#
332
star
10

shimit

A tool that implements the Golden SAML attack
Python
331
star
11

RPCMon

RPC Monitor tool based on Event Tracing for Windows
C#
317
star
12

Evasor

A tool to be used in post exploitation phase for blue and red teams to bypass APPLICATIONCONTROL policies
C#
309
star
13

BlobHunter

Find exposed data in Azure with this public blob scanner
Python
306
star
14

RiskySPN

Detect and abuse risky SPNs
PowerShell
258
star
15

secretless-broker

Secure your apps by making them Secretless
Go
234
star
16

White-Phoenix

A tool to recover content from files encrypted with intermittent encryption
Python
218
star
17

kubernetes-rbac-audit

Tool for auditing RBACs in Kubernetes
Python
210
star
18

MITM_Intercept

A little bit less hackish way to intercept and modify non-HTTP protocols through Burp & others.
Python
201
star
19

epv-api-scripts

These API scripts enable CyberArk users to automate privileged account management task like account creation, user management, and more.
PowerShell
197
star
20

ketshash

A little tool for detecting suspicious privileged NTLM connections, in particular Pass-The-Hash attack, based on event viewer logs.
PowerShell
168
star
21

rdpfuzz

Tools for fuzzing RDP
C
122
star
22

SkyWrapper

SkyWrapper helps to discover suspicious creation forms and uses of temporary tokens in AWS
Python
104
star
23

EasyPeasy

Find accounts using common and default passwords in Active Directory.
PowerShell
65
star
24

pas-on-cloud

CyberArk Privileged Access Security on Cloud
Python
60
star
25

ansible-security-automation-collection

CyberArk Ansible Security Automation Collection
Python
59
star
26

CYBRHardeningCheck

A utility to check CyberArk component servers hardening status
PowerShell
53
star
27

summon-aws-secrets

Summon provider for AWS Secrets Manager
Go
52
star
28

cyberark-aws-auto-onboarding

Solutions for automatically detecting, managing and securing privileged accounts in AWS EC2
Python
40
star
29

pas-orchestrator

CyberArk Privileged Access Security automatic deployment using Ansible
Python
38
star
30

summon-conjur

CyberArk Conjur provider for Summon
Go
38
star
31

ansible-modules

Ansible Modules for CyberArk Privileged Account Security Web Service SDK
Python
37
star
32

conjur-quickstart

Start securing your secrets and infrastructure by installing Conjur, using Docker and the official Conjur containers on DockerHub.
Shell
33
star
33

malware-research

C++
33
star
34

Fuzzer-V

C
30
star
35

sidecar-injector

Sidecar Injector for the Conjur Kubernetes Authenticator and Secretless
Go
29
star
36

ChattyCaty

JavaScript
28
star
37

conjur-oss-helm-chart

Helm chart for deploying Conjur OSS to Kubernetes
Shell
27
star
38

secrets-provider-for-k8s

Cyberark secrets provider for k8s
Go
26
star
39

PwnKit-Hunter

PwnKit-Hunter is here to help you check if your systems are vulnerable to CVE-2021-4043, a.k.a. PwnKit
Python
25
star
40

PreCog

Discover "HotSpots" - potential spots for credentials theft
PowerShell
23
star
41

terraform-provider-conjur

Terraform provider for Conjur
Shell
21
star
42

pvwa

Ansible role to deploy Cyberark Password Vault Web Access
Jinja
20
star
43

conjur-api-go

Go client for the CyberArk Conjur API
Go
20
star
44

slosilo

A Ruby interface to standard cryptographic primitives
Ruby
17
star
45

password-lookup-plugin

cyberarkpassword Lookup Plugin
17
star
46

cyberark-conjur-cli

CyberArk Conjur command line interface written in Python
Python
17
star
47

psm

Ansible role to deploy Cyberark Privileged Session Manager
PowerShell
16
star
48

conjur-template

Template repo for Conjur repositories
16
star
49

conjur-api-java

Java client for the CyberArk Conjur API
Java
15
star
50

summon-keyring

Cross-platform provider for Summon that talks to keyrings.
Python
15
star
51

cpm

Ansible role to deploy Cyberark Central Policy Manager
PowerShell
15
star
52

conjur-api-dotnet

.NET client for the CyberArk Conjur API
C#
15
star
53

cyberark-conjur-cli-docker-based

CyberArk Conjur command line interface (Ruby)
Ruby
15
star
54

parse-a-changelog

A validator for changelogs using the Keep a Changelog standard (http://keepachangelog.com)
Ruby
15
star
55

ansible-aim-provider

Ansible Galaxy Role to install and uninstall Cyberark AIM provider
14
star
56

kubernetes-conjur-deploy

Scripts for deploying DAP followers to Kubernetes and OpenShift given an existing DAP master cluster
Shell
14
star
57

Symda

Python
13
star
58

summon-s3

AWS S3 provider for Summon
Shell
13
star
59

KDSnap

C++
12
star
60

ansible-role-conjur

Grants Conjur machine identity to hosts
Python
12
star
61

conjur-credentials-plugin

Conjur plugin for securely providing credentials to Jenkins jobs
Java
12
star
62

conjur-openapi-spec

OpenAPI v3 specification for Conjur / DAP v10+
Mustache
11
star
63

conjur-authn-k8s-client

Authentication sidecar for Conjur Kubernetes integration.
Shell
11
star
64

dev-flow

Opinionated CLI that standardizes and automates common development tasks
Go
11
star
65

ark-sdk-python

CyberArk's Official SDK and CLI - https://cyberark.github.io/ark-sdk-python/
Python
10
star
66

ansible-conjur-host-identity

This project encapsulates the functionality of our `cyberark.conjur-host-identity role for Ansible
Shell
8
star
67

conjur-api-python

Python client for the CyberArk Conjur API
Python
8
star
68

conjur-oss-suite-release

Under development - Latest stable releases of the Conjur OSS suite
Go
7
star
69

community

Information for the CyberArk contributor community
7
star
70

cyberark-aim-chef

Chef custom resource for CyberArk AIM
Ruby
7
star
71

homebrew-tools

Homebrew formulas for different CyberArk tooling.
Shell
7
star
72

cacookiecleaner

C#
7
star
73

conjur-policy-generator

Tools to create sample Conjur policies for testing, etc.
Ruby
7
star
74

atyourservice

The atyourservice project is intended to provide highly-customizable utilities for troubleshooting issues.
Python
7
star
75

identity-aws-verified-permissions-demo

Python
7
star
76

summon-chefapi

Summon provider for Chef encrypted data bags
Go
6
star
77

conjur-puppet

Official Puppet module for CyberArk Conjur
Ruby
6
star
78

sample-siem-dashboards

6
star
79

escape-the-cloud

Web Application for CyberArk Cloud Escape Room CTF challenge
JavaScript
5
star
80

helm-charts

CyberArk Helm charts repository.
Shell
5
star
81

psmp-deploy-ansible-role

Ansible role to deploy Cyberark Privileged Session Manager SSH Proxy (PSM-SSH)
Shell
5
star
82

ansible-conjur-collection

Ansible Collection for Conjur
Python
5
star
83

conjur-spring-boot-sdk

Java
5
star
84

SafeNet

5
star
85

conjur-azure-devops-extension

Azure DevOps Extension for retrieving secrets from CyberArk Conjur
JavaScript
5
star
86

conjur-api-ruby

Ruby client for the CyberArk Conjur API
Ruby
4
star
87

conjur-base-image

Base Docker images for CyberArk Conjur
Shell
4
star
88

conjur-cli-go

CyberArk Conjur command line interface (Go)
Go
4
star
89

conjur-aws

[DEPRECATED] - AWS CloudFormation templates for Conjur
Shell
4
star
90

conjur-google-cloud-marketplace

[DEPRECATED] Conjur application for Google Cloud Marketplace
Shell
4
star
91

conjur-service-broker

Implementation of the Open Service Broker API for Conjur
Ruby
4
star
92

identity-demo-android

This is to demonstrate CyberArk Identity capabilities of Android SDK in a sample app.
Kotlin
3
star
93

pas-reporter-dataprocessing

C#
3
star
94

conjur-inspect

Go
3
star
95

dap-web-utility

DAP web utility to simplify DAP/Conjur deployment & operations
JavaScript
3
star
96

urbancode-conjur-aim

A plugin which allows UrbanCode Deploy to get credentials from EPV via AIM, and to get secrets from Conjur for setting up a CI/CD workflow
Groovy
3
star
97

psmp-activate-ansible-role

Ansible role to activate Cyberark Privileged Session Manager SSH Proxy (PSM-SSH)
Shell
3
star
98

conjur-tutorials

A repository for tutorials related to Conjur
Shell
2
star
99

conjur-authn-iam-client-python

Python client for using Conjur with authn-iam
Python
2
star
100

aim-puppet

Ruby
2
star