[arrow down]

[arrow up]

Where:

• -q stands for "quiet";

• -v for "verbose"

If you prefer to be able to examine the script before executing code piped straight off the Internet, I don’t blame you. You are cautious and smart.

For folks like you, here is a slightly more secure way of doing the same thing:

This method allows you to examine the /tmp/install script before running it.

Below are some of the explanations

7.2.1. Installing a Particular Version or a Branch

7.2.2. Customizing the Installer Script

bash -c "$(curl -fsSL https://bashmatic.re1.re); \
    bashmatic-install -d -v -f -b v2.4.1 -H ~/workspace/bashmatic"

When you run bash -c "$(curl -fsSL https://bashmatic.re1.re); bashmatic-install", the following typically happens:

• curl downloads the bin/bashmatic-install script and passes it to the built-in BASH for evaluation.

• Once evaluated, function bashmatic-install is invoked, which actually performs the installation.

  • This is the function that accepts the above listed arguments.

• The script may ask for your password to enable sudo access - this may be required on OS-X to install XCode Developer tools (which include git)

• If your version of BASH is 3 or older, the script will download and build from sources version 5+ of BASH, and install it into /usr/local/bin/bash. SUDO may be required for this step.

• On OS-X the script will install Homebrew on OS-X, if not already there.

  • Once Brew is installed, brew packages coreutils and gnu-sed are installed, as both are required and are relied upon by Bashmatic.

• The script will then attempt to git clone the bashmatic repo into the Bashmatic home folder, or - if it already exists - it will git pull latest changes.

• Finally, unless you specify -l or --skip-on-login the script will check your bash dot files, and will add the hook to load Bashmatic from either ~/.bashrc or ~/.bash_profile.

The last part my require some explanation.

7.3.1. To load Bashmatic at Login, or Not?

# Let's see if ~/.bashrc mentions Bashmatic:
$ grep bashmatic ~/.bashrc
[[ -f ~/.bashmatic/init.sh ]] && source ~/.bashmatic/init.sh

❯ time source init.sh

real  0m0.134s
user  0m0.078s
sys	  0m0.074s

bash -c "$(curl -fsSL https://bashmatic.re1.re); bashmatic-install -l"

# ~/.bashrc
export BASHMATIC_HOME="${HOME}/.bashmatic"
export PATH="${BASHMATIC_HOME}/bin:${PATH}"

# Eg, if as in the previous example you sourced in Bashmatic:
$ bashmatic.version
2.1.2

# If you have not, you can still invoke 'bashmatic.version':
$ bashmatic version

# Or another function, 'array.join' — if you sourced in init.sh:
$ array.join '|' hello goodbye
hello|goodbye

# Or using the script:
$ bashmatic array.join '|' hello goodbye
hello|goodbye

Therefore for situawtion where curl may not be available, offer the following shell function that works on Linux/Ubuntu and OS-X-based systems. It can be easily extended with new operating systems:

7.4.1. Discovering Available Functions

# List all functions using 4-column mode; print top 5 lines.
❯ bashmatic functions 4 | head -5
7z.a         db.psql.connect.db-set hl.yellow-on-gray  run.inspect-variables
7z.install   db.psql.connect.db-set hr                 run.inspect-variables-
7z.unzip     db.psql.connect.just-d hr.colored         run.inspect.set-skip-f
7z.x         db.psql.connect.table- http.servers       run.on-error.ask-is-en
7z.zip       db.psql.connect.table- https.servers      run.print-command

# or, to get the count of all functions, use 1 column output:
$ bashmatic functions 1 | wc -l
773

To install Bashmatic manually, follow these steps (feel free to change BASHMATIC_HOME if you like):

Sometimes you may not be able to use git (I have seen issues ranging from local certificate mismatch to old versions of git, and more), but maybe able to download with curl. In that case, you can lookup the latest tag (substitute "v1.6.0" below with that tag), and then issue this command:

You can always reload Bashmatic® with bashmatic.reload function. This simply performs the sourcing of ${BASHMATIC_HOME}/init.sh.

When you install Bashmatic it automatically adds a hook to your ~/.bash_profile, but if you are on ZSH you may need to add it manually (for now).

Add the following to your ~/.zshrc file:

Makefile is provided as a convenience for running most common tasks and to simplify running some more complex tasks that require remembering many arguments, such as make setup. You might want to use the Makefile for several reasons:

1. make open-readme

   This tasks opens the PDF version of the README in your PDF system viewer.

2. make install

   This allows you to install the Bashmatic Framework locally. It simply runs bin/bashmatic-install script. At most this will add hooks to your shell init files so that Bashmatic is loaded at login.

3. make setup

   This task invokes the bin/dev-setup script under the hood, so that you can setup your local computer developer setup for software development.

   Now, this script offers a very rich CLI interface, so you can either run the script directly and have a fine-grained control over what it’s doing, or you can run it with default flags via this make target.

   This particular make target runs bin/dev-setup script with the following actions:

   dev, cpp, fonts, gnu, go, java, js, load-balancing, postgres, ruby

4. make test and make test-parallel are both meant for Bashmatic Developers and contributors. Please see the Contributing section on how to run and what to expect from the UNIT tests.

5. make update is the task that should be run by library contributors after they’ve made their their changes and want the auto-generated documentation to reflect the new functions added and so on and so force. This tasks also generates the function index, re-generate the latest PDFs of README, USAGE or the CHANGELOG files.

Bashmatic comes with a Dockerfile that can be used to run tests or jsut manually validate various functionality under linux, and possibly to experiment.

Run make docker-build to create an docker image bashmatic:latest.

Run make docker-run-bash (or …​-zsh or …​-fish) to start a container with your favorite shell, and then validate if your functions work as expected.

Note how this dropped me straight into the Linux environment prompt with Bashmatic already installed.

Just look at this tiny, five-line script:

Results in this detailed and, let’s be honest, gorgeous ASCII output:

Tell me you are not at all excited to start writing complex installation flows in BASH right away?

Not only you get pretty output, but you can each executed command, it’s exit status, whether it’s been successful (green/red), as well each command’s bloody duration in milliseconds. What’s not to like?!?

Still not convinced?

Take a look at a more comprehensive example next.

In this example, we’ll download and install binaries kubectl and minikube binaries into /usr/local/bin

We provided an example script in examples/k8s-installer.sh. Please click and take a look at the source.

Here is the output of running this script:

Why do we think this type of installer is pretty awesome, compared to a silent but deadly shell script that "Jim-in-the-corner" wrote and now nobody understands?

Because:

1. The script goes out of its way to over-communicate what it does to the user.

2. It allows and reminds about a clean getaway (Ctrl-C)

3. It shares the exact command it runs and its timings so that you can eyeball issues like network congestions or network addresses, etc.

4. It shows in green exit code '0' of each command. Should any of the commands fail, you’ll see it in red.

5. It’s source code is terse, explicit, and easy to read. There is no magic. Just BASH functions.

Let’s get back to the Earth, and talk about how to install Bashmatic, and how to use it in more detail right after.

This final and most feature-rich example is not just an example – it’s a working functioning tool that can be used to install a bunch of developer dependencies on your Apple Laptop.

It’s located in bin/dev-setup and has many CLI flags:

In the example below we’ll use dev-setup script to install the following:

• Dev Tools

• PostgreSQL

• Redis

• Memcached

• Ruby 2.7.1

• NodeJS/NPM/Yarn

Despite that this is a long list, we can install it all in one command.

We’ll run this from a folder where our application is installed, because then the Ruby Version will be auto-detected from our .ruby-version file, and in addition to installing all the dependencies the script will also run bundle install and npm install (or yarn install). Not bad, huh?

This compact command line installs a ton of things, but don’t take our word for it - run it yourself. Or, at the very least enjoy this one extremely long screenshot :)

This is a great tool that colorizes nearly any other tool''s output.

Run it like so:

You might need to enter your password for SUDO.

Once it completes, run source ~/.bashrc (or whatever shell you use), and type something like ls -al or netstat -rn or ping 1.1.1.1 and notice how all of the above is nicely colored.

If you are using PostgreSQL, you are in luck! Bashmatic includes numerous helpers for PostreSQL’s CLI utility psql.

What is db top anyway?

Just like with the regular top you can see the "top" resource-consuming processes running on your local system, with dbtop you can observe a self-refreshing report of the actively running queries on up to three database servers at the same time.

Here is the pixelated screenshot of dbtop running against two live databases:

In order for this to work, you must first define database connection parameters in a YAML file located at the following PATH: ~/.db/database.yml.

Here is how the file should be organized (if you ever used Ruby on Rails, the standard config/database.yml file should be fully compatible):

Given the above file, you should be able to run the following command to see all available (registered in the above YAML file) connections:

Once that’s working, you should be able run dbtop:

DB Top Configuration:

You can configure the following settings for db top:

1. You can change the location of the database.yml file with db.config.set-file <filepath>

2. You can change the refresh rate of the dbtop with eg. db.top.set-refresh 0.5 (in seconds, fractional values allowed). This sets the sleep time between the screen is fully refreshed.

If you run db without any arguments, or with -h you will see the following:

As you might notice, there is an ever-growing list of "actions" — the sub-commands to the db script.

You can view the full list by passing --commands flag:

Altgernatively, here is the --examples view:

9.7.1. Sub-Command db connections

db connections
# OR
db --connections

9.7.2. Sub-Command db pga (eg. pg_activity)

9.7.3. Other Sub-Commands

$ db csv filestore "select * from files limit 2"

component_id,file_path,fingerprint_sha_256,fingerprint_comment_stripped_sha_256,license_info
6121f5b3-d68d-479d-9b83-77e9ca07dd2b,weiboSDK/src/main/java/com/sina/weibo/sdk/openapi/models/Tag.java,
6121f5b3-d68d-479d-9b83-77e9ca07dd2b,weiboSDK/src/main/java/com/sina/weibo/sdk/openapi/models/Comment.java,

Building atop of the powerful db script mechanics, is another powerful script called tablet.

The script is meant to be run against one database, and perform a table-level operation on a set of tables that can be specified in numerous ways. It started with the need to ANALYZE only some of the tables, specifically those that have not been auto-analyzed, but grew into a much more capable tool that can do things like:

• Analyze all tables in a database that have never been analyzed`

• Analyze all tables in a database that have not been analyzed in N days

• Analyze a set of specific tables, or exclude tables using regular expression

• Instead of analyzing tables, perform any other table-level command such as:

  • TRUNCATE

  • VACUUM and VACCUUM FULL

  • DROP TABLE

  • REINDEX TABLE

  • etc..

Below is the screenshot of the help screen from this script:

Bashmatic® provides a large number of functions, which are all loaded in your current shell. The functions are split into two fundamental groups:

• Functions with names beginning with a . are considered "private" functions, for example .run.env and .run.initializer

• All other functions are considered public.

The following conventions apply to all functions:

• We use the "dot" for separating namespaces, hence git.sync and gem.install.

• Function names should be self-explanatory and easy to read.

• DO NOT abbreviate words.

• All public functions must be written defensively: i.e. if the function is called from the Terminal without any arguments, and it requires arguments, the function must print its usage info and a meaningful error message.

For instance:

Now let’s run it properly:

The naming convention we use is a derivative of Google’s Bash StyleGuide, using . to separate BASH function namespaces instead of much more verbose ::.

After running the above, run bashmatic.functions function to see all available functions. You can also open the FUNCTIONS.adoc file to see the alphabetized list of all 422 functions.

To get a list of module or pattern-specific functions installed by the framework, run the following:

For instance:

You can list various modules by listing the lib sub-directory of the ${BASHMATIC_HOME} folder.

Note how we use Bashmatic® helper columnize [ columns ] to display a long list in five columns.

At a high level, the following modules are provided, in order of importance:

10.5.1. Runtime Framework — Executing Commands The Right Way™

❯ run.set-next show-output-off; run "ls -1 | wc -l";  run.set-next show-output-on; run "ls -1 | wc -l";
  ✔︎   ❯ ls -1 | wc -l ▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪〔     74 ms 〕    0
       # Command below will be shown with its output:
       ❯ ls -1 | wc -l
      17

  ✔︎  ▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪▪〔     80 ms 〕    0

$ bashmatic.functions-from 'run*'

run                                  run.set-next
run.config.detail-is-enabled         run.set-next.list
run.config.verbose-is-enabled        run.ui.ask
run.inspect                          run.ui.ask-user-value
run.inspect-variable                 run.ui.get-user-value
run.inspect-variables                run.ui.press-any-key
run.inspect-variables-that-are       run.ui.retry-command
run.inspect.set-skip-false-or-blank  run.variables-ending-with
run.on-error.ask-is-enabled          run.variables-starting-with
run.print-variable                   run.with.minimum-duration
run.print-variables                  run.with.ruby-bundle
run.set-all                          run.with.ruby-bundle-and-output
run.set-all.list

#!/usr/bin/env bash

# (See below on the location of .bashmatic and ways to install it)
source ${BASHMATIC_HOME}/init.sh

# configure global behavior of all run() invocations
run.set-all abort-on-error show-output-off

run "git clone https://gthub.com/user/rails-repo rails"
run "cd rails"
run "bundle check || bundle install"

# the following configuration only applies to the next invocation of `run()`
# and then resets back to `off`
run.set-next show-output-on
run "bundle exec rspec"

10.5.2. Controlling Output

$ bashmatic.functions-from output 3
abort                 error:               left-prefix
ascii-clean           h.black              ok
box.blue-in-green     h.blue               okay
box.blue-in-yellow    h.green              output.color.off
box.green-in-cyan     h.red                output.color.on
box.green-in-green    h.yellow             output.is-pipe
box.green-in-magenta  h1                   output.is-redirect
box.green-in-yellow   h1.blue              output.is-ssh
box.magenta-in-blue   h1.green             output.is-terminal
box.magenta-in-green  h1.purple            output.is-tty
box.red-in-magenta    h1.red               puts
box.red-in-red        h1.yellow            reset-color
box.red-in-yellow     h2                   reset-color:
box.yellow-in-blue    h2.green             screen-width
box.yellow-in-red     h3                   screen.height
box.yellow-in-yellow  hdr                  screen.width
br                    hl.blue              shutdown
center                hl.desc              stderr
columnize             hl.green             stdout
command-spacer        hl.orange            success
cursor.at.x           hl.subtle            test-group
cursor.at.y           hl.white-on-orange   ui.closer.kind-of-ok
cursor.down           hl.white-on-salmon   ui.closer.kind-of-ok:
cursor.left           hl.yellow            ui.closer.not-ok
cursor.rewind         hl.yellow-on-gray    ui.closer.not-ok:
cursor.right          hr                   ui.closer.ok:
cursor.up             hr.colored           warn
debug                 inf                  warning
duration              info                 warning:
err                   info:
error                 left

function valid-cask()  { sleep 1; return 0; }
function verify-cask() {
  inf "verifying brew cask ${1}...."
  if valid-cask ${1}; then
    ok:
  else
    not-ok:
  fi
}

 $ verify-cask TextMate
   ✔︎  verifying brew cask TextMate....

function valid-cask()  { sleep 1; return 1; }

$ verify-cask TextMate
  ✘    verifying brew cask TextMate....

$ h1 Hello

┌───────────────────┐
│ Hello             │
└───────────────────┘

output.is-tty && h1 "Yay For Terminals!"
output.has-stdin && echo "We are being piped into..."

10.5.3. Package management: Brew and RubyGems

#!/usr/bin/env bash

source ${BASHMATIC_HOME}/init.sh
h2 "Installing ruby gem sym and brew package curl..."
gem.install sym
brew.install.package curl

success "installed Sym version $(gem.version sym)"

10.5.4. Shortening URLs and Github Access

export BITLY_LOGIN="<your login>"
export BITLY_API_KEY="<your api key>"

$ url.shorten https://raw.githubusercontent.com/kigster/bashmatic/main/bin/install
# http://bit.ly/2IIPNE1

10.5.5. Github Access

github.clone                  github.setup
github.org                    github.validate

$ github.clone sym

  ✘    Validating Github Configuration...

       Please enter the name of your Github Organization:
       $ kigster

  Your github organization was saved in your ~/.gitconfig file.
  To change it in the future, run:

       $ github.org <org-name>

  ✔︎ $ git clone [email protected]:kigster/sym ▪▪▪▪▪▪〔     931 ms 〕

10.5.6. File Helpers

$ bashmatic.functions-from file

file.exists_and_newer_than     file.list.filter-non-empty
file.gsub                      file.size
file.install-with-backup       file.size.mb
file.last-modified-date        file.source-if-exists
file.last-modified-year        file.stat
file.list.filter-existing

 $ file.stat README.md st_size
22799

10.5.7. Array Helpers

$ bashmatic.functions-from array

array.to.bullet-list         array.includes
array.has-element            array.includes-or-exit
array.to.csv                 array.from.stdin
array-join                   array.join
array-piped                  array.to.piped-list
array.includes-or-complain

$ declare -a farm_animals=(chicken duck rooster pig)
$ array.to.bullet-list ${farm_animals[@]}
 • chicken
 • duck
 • rooster
 • pig
$ array.includes "duck" "${farm_animals[@]}" && echo Yes || echo No
Yes
$ array.includes  "cow" "${farm_animals[@]}" && echo Yes || echo No
No

10.5.8. Utilities

$ bashmatic.functions-from util

pause.long                     util.install-direnv
pause                          util.is-a-function
pause.short                    util.is-numeric
pause.medium                   util.is-variable-defined
util.append-to-init-files      util.lines-in-folder
util.arch                      util.remove-from-init-files
util.call-if-function          util.shell-init-files
shasum.sha-only                util.shell-name
shasum.sha-only-stdin          util.ver-to-i
util.functions-starting-with   util.whats-installed
util.generate-password         watch.ls-al

$ util.ver-to-i '12.4.9'
112004009
$ util.i-to-ver $(util.ver-to-i '12.4.9')
12.4.9

10.5.9. Ruby and Ruby Gems

$ bashmatic.functions-from ruby

bundle.gems-with-c-extensions  ruby.install-ruby-with-deps
interrupted                    ruby.install-upgrade-bundler
ruby.bundler-version           ruby.installed-gems
ruby.compiled-with             ruby.kigs-gems
ruby.default-gems              ruby.linked-libs
ruby.full-version              ruby.numeric-version
ruby.gemfile-lock-version      ruby.rbenv
ruby.gems                      ruby.rubygems-update
ruby.gems.install              ruby.stop
ruby.gems.uninstall            ruby.top-versions
ruby.init                      ruby.top-versions-as-yaml
ruby.install                   ruby.validate-version
ruby.install-ruby

$ ruby.top-versions
2.0.0-p648
2.1.10
2.2.10
2.3.8
2.4.9
2.5.7
2.6.5
2.7.0
2.8.0-dev

$ ruby.top-versions jruby
jruby-1.5.6
jruby-1.6.8
jruby-1.7.27
jruby-9.0.5.0
jruby-9.1.17.0
jruby-9.2.10.0

$ ruby.top-versions mruby
mruby-dev
mruby-1.0.0
mruby-1.1.0
mruby-1.2.0
mruby-1.3.0
mruby-1.4.1
mruby-2.0.1
mruby-2.1.0

$ bashmatic.functions-from gem

g-i                                           gem.gemfile.version
g-u                                           gem.global.latest-version
gem.cache-installed                           gem.global.versions
gem.cache-refresh                             gem.install
gem.clear-cache                               gem.is-installed
gem.configure-cache                           gem.uninstall
gem.ensure-gem-version                        gem.version

$ g-i awesome_print
  ✔︎    gem awesome_print (1.8.0) is already installed
$ gem.version awesome_print
1.8.0

10.5.10. Audio & Video Compression Helpers

 ❯ bashmatic.functions 1 | egrep -i 'video|audio'
audio.dir.mp3-to-wav
audio.dir.rename-karaoke-wavs
audio.dir.rename-wavs
audio.file.frequency
audio.file.mp3-to-wav
audio.make.mp3
audio.make.mp3.usage
audio.make.mp3s
video-squeeze
video.convert.compress

10.5.11. Additional Helpers

The following example is the actual code from a soon to be integrated AWS credentials install script. This code below checks that a user has a local ~/.aws/credentials file needed by the awscli, and in the right INI format. If it doesn’t find it, it checks for the access key CSV file in the ~/Downloads folder, and converts that if found. Now, if even that is not found, it prompts the user with instructions on how to generate a new key pair on AWS IAM website, and download it locally, thereby quickly converting and installing it as a proper credentials file. Not bad, for a compact BASH script, right? (of course, you are not seeing all of the involved functions, only the public ones).

Now, how would you use it in a script? Let’s say you need a script to upload something to AWS S3. But before you begin, wouldn’t it be nice to verify that the credentials exist, and if not – help the user install it? Yes it would.

And that is exactly what the code above does, but it looks like a DSL. because it is a DSL.

This script could be your bin/s3-uploader

Some bash files exists as libraries to be "sourced in", and others exist as scripts to be run. But users won’t always know what is what, and may try to source in a script that should be run, or vice versa - run a script that should be sourced in.

What do you, programmer, do to educate the user about correct usage of your script/library?

Bashmatic® offers a reliable way to test this:

If you’rather require a library to be sourced in, but not run, use the code as follows:

If you are stuck working on a monitor that does not support switching digit input from TV to PC, NOR does OS-X show the "underscan" slider in the Display Preferences, you may be forced to change the underscan manually. The process is a bit tricky, but we have a helpful script to do that:

This will reduce underscan by 5% compared to the current value. The total value is 10000, and is stored in the file /var/db/.com.apple.iokit.graphics. The tricky part is determining which of the display entries map to your problem monitor. This is what the script helps with.

Do not forget to restart after the change.

Acknowledgements: the script is an automation of the method offered on this blog post.

The framework comes with a bunch of automated unit tests based on the fantastic framework bats.

Bats is auto-installed by the bin/specs script.

12.1.1. Run Tests Using the Provided bin/specs script

12.1.2. Running Specs Sequentially with bin/spec -P

12.1.3. Run Tests Parallel using the Makefile

make test

12.1.4. Run Tests Sequentially using the Makefile

make test-sequential