SHCL: Shell Meets Common Lisp

SHCL is

1. a very customizable shell made with secret alien technology, and
2. an unholy union of POSIX Shell and Common Lisp.

SHCL is more than just a shell. It is a mutual embedding of POSIX Shell and Common Lisp. Behold Common Lisp embedded in POSIX shell embedded in Common Lisp! Notice that the Common Lisp form embedded in the shell expression can access the lexical environment.

(let ((rld "rld"))
  (capture (:stdout)
    #$ echo Hello ,(concatenate 'string "Wo" rld) #$))
; => "Hello World"

Now lay your eyes on a lisp function participating in a pipeline!

shcl> : ,(shcl/core/debug:graph-dependencies) | dot -Tpng > graph.png

The #$ reader macro isn’t just some hack that constructs a string to be evaluated by a “real” shell. The #$ reader macro fully parses the shell expression and constructs an equivalent Common Lisp form. SHCL IS the “real” shell!

SHCL/CORE/LISP-INTERPOLATION> (macroexpand-1 '#$ if true; then echo woo; fi #$)
(SHCL/CORE/SHELL-FORM:SHELL-IF
 (SHCL/CORE/SHELL-FORM:SHELL-RUN
  (WITH-FD-STREAMS NIL
    (EXPANSION-FOR-WORDS (LIST #<NAME "true">) :EXPAND-ALIASES T
                         :EXPAND-PATHNAME-WORDS T :SPLIT-FIELDS NIL))
  :ENVIRONMENT-CHANGES NIL :FD-CHANGES NIL)
 (SHCL/CORE/SHELL-FORM:SHELL-RUN
  (WITH-FD-STREAMS NIL
    (EXPANSION-FOR-WORDS (LIST #<NAME "echo"> #<NAME "woo">) :EXPAND-ALIASES T
                         :EXPAND-PATHNAME-WORDS T :SPLIT-FIELDS NIL))
  :ENVIRONMENT-CHANGES NIL :FD-CHANGES NIL))
T

Building SHCL

SHCL is only really tested against SBCL and CCL, but it should be portable to other lisp compilers. Be aware that ECL is known to be problematic because it tries to reap child processes automatically.

First, you’ll need to install some dependencies. To start with, you’ll need Clang and libedit. There’s also some Common Lisp dependencies that need to be taken care of: SBCL, Quicklisp, and cffi-grovel. If you’re new to building Common Lisp projects, you might want to let Roswell set up your lisp environment for you.

# Set up Clang, libedit, and Roswell
make LISP='ros -s cffi-grovel run --'

You can skip Roswell if you want. Just make sure that you set LISP to a command that runs SBCL with Quicklisp and cffi-grovel loaded. For example,

# Set up Clang, libedit, SBCL, and Quicklisp
QUICKLISP_SETUP=~/quicklisp/setup.lisp # or wherever you installed quicklisp
make LISP="sbcl --no-userinit --load \"$QUICKLISP_SETUP\" --eval '(ql:quickload :cffi-grovel)'"

If you use the Nix package manager, building SHCL is super easy! SHCL has a default.nix file, so you just need to run nix-build.

nix-build

Congratulations! You built SHCL! If you try to run shcl you’ll probably find that it doesn’t work because it can’t find libshcl-support. As part of the build, SHCL produces a shared library named (you guessed it!) libshcl-support. That library needs to be installed somewhere that the dynamic linker can find it. So, go ahead and use sudo make install to install SHCL and its support library! Don’t forget to set the PREFIX to something you’re happy with. Alternatively, you can just use the run-shcl script included in the repository. run-shcl just adds $(pwd) to the dynamic linker’s search path before invoking ./shcl.

Note: if you build SHCL using nix-build, then you don’t have to worry about libshcl-support. SHCL will know how to find it!

Example Usage

I don’t know what you’re expecting to see here. Its a POSIX-like shell. You can do (almost) all your normal POSIX shell stuff in it.

shcl> echo foobar
foobar
shcl> { echo foobar ; echo baz ; echo blip ; } | tail -n 1
blip
shcl> shcl-enable-lisp-syntax
shcl> if [ ,(+ 1 2 3) = ,(* 2 3) ]; then
> echo woah wait what
> fi
woah wait what
shcl> shcl-repl
shcl (lisp)> (define-builtin upcase ()
> (loop :for line = (read-line *standard-input* nil :eof)
>       :until (eq line :eof) :do
>       (format "~A~%" (string-upcase line)))
> 0)
UPCASE
shcl (lisp)> ^D
shcl> { echo ahhh ; echo what is going on ; } | upcase
AHHH
WHAT IS GOING ON

Okay, actually, that kind of went off the rails.