• Stars
    star
    228
  • Rank 175,267 (Top 4 %)
  • Language
    Python
  • License
    MIT License
  • Created over 4 years ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

a shell resolver? :) (find and resolve shell script dependencies)

resholve references to external dependencies in shell scripts

Test

resholve replaces references to a bash/shell script's external dependencies (commands and sourced scripts) with absolute paths, ensuring they are declared, present, and don't shift if PATH changes.

Comparisons:

  • a linker for bash scripts
  • patchelf for shell scripts

It treats references it can't resolve as build-blocking errors until you tell it how to handle them, and then rewrites the script.

Convinced? Jump to the Quickstart. Otherwise, read on.

Wait, what?

Fair! It took me a while to figure out resholve should exist, myself. If you aren't sure you need it, you probably don't. Still, you should read a bit further so you'll recognize the needs resholve meets if you encounter them someday :)

(If you change your mind from here on, please open an issue to help me understand how you're thinking about this and refine the intro.)

  • If you want to understand what problems resholve addresses, read the next section.
  • If you want to see invocations and output, the Demos document is a good place to start.

What problem(s) does this solve?

resholve is a generic tool, but I'm building it so Nix/Nixpkgs can have have great shell packaging.

In the Nix ecosystem, resholve is already helping us:

  • discover and declare dependencies at package time instead of after runtime failures
    • keep unexpected versions of an executable or script from causing undefined behavior
  • avoid polluting PATH with all of a script's dependencies, which also means
    • no conflicts between tools different scripts need on PATH
    • no conflicts with other packages a user expects on PATH
    • no implicit dependency on the content of fragile rc/profile scripts
  • work directly with "normal" shell scripts
    • no polluting source scripts with template variables/syntax
    • no inlining shell scripts to readily inject absolute paths
    • no fragile .patch files
    • no fragile sed/awk text substitutions that might over-match

(Please open a PR/issue to document uses you discover!)

Quickstart

Note: resholve works for a living, but it is young. For example, it is only packaged for the Nix package manager. You may want to review the limitations section for more.

Here's how to get resholve to play with it. You should also review man resholve (or the plaintext copy).

CLI

NIXPKGS_ALLOW_INSECURE=1 nix-shell -p resholve

Note: resholve uses python2 because the high-quality parser it's built on does. Setting the NIXPKGS_ALLOW_INSECURE env is necessary to try resholve out in a shell because nixpkgs has taken steps to root out run-time usage of python2. resholve will still work at build-time for use in Nix packages. To be safe, don't run resholve on untrusted input.

Nix builder

The best documentation on the Nix builder is currently in nixpkgs.

DIY

git clone https://github.com/abathur/resholve.git
cd resholve
nix-shell
# I recommend nix-shell for 'man resholve' support

Acknowledgements

  • resholve leverages the Oil shell's OSH parser) and wouldn't be feasible without Andy Chu's excellent work on that project.

Limitations

We can't all be perfect :)

Documentation

  • The manpage is currently the canonical reference to resholve's options and behavior; the only online format is plaintext. See #19.

Packaging

  • My short-term goal is to support packaging shell projects for the Nix package manager. As such, the current build process depends on Nix.
    • The Nix API probably isn't granular enough, yet.
  • If you're interested in using resholve without Nix, I'll appreciate contributions that fill in build support for other environments.
  • For simplicity, resholve uses nixpkgs' python2 rather than Oil's fork of python2. This may cause subtle problems.

Known Gaps & Edge Cases in the utility itself

Because Shell is a very flexible, tricky language, resholve necessarily focuses on low-hanging-fruit tasks. Some of these will inevitably be supported over time, while others may stay out of reach. Please open an issue if you find a new one (and can't find an existing issue first).

The main areas I'm currently aware of:

  • In any Nix build, resholve now blocks resolution of some fundamental external utilities (such as su and sudo) that use run wrappers in NixOS. See #29 for more.
  • Because resholve makes assumptions about the behavior of some builtins in order to resolve scripts, it blocks if it looks like one is overridden by a function or alias. (This can likely be relaxed once I have a better sense of who/what/when/where/why/how these are overridden).
  • resholve doesn't have robust handling of variables that get executed like commands (this includes things like eval $variable and "$run_as_command" and $GIT_COMMAND status). There's some room for improvement here, but I also want to manage expectations because some cases are completely intractable without evaluating the script.
    • there's a first-level complication about seeing-through the variables themselves
    • and then a second-level issue with seeing-through double-quoted strings (for example, an eval )
  • fc -s has interesting behavior that makes it hard to account for
    • if I run ls /tmp and then echo blah and then fc -s 'ls', it'll re-run that previous ls command
      • if resholve rewrites ls to an absolute path, the fc -s command won't work as expected unless we also expand the ls inside the fc command
    • if I run ls /tmp and then fc -s tmp=sbin, it'll run ls /sbin; if I then run fc -s ls=stat, it runs stat /sbin
      • accounting for and triaging this will be very hard; there are no strict semantics here; we can substitute arbitrary text which could be executable names or arguments or even just parts of them; we'd have to be very explicitly parsing things out, or maybe extracting them into a mock test and running it, to know what to do
    • For now this is unaddressed. It probably makes the most sense to just raise a warning about not handling fc and link to a doc or issue about it, but I'm inclined to put this off until someone asks about it.

More Repositories

1

lilgit

a smol (quick) git status prompt plugin
Shell
19
star
2

shellswain

Shellswain enables simpler event-driven bash profile scripts & modules
Shell
11
star
3

binlore

YARA
8
star
4

shell-hag

shell hag is a shell-history aggregator
Shell
8
star
5

tdver

dreaming about test-driven versioning
8
star
6

SublimeLPC

Sublime syntax highlighting support for LDMud / LPC (Lars Pensjรถ C)
C
7
star
7

comity

bash library for neighborly trap/signal sharing in modular scripts
Shell
6
star
8

roomstead

may eventually contain t-ravis.com's ~MUD-engine (as a library? as a hugo theme? IDK.)
6
star
9

bashrc.nix

A nix flake for my bashrc to show how I build lilgit and shell-hag into my shell init
Nix
3
star
10

tdverpy

a WIP test-driven versioning tool (see https://github.com/abathur/tdver)
Python
2
star
11

dbg_census

Minimalistic Python wrapper for the Daybreak Games Census API.
Python
2
star
12

soe_api

Rudimentary Python API package for Sony Online Entertainment (SOE)'s stats/census API, includes Planetside 2 and Everquest 2
Python
2
star
13

nix-py-dev-oil

Nix
1
star
14

flask-neomodel-admin

Administrative backend scaffolding for neomodel, the neo4j ORM.
1
star
15

d-mark-python

Python port of the Dโ˜…Mark (semantic) markup language (https://github.com/denisdefreyne/d-mark)
Python
1
star
16

wordswurst

Python
1
star
17

syspolicyd_assessments

Shell
1
star
18

NeighborlyShell

brainstorming about a better Shell ecosystem
1
star
19

LittleGaryGitles

1
star
20

constellation

a simple take on project / workflow management in Sublime Text 3
Python
1
star
21

chef-autopackage

Allows you to include a list of string package names as node variable :autopackage to have these packages installed. Useful for installing packages from Vagrantfile without a custom recipe.
Ruby
1
star
22

yallback

Shell
1
star
23

doxy_db

WIP. Not sure how this will be eventually be released. Builds on work on updates to Doxygen's sqlite3 extension released in Doxygen 1.8.15 (but there's an unreleased memory safety fix, so I can only recommend building against Doxygen's master branch for now.)
Python
1
star