lukeed/ley

TODO

WIP: What's here is the end of Night #1

• driver support:
  • pg
  • postgres
  • mysql
  • mysql2
  • better-sqlite3
  • sqlite
• complete test coverage
• complete documentation

Features

• Agnostic
  Supports postgres, pg, better-sqlite3, sqlite, mysql, mysql2, and custom drivers!

• Lightweight
  Does not include any driver dependencies.

• Transactional
  Runs all migration files within a transaction for rollback safety.

• Familiar
  Does not invent new syntax or abstractions.
  You're always working directly with your driver of choice.

• Flexible
  Find the CLI to restrictive? You may require ley for your own scripting!

Install

$ npm install --save-dev ley

Usage

Both Programmatic and CLI usages are supported.

Setup

You must have a migrations directory created, preferably in your project's root.

Note: You may configure the target directory and location.

Your filenames within this directory determine the order of their execution.
Because of this, it's often recommended to prefix migrations with a timestamp or numerical sequence.

Numerical Sequence

/migrations
  |-- 000-users.js
  |-- 001-teams.js
  |-- 002-seats.js

Note: You may create the next file via ley new todos --length 3 where todos is a meaningful name.
The above command will create the migrations/003-todos.js filepath.

Timestamped

/migrations
  |-- 1581323445-users.js
  |-- 1581323453-teams.js
  |-- 1581323458-seats.js

Note: You may create the next file via ley new todos --timestamp where todos is a meaningful name.
The above command will create the migrations/1584389617-todos.js filepath...or similar.

The order of your migrations is critically important!
Migrations must be treated as an append-only immutable task chain. Without this, there's no way to reliably rollback or recreate your database.

Example: (Above) You cannot apply/create 001-teams.js after 002-seats.js has already been applied.
Doing so would force your teammates or database replicas to recreate "the world" in the wrong sequence.
This may not always pose a problem (eg, unrelated tasks) but it often does and so ley enforces this practice.

Lastly, each migration file must have an up and a down task.
These must be exported functions — async okay! — and will receive your pre-installed client driver as its only argument:

exports.up = async function (DB) {
  // with `pg` :: DB === pg.Client
  await DB.query(`select * from users`);

  // with `postgres` :: DB === sql``
  await DB`select * from users`;
}

exports.down = async function (DB) {
  // My pre-configured "undo" function
}

CLI

1. Add ley as one of your package.json scripts; "migrate", for example:

   // package.json
   {
     "scripts": {
       "migrate": "ley"
     }
   }

2. Invoke ley up to apply new migrations, or ley down to rollback previous migrations.

   $ npm run migrate up
   $ yarn migrate up

   
   
   
   

Programmatic

Note: See API for documentation

With programmatic/scripting usage, you will not inherit any of ley's CLI tooling, which includes all colors and error formatting. Instead, you must manually catch & handle all thrown Errors.

const ley = require('ley');

const successes = await ley.up({ ... });

Config

TL;DR: The contents of a ley.config.js file (default file name) is irrelevant to ley itself!

A config file is entirely optional since ley assumes that you're providing the correct environment variable(s) for your client driver. However, that may not always be possible. In those instances, a ley.config.js file (default file name) can be used to adjust your driver's connect method – the file contents are passed directly to this function.

For example, if your hosting provider sets non-standard environment variables for the client driver (like Heroku does), you could extract the information and set the standard environment variables:

// ley.config.js
if (process.env.DATABASE_URL) {
  const { parse } = require('pg-connection-string');

  // Extract the connection information from the Heroku environment variable
  const { host, database, user, password } = parse(process.env.DATABASE_URL);

  // Set standard environment variables
  process.env.PGHOST = host;
  process.env.PGDATABASE = database;
  process.env.PGUSERNAME = user;
  process.env.PGPASSWORD = password;
}

Or, if your database provider requires certain SSL connection options to be set in production, you could do that:

// ley.config.js
const options = {};

if (process.env.NODE_ENV === 'production') {
  options.ssl = true;
}

module.exports = options;

When the config filename uses the .js extension, then ley will attempt to auto-load a .mjs or a .cjs variant of the file if/when the original .js file was not found. This means that, by default, these files are searched (in order):

• ley.config.js
• ley.config.mjs
• ley.config.cjs

ES Modules

As of [email protected] and Node.js 12+, you may choose to use ECMAScript modules (ESM). There are a few ways to take advantage of this:

Note: These are separate options. You do not need to perform both items

1. Define "type": "module" in your root package.json file.
   This signals the Node.js runtime that all *.js files in the project should be treated as ES modules. With this setting, you may only use CommonJS format within .cjs files.

   // package.json
   {
     "type": "module",
     "scripts": {
       "migrate": "ley"
     }
   }

2. Author ES modules only in .mjs files.
   Regardless of the value of the "type" field (above), .mjs files are always treated as ES modules and .cjs files are always treated as CommonJS.

In terms of ley usage, this means that your config file may use ESM syntax. Similarly, by default, both ley.config.mjs and ley.config.cjs will be auto-loaded, if found and ley.config.js is missing.

// ley.config.mjs
// or w/ "type": "module" ~> ley.config.js
export default {
	host: 'localhost',
	port: 5432,
	// ...
}

Finally, migration files may also be written using ESM syntax:

// migrations/000-example.mjs
// or w/ "type": "module" ~> migrations/000-example.js
export async function up(DB) {
  // with `pg` :: DB === pg.Client
  await DB.query(`select * from users`);

  // with `postgres` :: DB === sql``
  await DB`select * from users`;
}

export async function down(DB) {
  // My pre-configured "undo" function
}

You may generate new migration files in ESM syntax by passing the --esm flag to the ley new command:

$ ley new todos --esm
#=> migrations/003-todos.mjs

$ cat migrations/003-todos.mjs
#=> export async function up(client) {
#=> }
#=> 
#=> export async function down(client) {
#=> }

Drivers

Out of the box, ley includes drivers for the following npm packages:

• postgres
• pg
• mysql
• mysql2
• better-sqlite3

When no driver is specified, ley will attempt to autodetect usage of these libraries in the above order.

However, should you need a driver that's not listed – or should you need to override a supplied driver – you may easily do so via a number of avenues:

1. CLI users can add --driver <filename> to any command; or
2. Programmatic users can pass opts.driver to any command; or
3. A ley.config.js file can export a special driver config key.

With any of these, if driver is a string then it will be passed through require() automatically. Otherwise, with the latter two, the driver is assumed to be a Driver class and is validated as such.

Important: All drivers must adhere to the Driver interface!

Typed Migrations

For extra confidence while writing your migration file(s), there are two options:

TypeScript

1. Ensure tsm is installed

2. Run ley with the require option so that tsm can process file(s)

   $ ley -r tsm <cmd>
   # or
   $ ley --require tsm <cmd>

JSDoc

You may also use JSDoc annotations throughout your file to achieve (most) of the benefits of TypeScript, but without installing and configuring TypeScript.

/** @param {import('pg').Client} DB */
exports.up = async function (DB) {
  await DB.query(...)
}

API

Important: See Options for common options shared all commands.
In this API section, you will only find command-specific options listed.

ley.up(opts?)

Returns: Promise<string[]>

Returns a list of the relative filenames (eg, 000-users.js) that were successfully applied.

opts.single

Type: boolean
Default: false

Enable to apply only one migration file's up task.
By default, all migration files will be queue for application.

ley.down(opts?)

Returns: Promise<string[]>

Returns a list of the relative filenames (eg, 000-users.js) that were successfully applied.

opts.all

Type: boolean
Default: false

Enable to apply all migration files' down task.
By default, only the most recently-applied migration file is invoked.

ley.status(opts?)

Returns: Promise<string[]>

Returns a list of the relative filenames (eg, 000-users.js) that have not yet been applied.

ley.new(opts?)

Returns: Promise<string>

Returns the newly created relative filename (eg, 000-users.js).

opts.filename

Type: string

Required. The name of the file to be created.

Note: A prefix will be prepended based on opts.timestamp and opts.length values.
If your input does not already end with an extension, then .js or .mjs will be appended.

opts.esm

Type: boolean
Default: false

Create a migration file with ESM syntax.

Note: When true, the opts.filename will contain the .mjs file extension unless your input already has an extension.

opts.timestamp

Type: boolean
Default: false

Should the migration file have a timestamped prefix?
If so, will use Date.now() floored to the nearest second.

opts.length

Type: number
Default: 5

When not using a timestamped prefix, this value controls the prefix total length.
For example, 00000-users.js will be followed by 00001-teams.js.

Options

Note: These are available to all ley commands.
See API for programmatic command documentation.

opts.cwd

Type: string
Default: .

A target location to treat as the current working directory.

Note: This value is path.resolve()d from the current process.cwd() location.

opts.dir

Type: string
Default: migrations

The directory (relative to opts.cwd) to find migration files.

opts.driver

Type: string or Driver
Default: undefined

When defined and a string, this can be (a) the name of an internal driver, (b) the name of a third-party driver module, or (c) a filepath to a local driver implementation. It will pass through require() as written.

When defined an not a string, it's expected to match the Driver interface and will be validated immediately.

When undefined, ley searches for all supported client drivers in this order:

['postgres', 'pg', 'mysql', 'mysql2', 'better-sqlite3']

opts.config

Type: object
Default: undefined

A configuration object for your client driver to establish a connection.
When unspecified, ley assumes that your client driver is able to connect through process.env variables.

Note: The ley CLI will search for a ley.config.js config file (configurable).
If found, this file may contain an object or a function that resolves to anything of your chosing.
Please see Config for more information.

opts.require

Type: string or string[]
Default: undefined

A module name (or list of names) to be required by ley at startup.

For example, you may want to use dotenv to load existing .env file(s) in your project:

const ley = require('ley');

const files = await ley.status({
  require: ['dotenv/config']
});

Through CLI usage, this is equivalent to:

$ ley -r dotenv/config status
# or
$ ley --require dotenv/config status

License

MIT © Luke Edwards