• Stars
    star
    134
  • Rank 262,533 (Top 6 %)
  • Language
    Perl
  • License
    Other
  • Created over 9 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

CPAN Artifact Repository Manager

NAME

Carmel - CPAN Artifact Repository Manager

SYNOPSIS

# Run with a directory with cpanfile
carmel install

# list all the modules to be loaded
carmel list

# list all the modules in a tree
carmel tree

# show a location where a module is installed
carmel show Plack

# update Plack to the latest
carmel update Plack

# update all the modules in the snapshot
carmel update

# pin modules tp specific versions
carmel update [email protected] [email protected]

# show diffs for cpanfile.snapshot in a nice way
carmel diff

# Runs your perl script with modules from artifacts
carmel exec perl ...

# Requires all your modules in cpanfile in one shot
carmel exec perl -e 'use Carmel::Preload;'

# Roll out the currently selected modules into ./local
carmel rollout

# package modules tarballs and index into ./vendor/cache
carmel package

# use Carmel packages inside a script (without carmel exec)
perl -e 'use Carmel::Setup; ...'

# prints export PATH=... etc for shell scripting
carmel export

# find a module in a repository
carmel find DBI

# find a module matching the version query
carmel find Plack ">= 1.0000, < 1.1000"

DESCRIPTION

Carmel is yet another CPAN module manager.

Unlike traditional CPAN module installer, Carmel keeps the build of your dependencies in a central repository, then select the library paths to include upon runtime in development.

Carmel also allows you to rollout all the files in a traditional perl INC directory structure, which is useful to use in a production environment, such as containers.

WORKFLOW

Development

During the development, run carmel install when you setup a new environment, and any time you make changes to cpanfile. This will update your build artifacts, and saves the changes to cpanfile.snapshot. Commit the snapshot file in version control system so that you can reproduce the exact same versions across machines.

carmel exec makes it easy to run your application using the versions in cpanfile and cpanfile.snapshot dynamically.

# On your development environment
> cat cpanfile
requires 'Plack', '0.9980';
requires 'Starman', '0.2000';

> carmel install
> echo /.carmel >> .gitignore
> git add cpanfile cpanfile.snapshot .gitignore
> git commit -m "add Plack and Starman"

# On a new setup, or another developer's machine
> git pull
> carmel install
> carmel exec starman -p 8080 myapp.psgi

# Add a new dependency
> echo "requires 'Try::Tiny';" >> cpanfile
> carmel install
> git commit -am 'Add Try::Tiny'

# Update Plack to the latest
> carmel update Plack

Production Deployments

Carmel allows you to manage all the dependencies the same way across development environments and production environments. However, there might be cases where you want to avoid running your application with carmel exec in production, to avoid the overhead with large number of include paths, or to avoid installing Carmel in the production hosts. Carmel provides two easy ways to avoid depending on Carmel on the deploy target environments.

carmel rollout

carmel rollout rolls out the build artifacts into a regular perl5 library path in local. Once the rollout is complete, you can include the path just like a regular local::lib directory.

# Production environment: Roll out to ./local
> carmel rollout
> perl -Ilocal/lib/perl5 local/bin/starman -p 8080 myapp.psgi

You can run carmel rollout in a CI system to create the local directory next to your application code for a linux package (e.g. deb package), or Docker containers.

carmel package

carmel package (similar to carton bundle) creates a directory with tarballs and CPAN-style package index files, which you can pass to cpanm on a target machine. This way, you only need cpanm, which is available as a self-contained single executable, to bootstrap the installation on a host with a stock perl.

# Vendor all the packages to vendor/cache
> carmel package
> git add vendor/cache
> git commit -m 'Vendor all the tarballs'

# Remote environment (CI etc.)
> git clone https://.../myapp.git && cd myapp

# Install the modules to ./local (like carmel rollout)
> cpanm -L ./local --from $PWD/vendor/cache -nq --installdeps .

HOW IT WORKS

Carmel will keep the build directory (artifacts) after a cpanm installation in a repository, which defaults to $HOME/.carmel/{version}-{archname}/builds, and your directory structure would look like:

$HOME/.carmel/5.20.1-darwin-2level/builds
  Plack-1.0033/
    blib/
      arch/
      lib/
  URI-1.64/
    blib/
      arch/
      lib/
  URI-1.63/
    blib/
      arch/
      lib/

Carmel scans this directory and creates the mapping of which version of any package belongs to which build directory.

Given the list of modules and requirements from cpanfile, carmel install computes which versions satisfy the requirements best, and if there isn't, installs the modules from CPAN to put it to the artifact repository. The computed mappings are preserved as a snapshot in cpanfile.snapshot.

Once the snapshot is created, each following carmel command runs uses both cpanfile and cpanfile.snapshot to determine the best versions to satisfy the requirements. When you update cpanfile to bump a version or add a new module, carmel will install the new dependencies and update the snapshot accordingly.

carmel exec command, like install command, lists the build directories and .pm files you need from the repository, and then prepend the mappings of these files in the @INC hook. This is a handy way to run a perl program using the dependencies pinned by Carmel, without changing any include path.

carmel update command allows you to selectively update a dependency while preserving other dependencies in the snapshot. carmel update Plack for example pulls the latest version of Plack from CPAN (and its dependencies, if it needs a newer version than pinned in the snapshot), and updates the snapshot properly. Running carmel update without any arguments would update all the modules in cpanfile, including its dependencies.

On a production environment, you might want to use the carmel rollout command, which saves all the files included in the cpanfile, pinned with cpanfile.snapshot, to the local directory. This directory can be included like a regular perl's library path, with PERL5LIB=/path/to/local/lib/perl5, or with use lib, and you don't need to use carmel command in production this way.

SNAPSHOT SUPPORT

As of v0.1.29, Carmel supports saving and loading snapshot file in cpanfile.snapshot, in a compatible format with Carton. Versions saved in the snapshot file will be preserved across multiple runs of Carmel across machines, so that versions frozen in one environment can be committed to a source code repository, and can be reproduced in another box, so long as the perl version and architecture is the same.

CAVEATS / KNOWN ISSUES

  • If you run multiple instances of carmel, or hit Ctrl-C to interrupt the cpanm install session, Carmel might get into a state where some modules have been installed properly, while some modules in the dependency chain are missing. Carmel checks if there's another process running simultaneously using a lock file to prevent this problem, but make sure you let it finish the installation to get the full builds properly.

  • There're certain dependencies that get missed during the initial carmel install, and you'll see the error message "Can't find an artifact for Foo".

    Please report it to the issue tracker if you can reliably reproduce this type of errors. #74 has a list of known modules that could cause problems like this.

  • In some situation, you might encounter conflicts in version resolutions, between what's pinned in the snapshot and a new version that's needed when you introduce a new module.

    For example, suppose you have:

      # cpanfile
      requires 'Foo';
      requires 'Bar'; # which requires Foo >= 1.001
    

    Without a snapshot file, Carmel has no trouble resolving the correct versions for this combination. But if you have:

      # cpanfile.snapshot
      Foo-1.000
    

    The first time you run carmel install, Carmel will try to install Foo-1.000, because that's the version pinned in the snapshot, while trying to pull the module Bar, which would conflict with that version of Foo.

    This can happen 50% of the time, because if cpanm (called internally by Carmel) installs Bar first, then the resolution is done correctly and the version in the snapshot would be skipped, and the snapshot will be updated accordingly. This is due to perl's hash randomization after Perl 5.18.

    To avoid this, you're recommended to run carmel install before making any changes to cpanfile. That will put the build caches to satisfy what's in cpanfile and the snapshot. After that, adding a new dependency will likely reuse what's in the build cache, while adding a new dependency can update the transient dependency (for Foo) without having conflicts.

    If you encounter conflicts like this, you can work around it by:

    • Run carmel update Foo to pull the latest version of Foo from CPAN, ignoring what's in the snapshot.
    • Update cpanfile to explicitly update the version requirement for Foo.
  • Carmel doesn't support Taint mode (-T). You'll see an error message Insecure dependency in require while running with -T switch.

COMPARISONS WITH SIMILAR TOOLS

Carton

Carmel shares the same goal with Carton, where you can manage your dependencies by declaring them in cpanfile, and pinning them in cpanfile.snapshot. Most of the commands work the same way, so Carmel can most effectively be a drop-in replacement for Carton, if you're currently using it.

Here's a few key differences between Carmel and Carton:

  • Carton does not manage what's currently being installed in local directory. It just runs cpanm command with -L local, with a hope that nothing has changed the directory except Carton, and whatever is in the directory won't conflict with the snapshot file. This can easily conflict when cpanfile.snapshot is updated by multiple developers or when you continuously update the dependencies across multiple machines.

    Carmel manages all the dependencies for your project in the Carmel repository under $HOME/.carmel, and nothing is installed under your project directory on development. The local directory is only created when you request it via carmel rollout command, and it's safe to run multiple times. Running carmel install after pulling the changes to the snapshot file will always install the correct dependencies from the snapshot file, as compared to Carton, which doesn't honor the snapshot on a regular install command, if whatever version in local already satisfies the version in cpanfile.

  • Carton has no easy way to undo a change once you update a version of a module in local, because which version is actually selected is only preserved as a file inside the directory, that's not managed by Carton. To undo a change you have to remove the entire local directory to start over.

    Carmel preserves this information to the cpanfile.snapshot file, and every invocation of Carmel resolves the dependencies declared in cpanfile and pinned in cpanfile.snapshot dynamically, to create a stable dependency tree, without relying on anything in a directory under your project other than the snapshot file. Undoing the change in cpanfile.snapshot file immediately reverts the change.

cpm

App::cpm is an excellent standalone CPAN installer.

  • Like Carton, cpm installs the dependencies declared in cpanfile to local. Carmel installs them into a build cache, and doesn't use local directory for state management. You can run carmel rollout to copy the dependencies to local directory.
  • cpm installs the modules in parallel, which makes the installation very fast. Like Carmel, cpm also manages its build artifacts cache, so a module that has previously been installed would be very fast to install, since there's no build process.
  • Unlike Carton and Carmel, cpm doesn't have the ability to manage cpanfile.snapshot file on its own. It can read the snapshot however, so it's possible to use Carmel in a development environment, and then use cpm install instead of carmel install and carmel rollout, if all you need is to build out a perl5 library path out of cpanfile and cpanfile.snapshot in the source code repository.

COMMUNITY

AUTHOR

Tatsuhiko Miyagawa [email protected]

COPYRIGHT

Copyright 2015- Tatsuhiko Miyagawa

LICENSE

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

SEE ALSO

App::cpanminus Carton

More Repositories

1

cpanminus

cpanminus - get, unpack, build and install modules from CPAN
Perl
734
star
2

Starman

Starman is a high-performance preforking Perl PSGI web server
Perl
283
star
3

github-growler

Growl github updates
Perl
215
star
4

hubot-cron

Crontab like scheduling messages for Hubot
CoffeeScript
214
star
5

Tabelog-HonestStars

Make Tabelog Stars more honest
JavaScript
184
star
6

plagger

Pluggable RSS/Atom aggregator
Perl
181
star
7

plack-handbook

166
star
8

Tatsumaki

Plack-based nonblocking Web framework for IO-bound delayed response, server push (streaming) and long-poll comet
JavaScript
136
star
9

remedie

perl based pluggable media center application
Perl
121
star
10

Twiggy

AnyEvent based non-blocking PSGI web server (like Thin)
JavaScript
104
star
11

web-scraper

Perl web scraping toolkit
Perl
100
star
12

heroku-buildpack-perl

Heroku buildpack to run Perl/PSGI apps using Starman HTTP server
Shell
98
star
13

hubot-standup

Agile style standup bot with hubot, ala tender
CoffeeScript
95
star
14

cpanfile

Yet another way to declare CPAN dependencies
Perl
81
star
15

faraday-cookie_jar

Client-side cookie management for Faraday
Ruby
75
star
16

ripdiko

Rip radiko programs
Ruby
62
star
17

rack-vcr

Rack middleware to capture incoming HTTP requests/responses with VCR
Ruby
56
star
18

Dist-Milla

Opinionated and Unobtrusive distribution builder
Perl
42
star
19

AnyEvent-Redis

Asynchronous Redis client
Perl
41
star
20

AnyEvent-Twitter-Stream

AnyEvent based Twitter stream consumer
Perl
39
star
21

Corona

Coro based high-performance, asynchronous PSGI web server
Perl
39
star
22

plenv-contrib

Perl
38
star
23

Filesys-Notify-Simple

Simple and dumb file system watcher
Perl
32
star
24

xml-atom

XML::Atom perl module
Perl
29
star
25

fatbundle

Fat bundle for your Ruby script
Ruby
29
star
26

docker-plenv-vanilla

Shell
29
star
27

plack-dispatching-samples

Examples of Plack dispatcher using various CPAN modules
Perl
28
star
28

gistp

Yet another gist command line tool (like gisty)
Perl
28
star
29

Webhook-Growler

Receives Web hooks over Reverse HTTP and notifies via Growl
Perl
28
star
30

fastpass

Tiny, XS free, standalone and preforking FastCGI daemon for PSGI
Perl
26
star
31

Plack-Middleware-Debug

Debug panel middleware for Plack
Perl
26
star
32

HTTPx-Weblet

the Request/Response objects formerly known as the-old-Plack::Request
Perl
26
star
33

Catalyst-Engine-PSGI

PSGI engine for Catalyst
Perl
25
star
34

resque-top

top for Resque
Ruby
25
star
35

TheSchwartz-Web

Web UI to list and control TheSchwartz jobs
JavaScript
24
star
36

MT-PubSubHubbub

PubSubHubbub plugin for Movable Type
Perl
23
star
37

Plack-Middleware-InteractiveDebugger

Interactive on-browser debugger for Plack ala Werkzeug
Perl
23
star
38

daily_digest

Daily Digest for Pocket (with Readability, calibre and IFTTT)
Ruby
23
star
39

Hash-MultiValue

Hash with multiple values per key, inspired by MultiDict.py of WebOb
Perl
22
star
40

plackperl.org

plackperl.org website files
HTML
21
star
41

Subfeedr

Open PubSubHubbub Hub that does polling for you, built on top of Tatsumaki and AnyEvent
Perl
21
star
42

dircaster

Generate RSS 2.0 podcast out of directory with *.mp3s
Ruby
20
star
43

perl-app-builder

Turns your perl script into .app using Platypus
Perl
19
star
44

hubot-lingr

CoffeeScript
19
star
45

asana_digest

Asana digest to Hipchat
Ruby
18
star
46

cpanf

Query and install CPAN modules realtime from the fresh mirror
Perl
18
star
47

catalyst-view-json

Not maintained: see https://github.com/shadowcat-mst/catalyst-view-json
Perl
18
star
48

Data-Section-Simple

read data from DATA for inline templates and more
Perl
17
star
49

jquery-fitimage

jQuery plugin to fit images while keeping the width/height ratio
JavaScript
16
star
50

Fakehose

Fake Twitter streaming API server
Perl
16
star
51

Net-Twitter-OAuth

Net::Twitter with OAuth-based authentication
Perl
16
star
52

yapcasia2015

HTML
15
star
53

Piglet

This is an incomplete project - DO NOT USE IT
Perl
15
star
54

Plack-App-PAR

Run .par as a PSGI application
Perl
15
star
55

cpanmetadb

CPAN metadata DB on Google AppEngine
Python
15
star
56

forkprove

Forking prove that allows you to preload modules with -M
Perl
14
star
57

dayone_import_tweets

Simple ruby script to import tweets into @dayoneapp
Ruby
13
star
58

google-contacts-gravatar

Import avatar photos from Gravatar to Google Contacts
Perl
13
star
59

www-nicovideo-download

Download videos from nicovideo.jp
Perl
13
star
60

yelp-bookmarks-vcards

Export Yelp bookmarks to vCard
Perl
12
star
61

cpan-module-bootstrap

Shell
12
star
62

HTTP-Tinyish

HTTP::Tiny compatible wrappers for LWP, curl, wget
Perl
12
star
63

module-install-repository

Module::Install plugin to auto-set repository URL
Perl
12
star
64

Sunaba

Plack/PSGI wrapper for dankogai's lleval sandbox
Perl
12
star
65

CGI-PSGI

Enable PSGI in your CGI.pm aware apps
Perl
12
star
66

Devel-StackTrace-AsHTML

Display stacktrace in HTML
HTML
11
star
67

cpan-timeline

Gives you a dashboard listing your friends activities on CPAN
Perl
11
star
68

HTTP-Server-Simple-PSGI

Semi dependency free PSGI web server
Perl
11
star
69

AnyEvent-ReverseHTTP

reversehttp client using AnyEvent
Perl
11
star
70

Perlbal-Plugin-PSGI

Perlbal plugin to run PSGI applications
Perl
10
star
71

test-synopsis

Test your code in SYNOPSIS
Perl
10
star
72

theschwartz-simple

Lightweight TheSchwartz job dispatcher using plain DBI
Perl
10
star
73

AnyEvent-DBI-Abstract

AnyEvent::DBI + SQL::Abstract
Perl
10
star
74

Acme-Test-VW

Makes your tests always pass under CI
Perl
10
star
75

geo-coder-google

Geo::Coder::Google
Perl
9
star
76

perldocjp-site

Perl
9
star
77

Module-Install-CPANfile

Include dependencies from cpanfile to replace META files
Perl
9
star
78

hubot-webhook

Generic webhook plugin for Hubot
CoffeeScript
9
star
79

Plack-Middleware-File-Sass

PSGI middleware for Sass templates
Perl
9
star
80

cpan-realtime-bot

Bot behind friendfeed.com/cpan
Perl
8
star
81

cpan-command-repo

Adds a new 'repo' command to CPAN shell
Perl
8
star
82

cpan-listchanges

List changes for CPAN modules
Perl
8
star
83

Plack-Middleware-REPL

REPL shell when your PSGI application raises exceptions
Perl
8
star
84

btbattery

Get BatteryPercent of your all BlueTooth devices
Ruby
8
star
85

Mac-Spotlight

CPAN Mac::Spotlight maint repo
C
8
star
86

cpan-mini-growl

Growl mirrored updates with minicpan
Perl
8
star
87

AnyEvent-FriendFeed-Realtime

Subscribe to FriendFeed real-time updates
Perl
7
star
88

File-Spotlight

List files from Smart Folder by reading .savedSearch files
Perl
7
star
89

CGI-Compile

Compile .cgi script into a sub
Perl
7
star
90

XML-Liberal

Patch XML
Perl
7
star
91

LWP-Protocol-PSGI

hijack LWP stack with your own PSGI application
Perl
7
star
92

HTML-Selector-XPath

Perl
7
star
93

Net-DNSSD

XS binding for dns_sd.h DNS Service Discovery (aka Bonjour)
C
7
star
94

Date-Japanese-Era

Perl
7
star
95

CPAN-Any

Install Perl modules using any CPAN clients
Perl
7
star
96

camp

stuff
Perl
7
star
97

Plack-Middleware-Hoptoad

Send application errors to Hoptoad
Perl
7
star
98

php-session

reads and writes PHP4 session files
Perl
7
star
99

App-PAUSE-Comaint

Make someone comaint of your module on PAUSE
Perl
7
star
100

nightflight

yakan-hiko.com → Gmail → Mailgun → Heroku → IFTTT
Ruby
7
star