• Stars
    star
    1,682
  • Rank 26,595 (Top 0.6 %)
  • Language
    Python
  • License
    BSD 2-Clause "Sim...
  • Created about 6 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

shiv is a command line utility for building fully self contained Python zipapps as outlined in PEP 441, but with all their dependencies included.

pypi ci codecov docs license supported

snake

shiv

shiv is a command line utility for building fully self-contained Python zipapps as outlined in PEP 441, but with all their dependencies included!

shiv's primary goal is making distributing Python applications fast & easy.

📗 Full documentation can be found here.

sys requirements

  • python3.6+
  • linux/osx/windows

quickstart

shiv has a few command line options of its own and accepts almost all options passable to pip install.

simple cli example

Creating an executable of flake8 with shiv:

$ shiv -c flake8 -o ~/bin/flake8 flake8
$ ~/bin/flake8 --version
3.7.8 (mccabe: 0.6.1, pycodestyle: 2.5.0, pyflakes: 2.1.1) CPython 3.7.4 on Darwin

-c flake8 specifies the console script that should be invoked when the executable runs, -o ~/bin/flake8 specifies the location of the generated executable file and flake8 is the dependency that should be installed from PyPI.

Creating an interactive executable with the boto library:

$ shiv -o boto.pyz boto
Collecting boto
Installing collected packages: boto
Successfully installed boto-2.49.0
$ ./boto.pyz
Python 3.7.4 (v3.7.4:e09359112e, Jul  8 2019, 14:54:52)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import boto
>>> boto.__version__
'2.49.0'

installing

You can install shiv by simply downloading a release from https://github.com/linkedin/shiv/releases or via pip / pypi:

pip install shiv

You can even create a pyz of shiv using shiv!

python3 -m venv .
source bin/activate
pip install shiv
shiv -c shiv -o shiv shiv

developing

We'd love contributions! Getting bootstrapped to develop is easy:

git clone [email protected]:linkedin/shiv.git
cd shiv
python3 -m venv venv
source ./venv/bin/activate
python3 -m pip install --upgrade build
python3 -m build
python3 -m pip install -e .

Don't forget to run and write tests:

python3 -m pip install tox
tox

To build documentation when you changed something in docs:

python3 -m pip install -r docs/requirements.txt
sphinx-build docs build/html

gotchas

Zipapps created with shiv are not guaranteed to be cross-compatible with other architectures. For example, a pyz file built on a Mac may only work on other Macs, likewise for RHEL, etc. This usually only applies to zipapps that have C extensions in their dependencies. If all your dependencies are pure python, then chances are the pyz will work on other platforms. Just something to be aware of.

Zipapps created with shiv will extract themselves into ~/.shiv, unless overridden via SHIV_ROOT. If you create many utilities with shiv, you may want to occasionally clean this directory.


acknowledgements

Similar projects:

Logo by Juliette Carvalho

More Repositories

1

school-of-sre

At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.
HTML
7,628
star
2

css-blocks

High performance, maintainable stylesheets.
TypeScript
6,334
star
3

Burrow

Kafka Consumer Lag Checking
Go
3,644
star
4

databus

Source-agnostic distributed change data capture system
Java
3,587
star
5

qark

Tool to look for several security related Android application vulnerabilities
Python
3,117
star
6

dustjs

Asynchronous Javascript templating for the browser and server
JavaScript
2,917
star
7

cruise-control

Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.
Java
2,634
star
8

rest.li

Rest.li is a REST+JSON framework for building robust, scalable service architectures using dynamic discovery and simple asynchronous APIs.
Java
2,435
star
9

kafka-monitor

Xinfra Monitor monitors the availability of Kafka clusters by producing synthetic workloads using end-to-end pipelines to obtain derived vital statistics - E2E latency, service produce/consume availability, offsets commit availability & latency, message loss rate and more.
Java
1,991
star
10

dexmaker

A utility for doing compile or runtime code generation targeting Android's Dalvik VM
Java
1,843
star
11

greykite

A flexible, intuitive and fast forecasting library
Python
1,788
star
12

ambry

Distributed object store
Java
1,717
star
13

swift-style-guide

LinkedIn's Official Swift Style Guide
1,436
star
14

dr-elephant

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark
Java
1,341
star
15

detext

DeText: A Deep Neural Text Understanding Framework for Ranking and Classification Tasks
Python
1,254
star
16

parseq

Asynchronous Java made easier
Java
1,158
star
17

luminol

Anomaly Detection and Correlation library
Python
1,151
star
18

oncall

Oncall is a calendar tool designed for scheduling and managing on-call shifts. It can be used as source of dynamic ownership info for paging systems like http://iris.claims.
Python
1,095
star
19

test-butler

Reliable Android Testing, at your service
Java
1,040
star
20

goavro

Go
948
star
21

PalDB

An embeddable write-once key-value store written in Java
Java
934
star
22

brooklin

An extensible distributed system for reliable nearline data streaming at scale
Java
887
star
23

iris

Iris is a highly configurable and flexible service for paging and messaging.
Python
791
star
24

photon-ml

A scalable machine learning library on Apache Spark
Terra
790
star
25

Hakawai

A powerful, extensible UITextView.
Objective-C
780
star
26

URL-Detector

A Java library to detect and normalize URLs in text
Java
778
star
27

eyeglass

NPM Modules for Sass
TypeScript
741
star
28

opticss

A CSS Optimizer
TypeScript
715
star
29

coral

Coral is a translation, analysis, and query rewrite engine for SQL and other relational languages.
Java
714
star
30

LiTr

Lightweight hardware accelerated video/audio transcoder for Android.
Java
590
star
31

pygradle

Using Gradle to build Python projects
Java
584
star
32

kafka-tools

A collection of tools for working with Apache Kafka.
Python
581
star
33

flashback

mock the internet
Java
578
star
34

LayoutTest-iOS

Write unit tests which test the layout of a view in multiple configurations
Objective-C
565
star
35

FeatureFu

Library and tools for advanced feature engineering
Java
564
star
36

FastTreeSHAP

Fast SHAP value computation for interpreting tree-based models
Python
490
star
37

venice

Venice, Derived Data Platform for Planet-Scale Workloads.
Java
413
star
38

Spyglass

A library for mentions on Android
Java
381
star
39

dagli

Framework for defining machine learning models, including feature generation and transformations, as directed acyclic graphs (DAGs).
Java
353
star
40

ml-ease

ADMM based large scale logistic regression
Java
333
star
41

cruise-control-ui

Cruise Control Frontend (CCFE): Single Page Web Application to Manage Large Scale of Kafka Clusters
Vue
326
star
42

transport

A framework for writing performant user-defined functions (UDFs) that are portable across a variety of engines including Apache Spark, Apache Hive, and Presto.
Java
288
star
43

dph-framework

HTML
285
star
44

spark-tfrecord

Read and write Tensorflow TFRecord data from Apache Spark.
Scala
276
star
45

openhouse

Open Control Plane for Tables in Data Lakehouse
Java
244
star
46

isolation-forest

A Spark/Scala implementation of the isolation forest unsupervised outlier detection algorithm.
Scala
217
star
47

LiFT

The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.
Scala
167
star
48

shaky-android

Shake to send feedback for Android.
Java
157
star
49

pyexchange

Python wrapper for Microsoft Exchange
Python
151
star
50

asciietch

A graphing library with the goal of making it simple to graphs using ascii characters.
Python
137
star
51

python-avro-json-serializer

Serializes data into a JSON format using AVRO schema.
Python
136
star
52

li-apache-kafka-clients

li-apache-kafka-clients is a wrapper library for the Apache Kafka vanilla clients. It provides additional features such as large message support and auditing to the Java producer and consumer in the open source Apache Kafka.
Java
132
star
53

gdmix

A deep ranking personalization framework
Python
131
star
54

dynamometer

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.
Java
129
star
55

Avro2TF

Avro2TF is designed to fill the gap of making users' training data ready to be consumed by deep learning training frameworks.
Scala
125
star
56

linkedin-gradle-plugin-for-apache-hadoop

Groovy
117
star
57

datahub-gma

General Metadata Architecture
Java
112
star
58

dex-test-parser

Find all test methods in an Android instrumentation APK
Kotlin
104
star
59

cassette

An efficient, file-based FIFO Queue for iOS and macOS.
Objective-C
95
star
60

spaniel

LinkedIn's JavaScript viewport tracking library and IntersectionObserver polyfill
JavaScript
92
star
61

migz

Multithreaded, gzip-compatible compression and decompression, available as a platform-independent Java library and command-line utilities.
Java
76
star
62

sysops-api

sysops-api is a framework designed to provide visability from tens of thousands of machines in seconds.
Python
75
star
63

Hoptimator

Multi-hop declarative data pipelines
Java
74
star
64

avro-util

Collection of utilities to allow writing java code that operates across a wide range of avro versions.
Java
71
star
65

iceberg

A temporary home for LinkedIn's changes to Apache Iceberg (incubating)
Java
60
star
66

kube2hadoop

Secure HDFS Access from Kubernetes
Java
59
star
67

linkedin.github.com

Listing of all our public GitHub projects.
JavaScript
59
star
68

dynoyarn

DynoYARN is a framework to run simulated YARN clusters and workloads for YARN scale testing.
Java
58
star
69

Tachyon

An Android library that provides a customizable calendar day view UI widget.
Java
57
star
70

DuaLip

DuaLip: Dual Decomposition based Linear Program Solver
Scala
56
star
71

iris-relay

Stateless reverse proxy for thirdparty service integration with Iris API.
Python
49
star
72

Cytodynamics

Classloader isolation library.
Java
48
star
73

concurrentli

Classes for multithreading that expand on java.util.concurrent, adding convenience, efficiency and new tools to multithreaded Java programs
Java
43
star
74

iris-mobile

A mobile interface for linkedin/iris, built for iOS and Android on the Ionic platform
TypeScript
41
star
75

instantsearch-tutorial

Sample code for building an end-to-end instant search solution
JavaScript
39
star
76

lambda-learner

Lambda Learner is a library for iterative incremental training of a class of supervised machine learning models.
Python
37
star
77

tracked-queue

An autotracked implementation of a ring-buffer-backed double-ended queue
TypeScript
35
star
78

self-focused

Helps make a single page application more friendly to screen readers.
JavaScript
35
star
79

PASS-GNN

Python
35
star
80

QueryAnalyzerAgent

Analyze MySQL queries with negligible overhead
Go
35
star
81

TE2Rules

Python library to explain Tree Ensemble models (TE) like XGBoost, using a rule list.
Python
31
star
82

performance-quality-models

Personalizing Performance model repository
Jupyter Notebook
31
star
83

Iris-message-processor

Iris-message-processor is a fully distributed Go application meant to replace the sender functionality of Iris and provide reliable, scalable, and extensible incident and out of band message processing and sending.
Go
26
star
84

smart-arg

Smart Arguments Suite (smart-arg) is a slim and handy python lib that helps one work safely and conveniently with command line arguments.
Python
23
star
85

data-integration-library

The Data Integration Library project provides a library of generic components based on a multi-stage architecture for data ingress and egress.
Java
22
star
86

linkedin-calcite

LinkedIn's version of Apache Calcite
Java
22
star
87

atscppapi

This library provides wrappers around the existing Apache Traffic Server API which will vastly simplify the process of writing Apache Traffic Server plugins.
C++
20
star
88

high-school-trainee

LinkedIn Women in Tech High School Trainee Program
Python
18
star
89

play-parseq

Play-ParSeq is a Play module which seamlessly integrates ParSeq with Play Framework
Scala
17
star
90

forthic

Python
17
star
91

icon-magic

Automated icon build system for iOS, Android and Web
TypeScript
17
star
92

QuantEase

QuantEase, a layer-wise quantization framework, frames the problem as discrete-structured non-convex optimization. Our work leverages Coordinate Descent techniques, offering high-quality solutions without the need for matrix inversion or decomposition.
Python
15
star
93

kafka-remote-storage-azure

Java
13
star
94

play-restli

A library that simplifies building restli services on top of the play server.
Java
12
star
95

spark-inequality-impact

Scala
11
star
96

AlerTiger

Jupyter Notebook
9
star
97

Li-Airflow-Backfill-Plugin

Li-Airflow-Backfill-Plugin is a plugin to work with Apache Airflow to provide data backfill feature, ie. to rerun pipelines for a certain date range.
Python
8
star
98

gobblin-elr

This is a read-only mirror of apache/gobblin
Java
5
star
99

o19-bmc-firmware

OpenBMC is an open software framework to build a complete Linux image for a Board Management Controller (BMC)
C
4
star
100

linkedin-gtm-community-template

Smarty
4
star