• Stars
    star
    398
  • Rank 108,325 (Top 3 %)
  • Language
    Shell
  • License
    Apache License 2.0
  • Created almost 10 years ago
  • Updated almost 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Containerized build environment for compiling an executable Golang package and packaging it in a light-weight Docker container.

golang-builder

Containerized build environment for compiling an executable Golang package and packaging it in a light-weight Docker container.

NOTE

This repo is no longer being maintained. Users are welcome to fork it, but we make no warranty of its functionality.

Overview

One of the (many) benefits of developing with Go is that you have the option of compiling your application into a self-contained, statically-linked binary. A statically-linked binary can be run in a container with NO other dependencies which means you can create incredibly small images.

With a statically-linked binary, you could have a Dockerfile that looks something like this:

FROM scratch
COPY hello /
ENTRYPOINT ["/hello"]

Note that the base image here is the 0 byte scratch image which serves as the root layer for all Docker images. The only thing in the resulting image will be the copied binary so the total image size will be roughly the same as the binary itself.

Contrast that with using the official golang image which weighs-in at 500MB before you even copy your application into it.

The golang-builder will accept your source code, compile it into a statically-linked binary and generate a minimal Docker image containing that binary.

The implementation of the golang-builder was heavily inspired by the Create the Smallest Possible Docker Container post on the Xebia blog.

Requirements

In order for the golang-builder to work properly with your project, you need to follow a few simple conventions:

Project Structure

The golang-builder assumes that your "main" package (the package containing your executable command) is at the root of your project directory structure.

.
├─Dockerfile
├─api
| ├─api.go
| └─api_test.go
├─greeting
| ├─greeting.go
| └─greeting_test.go
├─hello.go
└─hello_test.go

In the example above, the hello.go source file defines the "main" package for this project and lives at the root of the project directory structure. This project defines other packages ("api" and "greeting") but those are subdirectories off the root.

This convention is in place so that the golang-builder knows where to find the "main" package in the project structure.

If your "main" package does not reside at the root of your project structure, you can accomodate this by specifying the MAIN_PATH environment variable which references the relative path to your main package.

For example, suppose your structure is like the following (slightly modified from above so that your "main" package resides in the "cli" subfolder):

.
├─Dockerfile
├─api
| ├─api.go
| └─api_test.go
├─greeting
| ├─greeting.go
| └─greeting_test.go
└─cli
  ├─hello.go
  └─hello_test.go

You would then add MAIN_PATH=cli to your environment specification.

Canonical Import Path

In addition to knowing where to find the "main" package, the golang-builder also needs to know the fully-qualified package name for your application. For the "hello" application shown above, the fully-qualified package name for the executable is "github.com/CenturyLink/hello" but there is no way to determine that just by looking at the project directory structure (during the development, the project directory would likely be mounted at $GOPATH/src/github.com/CenturyLink/hello so that the Go tools can determine the package name).

In version 1.4 of Go an annotation was introduced which allows you to identify the canonical import path as part of your source code. The annotation is a specially formatted comment that appears immediately after the package clause:

package main // import "github.com/CenturyLink/hello"

The golang-builder will read this annotation from your source code and use it to mount the source code into the proper place in the GOPATH for compilation.

Dependencies

There's a good chance that your project imports at least one third-party Go package. The golang-builder obviously needs access to any packages that you've imported in order to compile your code. By default, golang-builder will go get any packages you've imported which aren't part of your project already.

The problem with doing a go get with each build is that golang-builder may end up with versions of packages which are different than those you developed against. Depending on the stability of the packages that you are importing this may not be an issue. However, if you want to maintain strict control over your dependency versions you may want to look at the Godep tool.

If you are using Godep to manage your dependencies golang-builder will reference the packages in your Godeps/_workspace directory instead of downloading them via go get.

Dockerfile

If you would like to have golang-builder package your compiled Go application into a Docker image automatically then the final requirement is that your Dockerfile be placed at the root of your project directory structure.

If your Dockerfile resides somewhere else, you can acommodate this by specifying the DOCKER_BUILD_CONTEXT env var. This will be used to override the default value of ".".

After compiling your Go application, golang-builder will execute a docker build with your Dockerfile.

The compiled binary will be placed (by default) in the root of your project directory so your Dockerfile can be written with the assumption that the application binary is in the same directory as the Dockerfile itself:

FROM scratch
EXPOSE 3000
COPY hello /
ENTRYPOINT ["/hello"]

In this case, the hello binary will be copied right to the root of the image and used as the entrypoint. Since we're using the empty scratch image as our base, there is no need to set-up any sort of directory structure inside the image.

If golang-builder does NOT see a Dockerfile in your project directory (or DOCKER_BUILD_CONTEXT directory) it will simply stop after compiling your application.

Usage

There are a few things that the golang-builder needs in order to compile your application code and wrap it in a Docker image:

  • Access to your source code. Inject your source code into the container by mounting it at the /src mount point with the -v flag.
  • Access to the Docker API socket. Since the golang-builder code needs to interact with the Docker API in order to build the final image, you need to mount /var/run/docker.sock into the container with the -v flag when you run it. If you omit the volume mount for the Docker socket, the application will be compiled but not packaged into a Docker image.

Assuming that the source code for your Go executable package is located at /home/go/src/github.com/CenturyLink/hello on your local system and you're currently in the hello directory, you'd run the golang-builder container as follows:

docker run --rm \
  -v "$(pwd):/src" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  centurylink/golang-builder

This would result in the creation of a new Docker image named hello:latest.

Note that the image tag is generated dynamically from the name of the Go package. If you'd like to specify an image tag name you can provide it as an argument after the image name.

docker run --rm \
  -v "$(pwd):/src" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  centurylink/golang-builder \
  centurylink/hello:1.0

If you just want to compile your application without packaging it in a Docker image you can simply run golang-builder without mounting the Docker socket.

docker run --rm -v $(pwd):/src centurylink/golang-builder

Additional Options

  • CGO_ENABLED - whether or not to compile the binary with CGO (defaults to false)
  • LDFLAGS - flags to pass to the linker (defaults to '-s')
  • COMPRESS_BINARY - if set to true, will use UPX to compress the final binary (defaults to false)
  • OUTPUT - if set, will use the -o option with go build to output the final binary to the value of this env var
  • MAIN_PATH - if set, this (relative) path will be used as the location of the "main" package
  • DOCKER_BUILD_CONTEXT - if set, this (relative) path will be used as the build context location (where the Dockerfile should reside)

The above are environment variables to be passed to the docker run command:

docker run --rm \
  -e CGO_ENABLED=true \
  -e LDFLAGS='-extldflags "-static"' \
  -e COMPRESS_BINARY=true \
  -e OUTPUT=/bin/my_go_binary \
  -v $(pwd):/src \
  centurylink/golang-builder

Cross-compilation

An additional image, centurylink/golang-builder-cross, exists that works identically to golang-builder save for the presence of the additional options presented above. This uses a larger base image that will build linux and OSX binaries for 32- and 64-bit, named like mypackage-darwin-amd64. This will use CGO, and you may find that some code – for example things from the os package – do not behave the same under cross-compilation in a container as they do natively compiled in OSX.

By default it will build Linux and OSX binaries for 32- and 64-bit but you can override this with environment variables.

docker run --rm
-e BUILD_GOOS="linux"
-e BUILD_GOARCH="arm amd64"
-v $(pwd):/src
centurylink/golang-builder-cross

This command will build Linux binaries for amd64 and arm.

More information can be found in the Docker Hub page for the official Go images.

SSL Verification

If your Go application needs to make calls to SSL endpoints you may find your application failing with a message like:

x509: failed to load system roots and no roots provided

One of the down-sides to using the scratch image is that you no longer have access to the root CA certificates which come pre-installed in most base images. There are a few different options for dealing with this:

  • Disable SSL verification. This is not recommended for obvious reasons.
  • Bundle the necessary root CA certificates as part of your application.
  • Use a different base image which already contains the root CA certificates.

We've created a minimal base image for applications that require SSL verification. The centurylink/ca-certs image is simply the scratch image with the most common root CA certificates pre-installed. The resulting image is only 258 kB which is still a good starting point for creating your own minimal images.

More Repositories

1

panamax-ui

The Web GUI for Panamax
JavaScript
1,439
star
2

dockerfile-from-image

Ruby
574
star
3

dray

An engine for managing the execution of container-based workflows.
Go
383
star
4

building

Build a Docker container for any app using Heroku Buildpacks
Ruby
230
star
5

zodiac

A lightweight tool for easy deployment and rollback of dockerized applications.
Go
196
star
6

fig2coreos

Convert fig.yml to CoreOS formatted systemd configuration files
Ruby
174
star
7

docker-image-graph

HTML
114
star
8

panamax-coreos

panamax-coreos installs the Panamax application, which is made up of the panamax-ui and panamax-api codebases.
Shell
84
star
9

panamax-api

The Local Panamax Agent
Ruby
82
star
10

ca-certs-base-image

Minimal Docker base image with standard certificate authority root certificates
Dockerfile
68
star
11

lorry-ui

Docker Compose YAML Editor
JavaScript
64
star
12

panamax-public-templates

Ruby
63
star
13

alpine-rails

A lightweight Rails image based on Alpine Linux
Dockerfile
45
star
14

panamax-contest-templates

Ruby
43
star
15

lorry

The API portion of Lorry UI a Docker Compose YAML Editor
Ruby
34
star
16

fleet-api

Ruby
28
star
17

docker-mysql

Shell
21
star
18

docker-ngrok

Dockerfile
19
star
19

docker-serf

Docker image for serf
Dockerfile
19
star
20

flatcar

Rails development environment manager
Ruby
17
star
21

ctlc-docker-wordpress

PHP
15
star
22

ctlc-docker-dropbox

A Dropbox Docker container
Dockerfile
14
star
23

nginx-ssl-proxy

Shell
14
star
24

docker-reg-client

Go Wrapper for the Docker Registry v1 API
Go
13
star
25

ctlc-docker-ambassador

Dockerfile
13
star
26

panamax-kubernetes-adapter-go

The Kubernetes adapter in combination with the Panamax remote agent enables the deployment of a Panamax template to a Kubernetes cluster.
Go
13
star
27

ctl-base-ui

A library of UI components for CTL labs web projects
CSS
12
star
28

ctlc-docker-haproxy-serf

Serf-aware Haproxy Docker Container
Shell
11
star
29

ctlc-docker-haproxy

Serf enabled haproxy docker container
Shell
11
star
30

openssl

Dockerfile
10
star
31

docker-sphinx

Docker image of Sphinx search
Dockerfile
9
star
32

docker-wordpress

PHP
8
star
33

panamax-remote-agent

The remote agent for managing Panamax deployments
Ruby
8
star
34

haproxy-etcd

Shell
8
star
35

docker-gitlab

Docker image for use with linked containers
Shell
8
star
36

docker-drupal

Dockerfile
8
star
37

panamax-kubernetes-adapter

The Kubernetes adapter in combination with the Panamax remote agent enables the deployment of a Panamax template to a Kubernetes cluster.
Ruby
8
star
38

docker-apache-php

Dockerfile
7
star
39

docker-wetty-cli

Dockerfile
7
star
40

panamax-ruby-base

Base image for Ruby-based container services
Ruby
6
star
41

ctlc-docker-amb-serf

Docker Ambassador with Serf registration built-in
Shell
6
star
42

ruby-base-image

Dockerfile
6
star
43

ctlc-docker-amb-etcd

Docker Ambassador with Etcd registration built-in
Shell
6
star
44

ctlc-docker-wordpress-serf

WordPress Docker Container with Serf built-in
Dockerfile
6
star
45

panamax-template-validator

Ruby
5
star
46

panamax-fleet-adapter

Ruby
5
star
47

panamax-marathon-adapter

The Marathon adapter in combination with the Panamax remote agent enables the deployment of a Panamax template to a Mesos cluster.
Go
5
star
48

buildpack-runner

Shell
5
star
49

prettycli

A generic library of helpers for pretty text output, intended for TUI applications.
Go
5
star
50

kube-install

Shell
5
star
51

docker-coreos-cli

Dockerfile
4
star
52

pmx-runner

Ruby
4
star
53

ctlc-docker-mysql-serf

Serf-aware MySQL Docker Container
Shell
4
star
54

panamaxcli

A Panamax Remote Agent command-line application
Go
4
star
55

docket

Python
4
star
56

alpine-nginx

Base Docker image containing Nginx running in Alpine Linux version 3.1.
Dockerfile
4
star
57

remote-agent-setup

Shell
3
star
58

ubuntu-rails

A lightweight Rails image based on Ubuntu
Dockerfile
3
star
59

lighttpd

Dockerfile
3
star
60

stackedit

Dockerfile for an image to run StackEdit https://stackedit.io
Shell
3
star
61

docker-postgresql

Shell
3
star
62

panamax-contrib

Shell
3
star
63

dot-files

Team mandated dot files
Vim Script
3
star
64

ctlc-docker-serf-members

Connect to a serf agent and print the active members
Dockerfile
3
star
65

docker-openstack-cli

Dockerfile
2
star
66

sample-go-adapter

Go
2
star
67

chinook

Ruby
2
star
68

panamax-remote-agent-go

Go
2
star
69

ctlc-docker-mysql

Shell
2
star
70

docker-data-container

Dockerfile
2
star
71

clcgo

A Go client for the CenturyLinkCloud API
Go
2
star
72

coreos-kubernetes

Shell
2
star
73

panamax-image-packager

Shell
1
star
74

kube-cluster-deploy

Go
1
star
75

docker-deis-cli

Dockerfile
1
star
76

docker-events

Ruby
1
star
77

socialize-etl

Ruby
1
star
78

ctlc-docker-nginx

Serf enabled nginx docker container
1
star
79

badge-monitor

Ruby
1
star
80

debian-rails

An optimized Debian-based Rails image
Dockerfile
1
star
81

dray-demo

Dray demo images
1
star
82

kwalify

Ruby
1
star
83

docker-aws-cli

Dockerfile
1
star
84

pmxadapter

Go
1
star
85

socialize-api

where the magic happens
Ruby
1
star
86

chimera

Shell
1
star
87

redis

Redis Docker image based on busybox
Dockerfile
1
star
88

socialize-ui

game changer
JavaScript
1
star
89

draycluster

Go
1
star
90

panamax-remote-agent-installer

Shell
1
star
91

testmux

Go
1
star
92

agent-server-deploy

Go
1
star
93

panamax-integration-suite

Ruby
1
star