• Stars
    star
    190
  • Rank 199,014 (Top 4 %)
  • Language
    Shell
  • License
    Other
  • Created over 5 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Image validation, automation, and other tools for DigitalOcean Marketplace Vendors and Custom Image users

DigitalOcean Marketplace Partner Tools

Apache license Pull Requests Welcome

This repository contains resources for DigitalOcean Marketplace partners, like documentation on image requirements and creation, tools for image cleanup and validation, and templates for build automation.

Getting Started creating your Kubernetes based 1-Click App

Please visit our Kubernetes Marketplace repo for instructions on how to create and submit your Kubernetes based 1-Click App.

Kubernetes 1-Click Apps can be submitted via our Marketplace Kubernetes Github repo.

Getting Started creating your Droplet based 1-Click App

The overall process for creating an image that you can submit as a Droplet based 1-Click App is as follows:

  1. Create and configure a build Droplet manually first to make sure your configuration works. You can create a build Droplet with any method, like the control panel, the API, or command-line tools like doctl. We strongly encourage you to use a $6 Droplet as your build Droplet. Using a $6 Droplet as your build Droplet will ensure that all Droplet types will be available for usage with your 1-Click App.

  2. Clean up and validate the build Droplet with the provided scripts, cleanup.sh and img_check.sh. The scripts will check for and fix potential security concerns and verify that the image will be compatible with Marketplace.

  3. Use Packer to create a fresh snapshot of the image that you want to create. While there are several ways to create an image, we recommend using Packer as the most simple and consistent option.

  4. Submit your final image to the Marketplace team for review. This can be made through our Vendor Portal. If you've signed expressed interest in joining the Marketplace through the form on this page but you've not received a login for the Vendor Portal, please reach out to [email protected] and we'll help you out.

  1. (Optional) Integrate a DigitalOcean Managed Database into your 1-Click App

    As a Vendor, you can offer a DigitalOcean Managed Database (DBaaS) to any DigitalOcean customer at the time they spin up your Droplet based 1-Click App. You can customize your app image to integrate with the managed database directly, or let your customers complete the configuration themselves after the 1-Click App boots up. This benefits your customers in terms of database scalability and ease of management, and reduces the burden of database support for you as a vendor.

    To enable this option, use the checkboxes that are shown in the Vendor Portal related to enablement of managed databases. If you’ve checked at least 1 box, when a user attempts to create your 1-Click App, they’ll receive a prompt like this one:

    dbaas_1_click_offer

    When a user selects the Managed Database option, DigitalOcean handles the creation of the database cluster as well as the user's Droplet. The Droplet will have a DATABASE_URL environment variable configured including a database connection string, such as:

    postgresql://doadmin:<password>@dbaas-db-11111-do-user-1111111-1.b.db.ondigitalocean.com:25060/defaultdb?sslmode=require

    The user’s managed database configuration and credentials will be stored in /root/.digitalocean_dbaas_credentials in the following format.

    db_protocol=
    db_username=
    db_password=
    db_host=
    db_port=
    db_database=
    

    To disable this feature for any new users of your 1-Click App, simply visit the Vendor Portal to edit your 1-Click App, removing the checkboxes next to all database engines. Once you save the edits, customers will no longer be given the option to add a managed database. Note that existing customers who have already deployed a managed database in conjunction with your 1-Click App will not be affected by that removal, and their managed databases will continue to operate.

Build Automation with Packer

Packer is a tool for creating images from a single source configuration. Using this Packer template reduces the entire process of creating, configuring, validating, and snapshotting a build Droplet to a single command:

packer build marketplace-image.json

By doing this, there is a reduced likelihood of having to submit an image multiple times as a result of falling in any of the next steps:

  • Installing OS updates
  • Deleting bash history.
  • Removing log files and SSH keys from the root user
  • Enabling the firewall (i.e. ufw if you use Ubuntu)

This repository is itself a Packer template for a LAMP stack. You can modify this template to use as a starting point for your image. Note that not all of the scripts/files in this repository are strictly necessary, as this aim at covering a broad case of application.

Usage

To run the LAMP stack in this template, you'll need to install Packer and create a DigitalOcean personal access token and set it to the DIGITALOCEAN_TOKEN environment variable. Running packer build marketplace-image.json without any other modifications will create a build Droplet configured with LAMP, clean and verify it, then power it down and snapshot it.

To start adapting this template for your own image, you can customize some variables in marketplace-image.json:

  • apt_packages lists the APT packages to install on the build Droplet.
  • image_name defines the name of the resulting snapshot, which by default is marketplace-snapshot- with a UNIX timestamp appended.

You can also modify these variables at runtime by using the -var flag.

Please see the RStudio Server 1-Click Scripts to see an example of Packer usage.

A successful run would look like this output:

pacha@pop-os:~/github/marketplace-partners$ packer build marketplace-image.json
digitalocean output will be in this color.

==> digitalocean: Creating temporary ssh key for droplet...
==> digitalocean: Creating droplet...
==> digitalocean: Waiting for droplet to become active...
==> digitalocean: Using ssh communicator to connect: 165.227.211.66
==> digitalocean: Waiting for SSH to become available...
==> digitalocean: Connected to SSH!
==> digitalocean: Provisioning with shell script: /tmp/packer-shell581341144
    digitalocean: .............................................................
    digitalocean: status: done
==> digitalocean: Uploading files/etc/ => /etc/
==> digitalocean: Uploading files/var/ => /var/
==> digitalocean: Provisioning with shell script: /tmp/packer-shell079619818
    digitalocean:
    digitalocean: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

    ...

    digitalocean: The following NEW packages will be installed:
    digitalocean:   linux-headers-5.4.0-66 linux-headers-5.4.0-66-generic

    digitalocean: The following packages will be upgraded:
    digitalocean:   accountsservice alsa-ucm-conf apport apt apt-utils base-files bind9-dnsutils

    ... MANY MORE LINES OMITTED HERE ...
    
    digitalocean: Distribution: Ubuntu
    digitalocean: Version: 20.04
    digitalocean:
    digitalocean: [PASS] Supported Operating System Detected: Ubuntu
    digitalocean: [PASS] Supported Release Detected: 20.04
    digitalocean: [PASS] Cloud-init is installed.
    digitalocean: [PASS] Firewall service (ufw) is active
    digitalocean:
    digitalocean: Updating apt package database to check for security updates, this may take a minute...
    digitalocean:
    digitalocean: [PASS] There are no pending security updates for this image.
    digitalocean:
    digitalocean:
    digitalocean: Checking for log files in /var/log
    digitalocean:
    digitalocean: [WARN] un-cleared log file, /var/log/auth.log found
    digitalocean: [WARN] un-cleared log file, /var/log/ufw.log found
    digitalocean:
    digitalocean:
    digitalocean: Checking all user-created accounts...
    digitalocean:
    digitalocean:
    digitalocean: Checking the root account...
    digitalocean: [PASS] User root has no password set.
    digitalocean: [ OK ] User root has no SSH keys present
    digitalocean: [PASS] root's Bash History appears to have been cleared
    digitalocean: [PASS] DigitalOcean Monitoring agent was not found
    digitalocean: [PASS] MongoDB is not installed
    digitalocean:
    digitalocean: ---------------------------------------------------------------------------------------------------
    digitalocean: Scan Complete.
    digitalocean: Some non-critical tests failed.  Please review these items.
    digitalocean: ---------------------------------------------------------------------------------------------------
    digitalocean: 8 Tests PASSED
    digitalocean: 2 WARNINGS
    digitalocean: 0 Tests FAILED
    digitalocean: ---------------------------------------------------------------------------------------------------
    digitalocean: Please review all [WARN] items above and ensure they are intended or resolved.  If you do not have a specific requirement, we recommend resolving these items before image submission
    digitalocean:
==> digitalocean: Gracefully shutting down droplet...
==> digitalocean: Creating snapshot: lemp-20-04-snapshot-1615212919
==> digitalocean: Waiting for snapshot to complete...
==> digitalocean: Destroying droplet...
==> digitalocean: Deleting temporary ssh key...
Build 'digitalocean' finished.

See that this output has two acceptable warnings, but something like having ufw disabled or present SSH keys means we couldn't accept the image.

Configuration Details

By using Packer's DigitalOcean Builder to integrate with the DigitalOcean API, this template fully automates Marketplace image creation.

This template uses Packer's file provisioner to upload complete directories to the Droplet. The contents of files/var/ will be uploaded to /var/. Likewise, the contents of files/etc/ will be uploaded to /etc/. One important thing to note about the file provisioner, from Packer's docs:

The destination directory must already exist. If you need to create it, use a shell provisioner just prior to the file provisioner in order to create the directory. If the destination directory does not exist, the file provisioner may succeed, but it will have undefined results.

This template also uses Packer's shell provisioner to run scripts from the /scripts directory and install APT packages using an inline task.

Learn more about using Packer in the official Packer documentation.

Other Examples

We also use Packer to build some of the Marketplace 1-Click Apps that DigitalOcean maintains. You can see the source code for these scripts in this repo.

Supported Operating Systems

To maintain compatibility with Marketplace tools and processes, we support a limited number of Linux distributions and releases for Marketplace images. These options provide either deb- or rpm-based packaging and will have security patches and updates for a reasonable time period.

We currently support the following OSes:

  • Debian 9 (stretch)
  • Debian 10 (buster)
  • Ubuntu 22.04 (LTS)
  • Ubuntu 20.04 (LTS)
  • Ubuntu 18.04 (LTS)
  • Ubuntu 16.04 (LTS)
  • CentOS 7.x
  • CentOS 8.x
  • CentOS Stream 8
  • CentOS Stream 9
  • AlmaLinux 8.x
  • AlmaLinux 9.x

All supported operating systems are available as base images to build on in the DigitalOcean cloud.

Software Prerequisites

The following software packages are necessary for the initial configuration of new Droplets and to ensure connectivity:

  • cloud-init 0.76 or higher (0.79 or higher recommended)
  • openssh-server (SFTP-enabled configuration recommended)

All of these packages are provided by default in the default DigitalOcean base images.

Image Configuration

Running Commands on First Boot

You can often pre-load much of what your image will need in your build system, but some setup (like setting database passwords or configuration that needs the Droplet's assigned IP address) will need to be run for each new Droplet created from your image.

You can create scripts that run on first boot using cloud-init. Droplets will attempt to run any scripts located in the /var/lib/cloud/scripts/per-instance directory when they're first created. Scripts in that directory are run in alphanumerical order, so we recommend using a number as the beginning of the file names (e.g. 01-example-script.sh).

Make sure you can run the script from the command line successfully and that it has execute permissions.

Running Commands on First Login

Some of your image setup may require information that you can't get automatically, like the domain name to use for a service. You may also need to run interactive third-party scripts, like LetsEncrypt's Certbot.

To run a script on the user's first login, we recommend adding a line to the root .bashrc file that runs the script and adding a line to the script that removes the line from the root .bashrc file.

More specifically, at the end of the script you want to run on first login, add the following line. For consistency, we recommend putting first login scripts in the /opt/your_company_name directory. Make sure the script has execute permissions.

cp -f /etc/skel/.bashrc /root/.bashrc

Then add a line to the end of /root/.bashrc that runs your script by specifying the full path to the script.

When the user first logs in, the system runs .bashrc, which will automatically run your script. The last line of the script overwrites the root .bashrc with the default .bashrc from the /etc/skel directory so the call to run your script no longer exists. Using this method, your script only runs once the first time the user logs in, but the file remains in the filesystem if they need to re-run or reference it later.

Recommendations

  • Use the smallest suitable disk size.

We don't support decreasing the size of a Droplet's disk because it poses data integrity issues. Building your image using the smallest disk size appropriate for your use case lets your users choose from the widest variety of Droplet plans.

  • Do not enable unnecessary DigitalOcean features on your build Droplet.

By not enabling features like monitoring, IPv6, or private networking when you create your build Droplet, you retain more of your distribution's standard configuration, meaning you'll need to do less cleanup before you create the final image.

  • Install software updates from the distribution's repositories before creating your final image.

This secures the system and can save your users time when they create new Droplets from your image.

  • Use official package repositories or well-maintained third-party repositories whenever possible. Packages installed through other means may not provide a mechanism for applying timely security updates.

For official distribution packages, we recommend maintaining the mirrors.digitalocean.com mirrors, which are direct mirrors of the distribution's package archive. These mirrors are provided by default and provide faster downloads because the mirrors are stored within our infrastructure.

  • If you need to provide a password to your user, consider configuring it so that it is randomly generated at boot time and explain to users via your Getting Started instructions how to access the password. Here's an example of how you can generate a high quality, 12 character password on a Linux Droplet, and store it in a file on the Droplet.
gpg --gen-random --armor 2 12 > /root/.secrets.txt
  • Add a message of the day (MOTD), which is text displayed when a user logs into their Droplet. We recommend writing an MOTD which introduces your image's features and points users to its documentation.

You can add an MOTD to your image by creating a text file in /etc/update-motd.d. Naming the file beginning with 99, like 99-image-readme, will display the MOTD as the last text the user sees before the login prompt.

Contributing

We'd love to have your contribution to this project! You can find more details here.

Caveats

Avoid building architecture specific components into your 1-Click App, as your App may be run by DigitalOcean customers across a variety of operating systems and underlying hypervisors. You should use generic machine architecture to ensure consistent use across all infrastructure.

For example, avoid building ruby gems with native extensions as the underlying machine architecture may use flags that don't exist across all hypervisors.

More Repositories

1

nginxconfig.io

⚙️ NGINX config generator on steroids 💉
JavaScript
27,244
star
2

doctl

The official command line interface for the DigitalOcean API.
Go
3,155
star
3

godo

DigitalOcean Go API client
Go
1,328
star
4

go-libvirt

Package libvirt provides a pure Go interface for interacting with Libvirt. Apache 2.0 Licensed.
Go
815
star
5

do_user_scripts

Shell
804
star
6

Kubernetes-Starter-Kit-Developers

Hands-on tutorial and Automation stack for an operations-ready DigitalOcean Kubernetes (DOKS) cluster.
HCL
705
star
7

firebolt

Golang framework for streaming ETL, observability data pipeline, and event processing apps
Go
688
star
8

go-qemu

Go packages to interact with QEMU using the QEMU Machine Protocol (QMP). Apache 2.0 Licensed.
Go
684
star
9

do-agent

Collects system metrics from DigitalOcean Droplets
Go
586
star
10

csi-digitalocean

A Container Storage Interface (CSI) Driver for DigitalOcean Block Storage
Go
565
star
11

clusterlint

A best practices checker for Kubernetes clusters. 🤠
Go
536
star
12

vulcan

Vulcan extends Prometheus adding horizontal scalability and long-term storage
Go
531
star
13

digitalocean-cloud-controller-manager

Kubernetes cloud-controller-manager for DigitalOcean (beta)
Go
517
star
14

hacktoberfest

Hacktoberfest - App to manage the annual open-source challenge, used for the 2019 & 2020 seasons.
Ruby
510
star
15

droplet_kit

DropletKit is the official DigitalOcean API client for Ruby.
Ruby
507
star
16

terraform-provider-digitalocean

Terraform DigitalOcean provider
Go
487
star
17

action-doctl

GitHub Actions for DigitalOcean - doctl
JavaScript
454
star
18

ceph_exporter

Prometheus exporter that scrapes meta information about a ceph cluster.
Go
396
star
19

engineering-code-of-conduct

Code of Conduct for DigitalOcean's Engineering Team
289
star
20

go-openvswitch

Go packages which enable interacting with Open vSwitch and related tools. Apache 2.0 Licensed.
Go
282
star
21

kubernetes-sample-apps

Example DigitalOcean Kubernetes workload with service exposed through a DO load-balancer.
Python
252
star
22

gta

gta: do transitive analysis to find packages whose dependencies have changed
Go
182
star
23

heartbot

A shot of love for your favorite chat client.
CoffeeScript
178
star
24

prometheus-client-c

A Prometheus Client in C
C
154
star
25

marketplace-kubernetes

This repository contains the source code and deployment scripts for Kubernetes-based applications listed in the DigitalOcean Marketplace.
Shell
154
star
26

go-smbios

Package smbios provides detection and access to System Management BIOS (SMBIOS) and Desktop Management Interface (DMI) data and structures. Apache 2.0 Licensed.
Go
152
star
27

kartograph

Kartograph makes it easy to generate and convert JSON. It's intention is to be used for API clients.
Ruby
147
star
28

OpenVPN-Pihole

https://marketplace.digitalocean.com/apps/openvpn-pihole
Shell
146
star
29

captainslog

A Syslog Protocol Parser
Go
136
star
30

resource_kit

Resource Kit provides tools to aid in making API Clients. Such as URL resolving, Request / Response layer, and more.
Ruby
134
star
31

go-workers2

better-go-workers
Go
121
star
32

doks-debug

A Docker image with Kubernetes manifests for investigation and troubleshooting.
Dockerfile
109
star
33

droplet-1-clicks

Packer build scripts for DigitalOcean Marketplace 1-clicks.
Shell
105
star
34

supabase-on-do

HCL
98
star
35

openapi

The OpenAPI v3 specification for DigitalOcean's public API.
JavaScript
97
star
36

container-blueprints

DigitalOcean Kubernetes(DOKS) Solution Blueprints
HCL
92
star
37

sample-dockerfile

⛵ App Platform sample Docker application.
Go
90
star
38

DOKS

Managed Kubernetes designed for simple and cost effective container orchestration.
80
star
39

app_action

Deploy to DigitalOcean Container Registry and App Platform
Go
78
star
40

navigators-guide

Book and code examples that help to build infrastructure on DigitalOcean
Shell
76
star
41

do-operator

The Kubernetes Operator for DigitalOcean
Go
76
star
42

pydo

Official DigitalOcean Python Client based on the DO OpenAPIv3 specification
Python
75
star
43

sample-django

Django sample app for DigitalOcean App Platform
Python
74
star
44

logtalez

logtalez is a minimal command line client (and API) for retrieving log streams from the rsyslog logging daemon over zeromq.
Go
73
star
45

do-markdownit

Markdown-It plugin for the DigitalOcean Community.
JavaScript
71
star
46

databases

66
star
47

sample-nodejs

⛵ App Platform sample Node.js application.
JavaScript
60
star
48

debian-sys-maint-roll-passwd

Script to update password for MySQL user "debian-sys-maint"
Shell
58
star
49

sample-nextjs

⛵ App Platform sample Next.js application.
JavaScript
57
star
50

sample-python

⛵ App Platform sample Python application.
Python
52
star
51

vmtop

Real-time monitoring of KVM/Qemu VMs
Python
52
star
52

kubecon-2022-doks-workshop

HCL
48
star
53

sample-flask

Sample Flask Application to be deployed on DigitalOcean's App Platform
HTML
45
star
54

sample-laravel

⛵ App Platform sample Laravel application.
PHP
43
star
55

pgremapper

CLI tool for manipulating Ceph's upmap exception table.
Go
43
star
56

k8s-staticroute-operator

Create static routes for your k8s nodes using CRDs.
Python
42
star
57

sample-functions-nodejs-qrcode

HTML
39
star
58

tos

DigitalOcean's Terms of Service agreement
37
star
59

sample-monorepo

Sample mono repo app (with multiple components) on the DigitalOcean App Platform.
Go
36
star
60

sample-golang

⛵ App Platform sample Golang application.
Go
36
star
61

droplet-agent

Droplet Agent is the daemon that runs on customer droplets to enable some features such as web console access.
Go
36
star
62

openvswitch_exporter

Command openvswitch_exporter implements a Prometheus exporter for Open vSwitch.
Go
32
star
63

sample-php

⛵ App Platform sample PHP application.
PHP
32
star
64

mastodon-on-kubernetes

Setting up Mastodon on DigitalOcean Kubernetes
HCL
30
star
65

sample-html

⛵ App Platform sample HTML application.
HTML
30
star
66

sample-functions-nodejs-helloworld

JavaScript
30
star
67

sample-functions-python-jokes

Python
30
star
68

flipop

Floating IP Controller for Kubernetes
Go
29
star
69

ansible-collection

DigitalOcean Ansible Collection
Python
28
star
70

sample-functions-golang-helloworld

Go
28
star
71

go-metadata

Go client for the metadata API.
Go
27
star
72

sample-react

⛵ App Platform sample React application.
JavaScript
27
star
73

marketplace-pi-hole-vpn

Pi-hole VPN image for Marketplace with Unbound & Wireguard
Shell
26
star
74

omniauth-digitalocean

DigitalOcean OAuth2 Strategy for OmniAuth
Ruby
26
star
75

github-changelog-generator

A tool to generate changelog entries from GitHub repositories.
Go
25
star
76

sample-functions-python-helloworld

Python
22
star
77

terraform-vault-github-oidc

Terraform module to configure Vault for GitHub OIDC authentication from Action runners.
HCL
22
star
78

sample-push-to-deploy-doks

Push-to-deploy example using DOCR and DOKS
Python
21
star
79

netbox-ip-controller

A Kubernetes controller to import the IP addresses and metadata of pods and services into NetBox.
Go
20
star
80

sample-expressjs

⛵ App Platform sample Express.js application.
19
star
81

terraform-provider-sendgrid

Sendgrid Terraform Provider
Go
19
star
82

sample-nuxtjs

⛵ App Platform sample Nuxt.js application.
Vue
19
star
83

sample-vuejs

⛵ App Platform sample Vue.js application.
Vue
17
star
84

production-ready-kubernetes-workshop

The repository for DigitalOcean's Production Ready Kubernetes Workshop
Python
16
star
85

sample-functions-python-twilio-sms

Sending sms via Twilio
Python
16
star
86

sample-rails

⛵ App Platform sample Ruby on Rails application.
Ruby
15
star
87

sample-functions-php-numberstowords

PHP
15
star
88

sample-functions-php-helloworld

A PHP helloworld sample function for Cloud Functions
PHP
14
star
89

sample-hugo

⛵ App Platform sample Hugo application.
14
star
90

github-pr-resource

Github pull request resource for Concourse
Go
13
star
91

sample-functions-python-sendgrid-email

Sending emails via Sendgrid API
Python
13
star
92

icingaweb2-module-netboximport

Icinga2 Director integration for Netbox
PHP
12
star
93

docker-shipit-engine

Docker image for https://github.com/Shopify/shipit-engine
Ruby
11
star
94

sample-functions-golang-presigned-url

Creating a presigned url for DO's Spaces
Go
10
star
95

digitalocean-ceph-lab

Terraform and Ansible automation to provision and configure a Ceph test environment on DigitalOcean.
HCL
10
star
96

k8s-adoption-journey

Hands-on tutorial for going from Day-1 to production on DigitalOcean Kubernetes. Goes with "Kubernetes Adoption Journey" document.
Python
9
star
97

sample-laravel-api

⛵ App Platform sample Laravel API application.
PHP
9
star
98

gnulib

A mirror of the gnulib portability and testing suite for internal builds that use it as a submodule
C
8
star
99

serverless-jamstack

Contains sample code for a serverless Jamstack tutorial published on docs.digitalocean.com
JavaScript
8
star
100

sample-gatsby

⛵ App Platform sample Gatsby application.
CSS
8
star