• Stars
    star
    184
  • Rank 209,187 (Top 5 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Lighthouse

Lighthouse is a lightweight ChatOps based webhook handler which can trigger Jenkins X Pipelines, Tekton Pipelines or Jenkins Jobs based on webhooks from multiple git providers such as GitHub, GitHub Enterprise, BitBucket Server and GitLab.

Installing

Lighthouse is bundled and released as Helm Chart. You find the install instruction in the Chart's README.

Depending on the pipeline engine you want to use, you can find more detailed instructions in one of the following documents:

Background

Lighthouse derived originally from Prow and started with a copy of its essential code.

Currently, Lighthouse supports the standard Prow plugins and handles push webhooks to branches to then trigger a pipeline execution on the agent of your choice.

Lighthouse uses the same config.yaml and plugins.yaml for configuration than Prow.

Comparisons to Prow

Lighthouse reuses the Prow plugin source code and a bunch of plugins from Prow

Its got a few differences though:

  • rather than being GitHub specific Lighthouse uses jenkins-x/go-scm so it can support any Git provider
  • Lighthouse does not use a ProwJob CRD; instead, it has its own LighthouseJob CRD.

Porting Prow commands

If there are any prow commands you want which we've not yet ported over, it is relatively easy to port Prow plugins.

We've reused the prow plugin code and configuration code; so it is mostly a case of switching imports of k8s.io/test-infra/prow to github.com/jenkins-x/lighthouse/pkg/prow, then modifying the GitHub client structs from, say, github.PullRequest to scm.PullRequest.

Most of the GitHub structs map 1-1 to the jenkins-x/go-scm equivalents (e.g. Issue, Commit, PullRequest). However, the go-scm API does tend to return slices to pointers to resources by default. There are some naming differences in different parts of the API as well. For example, compare the githubClient API for Prow lgtm versus the Lighthouse lgtm.

Development

Building

To build the code, fork and clone this git repository, then type:

make build

make build will build all relevant Lighthouse binaries natively for your OS which you then can run locally. For example, to run the webhook controller, you would type:

./bin/webhooks

To see which other Make rules are available, run:

make help

Environment variables

While Prow only supports GitHub as SCM provider, Lighthouse supports several Git SCM providers. Lighthouse achieves the abstraction over the SCM provider using the go-scm library. To configure your SCM, go-scm uses the following environment variables :

Name Description
GIT_KIND the kind of git server: github, gitlab, bitbucket, gitea, stash
GIT_SERVER the URL of the server if not using the public hosted git providers: https://github.com, https://bitbucket.org or https://gitlab.com
GIT_USER the git user (bot name) to use on git operations
GIT_TOKEN the git token to perform operations on git (add comments, labels etc.)
HMAC_TOKEN the token sent from the git provider in webhooks

Testing

To run the unit tests, type:

make test

For development purposes, it is also nice to start an instance of the binary you want to work. Provided you have a connection to a cluster with Lighthouse installed, the locally started controller will join the cluster, and you can test your development changes directly in-cluster.

For example to run the webhook controller locally:

make build
GIT_TOKEN=<git-token> ./bin/webhooks -namespace <namespace> -bot-name <git-bot-user>

In the case of the webhook controller, you can also test webhook deliveries locally using a ngrok tunnel. Install ngrok and start a new tunnel:

$ ngrok http 8080
ngrok by @inconshreveable                                                                                                                                                                     (Ctrl+C to quit)

Session Status                online
Account                       ***
Version                       2.3.35
Region                        United States (us)
Web Interface                 http://127.0.0.1:4040
Forwarding                    http://e289dd1e1245.ngrok.io -> http://localhost:8080
Forwarding                    https://e289dd1e1245.ngrok.io -> http://localhost:8080

Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00

Now you can use your ngrok URL to register a webhook handler with your Git provider.

NOTE Remember to append /hook to the generated ngrok URL. In the case of the above example http://e289dd1e1245.ngrok.io/hook

Any events that happen on your Git provider are now sent to your local webhook instance.

Debugging Lighthouse

You can setup a remote debugger for Lighthouse using delve via:

dlv --listen=:2345 --headless=true --api-version=2 exec ./bin/lighthouse -- $*  

You can then debug from your Go-based IDE (e.g. GoLand / IDEA / VS Code).

Debugging webhooks

If you want to debug lighthouse locally from webhooks in your cluster there are a couple of tools that could help:

Localizer

If you install localizer (see the blog for more detail you can easily debug webhooks on your cluster.

  • first run localizer:
sudo localizer

Then run/debug lighthouse locally.

e.g. in your IDE run the cmd/webhooks/main.go (passing --namespace jx as program arguments)

Then to get the webhooks to trigger your local process:

localizer expose jx/hook --map 80:8080

when you have finished debugging, return things to normal via:

localizer expose jx/hook --stop

Telepresence

You can replace the running version in your cluster with the one running locally using telepresence.
First install the telepresence cli on your device then the traffic-manager into your cluster For webhooks, just run:

telepresence intercept hook --namespace=jx --port 80 --env-file=/tmp/webhooks-env

in another terminal:

export $(cat /tmp/webhooks-env | xargs)
dlv --listen=:2345 --headless=true --api-version=2 exec ./bin/webhooks -- --namespace=jx

You can do the same for any other deployment (keeper, foghorn...), just make sur to check the command args used for it an set them instead of --namespace=jx.

to stop intercepting:

telepresence leave hook-jx # hook-jx is the name of the intercept

Using a local go-scm

If you are hacking on support for a specific Git provider, you may find yourself working on the Lighthouse code and the jenkins-x/go-scm code together. Go modules lets you easily swap out the version of a dependency with a local copy of the code; so you can edit code in Lighthouse and jenkins-x/go-scm at the same time.

Just add this line to the end of your go.mod file:

replace github.com/jenkins-x/go-scm => /workspace/go/src/github.com/jenkins-x/go-scm

Using the exact path to where you cloned jenkins-x/go-scm

More Repositories

1

jx

Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Requests using Cloud Native pipelines from Tekton
Go
4,398
star
2

jenkins-x-platform

Helm umbrella repo containing the helm charts for the Jenkins-X platform on Kubernetes
Makefile
296
star
3

jx-docs

Source code used to build the Jenkins X documentation and website
HTML
85
star
4

jx-cli

a simple small new modular CLI for Jenkins X v3
Go
74
star
5

exposecontroller

Automatically expose services creating ingress rules, openshift routes or modifying services to use kubernetes nodePort or loadBalancer service types
Go
69
star
6

terraform-aws-eks-jx

A Terraform module for creating Jenkins X infrastructure on AWS
HCL
63
star
7

bucketrepo

An artifacts repository using cloud storage (a lightweight replacement for Nexus)
Go
57
star
8

jx-pipelines-visualizer

an open source pipeline visualiser for Jenkins X
Go
41
star
9

terraform-google-jx

A Terraform module for creating Jenkins X infrastructure on Google Cloud
HCL
38
star
10

jenkins-x-serverless

Go
27
star
11

jx-tutorial

Shell
27
star
12

jx-git-operator

a simple operator to trigger a Job on a git commit in a repository
Go
26
star
13

gsm-controller

Go
25
star
14

jx3-pipeline-catalog

the default tekton catalog for Jenkins X 3.x
Makefile
25
star
15

vscode-jx-tools

Jenkins X extension for Visual Studio Code
TypeScript
22
star
16

jenkins-x-image

Customised Dockerfile and an opinionated list of plugins used on the Jenkins X platform.
Shell
22
star
17

jenkins-x-builders

Dockerfile
20
star
18

jenkins-x-versions

contains the consistent set of versions of packages and helm charts for use with Jenkins X and its associated Apps
Shell
20
star
19

draft-packs

Note these build packs are moving to https://github.com/jenkins-x-buildpacks
Groovy
18
star
20

terraform-provider-jxadmin

A Jenkins X provider for terraform
Go
14
star
21

jx3-versions

Shell
14
star
22

jx-convert-jenkinsfile

A plugin for [Jenkins X](https://jenkins-x.io) to assist in converting from the legacy `Jenkinsfile`-based pipelines to the modern `jenkins-x.yml`-based pipelines
Go
14
star
23

enhancements

Repository containing design proposals for Jenkins X enhancements
Shell
12
star
24

jx-helpers

a library of common helper utilities for working with commands and git
Go
11
star
25

jx-api

Go
10
star
26

cloud-environments

Makefile
10
star
27

homebrew-jx

This is the homebrew repo for installing the jx client for Jenkins X
Ruby
9
star
28

dev-env

A development environment built upon docker
Shell
8
star
29

builder-go

Dockerfile
8
star
30

builder-terraform

A Jenkins X builder for Terraform
Shell
8
star
31

jenkins-x-builders-base

Shell
7
star
32

jx-ui

Svelte
7
star
33

terraform-azurerm-jx

A Terraform module for creating Jenkins X infrastructure on Azure
HCL
7
star
34

jenkins-x-website

the google pages repository for the Jenkins X website at https://jenkins-x.io/. If you want to make changes please use the source repository: jx-docs
HTML
7
star
35

bdd-jx

BDD tests using ginkgo for Jenkins X
Go
6
star
36

jx-community

Welcome to the Jenkins X community!
6
star
37

builder-base

Shell
5
star
38

cd-indicators

Continuous Delivery Indicators for Jenkins X
Go
5
star
39

godog-jx

BDD tests using godog for Jenkins X
Go
5
star
40

jx3-gitops-template

the template git repository contains common source which is pulled into other git repositories via kpt
Shell
4
star
41

jx3-eagle

Shell
4
star
42

lighthouse-client

client library for working with lighthouse configuration and custom resources
Go
4
star
43

prow-config

Configuration of prow and it's plugins for Jenkins X's own pipelines
3
star
44

jenkins-x-openshift-image

the Docker image for Jenkins when running on OpenShift
Dockerfile
3
star
45

jx3-lts-versions

LTS Version Stream for Jenkins X 3, released on a monthly cadence
Shell
3
star
46

prow-pipeline-controller

A pipeline controller for Prow
Go
3
star
47

dev-env-base

Base container images for the dev-env (github.com/jenkins-x/dev-env)
Makefile
3
star
48

default-environment-charts

The default git repository used when creating new GitOps based Environments
Makefile
3
star
49

jenkins-x-builders-base-image

Swift
3
star
50

helm-unit-tester

a small golang library for unit testing helm charts from go code
Go
3
star
51

builder-python

Shell
2
star
52

jx-logging

Go
2
star
53

builder-maven

Builder image used in Jenkins-X pipelines to build maven based projects
Shell
2
star
54

pipeline-controller

Go
2
star
55

jx-java-client

A pure Java client for working with Jenkins X
Java
2
star
56

jx-intellj-plugin

Jenkins X IntelliJ plugin
Java
2
star
57

lighthouse-telemetry-plugin

a lighthouse plugin for telemetry
Go
2
star
58

jx-cdevents-adapter

CDEvents adapter for Jenkins X WIP
Go
2
star
59

hugo-extended-docker

Docker image for building sites that need Hugo extended
Dockerfile
2
star
60

builder-maven-node-yarn

A builder for maven + node + yarn such as for JHipster
Shell
2
star
61

steering

The Jenkins X Steering Committee (currently in bootstrap phase)
2
star
62

jx-cloud-provisioners

Repository to store plans / scripts for different technologies that will automate the creation of cloud resources for different Cloud Providers
HCL
2
star
63

jx-go-maven-base-image

base image for go + java + maven
Dockerfile
1
star
64

organisation-jenkins-x-infra

HCL
1
star
65

issues

Issue tracker for Jenkins X v3
1
star
66

jenkins-x-schemas

JSON Schemas for the Jenkins X Custom Resource Definitions
Go
1
star
67

jxr-packs-kubernetes

Makefile
1
star
68

go-chaos

a simple quickstart used for chaos testing
Go
1
star
69

lighthouse-config

Configuration library for Lighthouse
Go
1
star
70

jenkins-x-statistics

Groovy
1
star
71

lighthouse-jx-controller

DEPRECATED: Lighthouse controller for executing and reporting on Jenkins X pipelines
Go
1
star
72

jx-kube-client

Helper library to create a kubernetes client
Go
1
star
73

jenkins-x-serverless-filerunner

Contains the custom war file used to launch the serverless containers as part of jenkins-x-serverless
Dockerfile
1
star
74

jx-ts-client

TypeScript
1
star
75

jenkins-x-arm-support

a project to track progress on supporting ARM platforms with Jenkins X
1
star
76

jenkins-x-builders-ml

Machine learning builders for Jenkins X
Dockerfile
1
star
77

secrets-example

1
star
78

builder-gradle

a build pod for using gradle
Dockerfile
1
star
79

metrics

helm chart enabling capturing cluster level metrics and visualisation
Makefile
1
star
80

jx-cli-base-image

base image for jx-cli https://github.com/jenkins-x/jx-cli
Dockerfile
1
star
81

terraform-jx-health

Terraform module to install base set of helm charts for Jenkins X
HCL
1
star
82

builder-nodejs

Shell
1
star
83

pipeline-events-addon

Jenkins X addon that enables the capturing of pipeline event activities
Makefile
1
star
84

tekton-dashboard-helm-chart

Makefile
1
star