• Stars
    star
    218
  • Rank 181,805 (Top 4 %)
  • Language
    Elixir
  • License
    Apache License 2.0
  • Created over 3 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Keep track of your Broadway pipelines from Phoenix LiveDashboard

Broadway Dashboard

Online documentation

BroadwayDashboard is a tool to analyze Broadway pipelines. It provides some insights about performance and errors for your running pipelines.

It works as an additional page for the Phoenix LiveDashboard.

You can inspect pipelines on remote nodes that are not running BroadwayDashboard too. See Distribution for details.

Broadway Dashboard

Integration with Phoenix LiveDashboard

You can add this page to your Phoenix LiveDashboard by adding as a page in the live_dashboard macro at your router file.

live_dashboard "/dashboard",
  additional_pages: [
    broadway: {BroadwayDashboard, pipelines: [MyBroadway]}
  ]

The :pipelines option accept pipeline names (the :name option of your Broadway). By omitting the :pipelines option, BroadwayDashboard will try to autodiscover your pipelines.

live_dashboard "/dashboard",
  additional_pages: [
    broadway: BroadwayDashboard
  ]

Once configured, you will be able to access the BroadwayDashboard at /dashboard/broadway.

Using from the command line with PLDS

It's possible to use Broadway Dashboard without having to install it on your application. PLDS stands for Phoenix LiveDashboard Standalone and it's a CLI version of Phoenix LiveDashboard with some tools pre-installed. One of those tools is Broadway Dashboard.

To install PLDS on your machine, you can run:

$ mix escript.install hex plds

Then connect to your running node with:

$ plds server --connect mynode --open

For more information about the usage, please check the PLDS documentation.

Distribution

Phoenix LiveDashboard works with distribution out of the box, and it's not different with Broadway Dashboard! You can inspect your pipelines that are running on connected nodes.

You can also inspect pipelines from nodes that are not running the same system of your dashboard. This is possible because we "copy" the essential parts of this tool to the remote node when it's not running BroadwayDashboard. We stop the tracking once the node that started it is disconnected.

Installation

Add the following to your mix.exs and run mix deps.get:

def deps do
  [
    {:broadway_dashboard, "~> 0.4.0"}
  ]
end

After that, proceed with instructions described in Integration with Phoenix LiveDashboard above.

Acknowledgment

This project is based on Marlus Saraiva's work from his presentation at ElixirConf 2019.

In that talk he presented a graph showing the work of a Broadway pipeline, which is essentially the same we display in this project. Thank you, Marlus! <3

Development

After cloning this project, you can use the following command to run a development server with a sample pipeline:

$ mix dev

This is going to start a server running Phoenix LiveDashboard.

Use mix test to run the test suite.

License

Copyright 2021 Dashbit

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

More Repositories

1

broadway

Concurrent and multi-stage data ingestion and data processing with Elixir
Elixir
2,414
star
2

flow

Computational parallel flows on top of GenStage
Elixir
1,544
star
3

mox

Mocks and explicit contracts in Elixir
Elixir
1,343
star
4

nimble_parsec

A simple and fast library for text-based parser combinators
Elixir
807
star
5

nimble_csv

A simple and fast CSV parsing and dumping library for Elixir
Elixir
771
star
6

nimble_options

A tiny library for validating and documenting high-level options. πŸ’½
Elixir
505
star
7

nimble_publisher

A minimal filesystem-based publishing engine with Markdown support and code highlighting
Elixir
467
star
8

nimble_totp

A tiny Elixir library for time-based one time passwords (TOTP)
Elixir
396
star
9

nimble_pool

A tiny resource-pool implementation for Elixir
Elixir
344
star
10

bytepack_archive

Archive of bytepack.io
Elixir
328
star
11

broadway_kafka

A Broadway connector for Kafka
Elixir
232
star
12

mix_phx_gen_auth_demo

Example repository for mix phx.gen.auth
Elixir
211
star
13

broadway_rabbitmq

A Broadway producer for RabbitMQ
Elixir
192
star
14

broadway_sqs

A Broadway producer for Amazon SQS
Elixir
92
star
15

broadway_cloud_pub_sub

A Broadway producer for Google Cloud Pub/Sub
Elixir
70
star
16

nimble_ownership

Elixir
55
star
17

broadway_bike_sharing_rabbitmq_example

An example of a Broadway pipeline for a bike sharing app with RabbitMQ and PostgreSQL
Elixir
50
star
18

nimble_strftime

A simple and fast strftime-based datetime formatter
Elixir
37
star
19

unpickler

A library for loading data in the Python's pickle format
Elixir
14
star
20

broadway_website

Landing page for the Elixir Broadway library.
HTML
6
star