Flowpipe is a cloud scripting engine. It provides automation and workflow to connect your clouds to the people, systems and data that matter.
Connect people and tools. Connect your cloud data to people and systems using email, chat & APIs. Workflow steps can even run containers, custom functions, and more.
Orchestrate your cloud. Build simple steps into complex workflows. Run and test locally. Compose solutions across clouds using open source mods.
Respond to events. Run workflows manually or on a schedule. Trigger pipelines from webhooks or changes in data.
Code, not clicks. Build and deploy DevOps workflows like infrastructure. Code in HCL and deploy from version control.
Demo Time!
Getting Started
The downloads page shows you how but tl;dr:
Linux or WSL
sudo /bin/sh -c "$(curl -fsSL https://flowpipe.io/install/flowpipe.sh)"
MacOS
brew tap turbot/tap
brew install flowpipe
Now, create and run your first pipeline β
Libraries and samples
Flowpipe library mods are available for services including AWS, Azure, GCP, GitHub, Jira, Okta, PagerDuty, SendGrid, Slack, Teams, Zendesk ... and many more!
Check out Flowpipe samples for ready-to-run samples that use various library mods.
Developing
Developing Flowpipe
Prerequisites:
- Golang Version 1.21 or higher.
Clone github.com/flowpipe
and github.com/turbot/pipe-fittings
repositories:
git clone [email protected]:turbot/flowpipe
git clone [email protected]:turbot/pipe-fittings
cd flowpipe
Build will build flowpipe binary in the current directory:
make
Check the version:
./flowpipe --version
Flowpipe v0.0.1-local.1
Flowpipe local version will always be v0.0.1-local.1
. The real version is generated during the release process.
Try it!
./flowpipe pipeline list --mod-location ./internal/es/estest/test_suite_mod/
MOD NAME DESCRIPTION
mod.mod_depend_a mod_depend_a.pipeline.echo_one_depend_a
mod.test_suite_mod test_suite_mod.pipeline.any_param
mod.test_suite_mod test_suite_mod.pipeline.bad_email_with_expr
mod.test_suite_mod test_suite_mod.pipeline.bad_http_ignored Ignored bad HTTP step.
mod.test_suite_mod test_suite_mod.pipeline.bad_http_not_ignored Pipeline with a HTTP step that will fail. Error is not ignored.
</snip>
Now run a simple pipeline:
./flowpipe pipeline run --mod-location ./internal/es/estest/test_suite_mod/ simple
[flowpipe] Execution ID: exec_clsm62ko47mjp5f74730
[simple] Starting pipeline
[simple.echo] Starting transform
[simple.echo] Output echo_1 = echo 1
[simple.echo] Output echo_2 = echo 2
[simple.echo] Complete 2ms
[simple] Output val = Hello World
[simple] Complete 12ms exec_clsm62ko47mjp5f74730
That's it! You're ready to start developing.
There are other third party tools that are required for the full development suite. These are not required for basic development. To make development easy, we have built a DevContainer that has all the required tools installed. See the Developer Setup section for more details.
Developer Setup
Open Source & Contributing
This repository is published under the AGPL 3.0 license. Please see our code of conduct. Contributors must sign our Contributor License Agreement as part of their first pull request. We look forward to collaborating with you!
Flowpipe is a product produced from this open source software, exclusively by Turbot HQ, Inc. It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.
Get Involved
Want to help but don't know where to start? Pick up one of the help wanted
issues: