• Stars
    star
    753
  • Rank 58,165 (Top 2 %)
  • Language FLUX
  • License
    MIT License
  • Created almost 6 years ago
  • Updated 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Flux is a lightweight scripting language for querying databases (like InfluxDB) and working with data. It's part of InfluxDB 1.7 and 2.0, but can be run independently of those.

Flux - Influx data language

CircleCI

Flux is a lightweight scripting language for querying databases (like InfluxDB) and working with data. It is part of InfluxDB 1.7 and 2.0, but can be run independently of those. This repository contains the language definition and an implementation of the language core.

Specification

A complete specification can be found in SPEC.md. The specification contains many examples to start learning Flux.

Requirements

Building Flux requires the following:

  • Go 1.16 or greater with module support enabled
  • Latest stable version of Rust and Cargo (can be installed with rustup)
  • Clang

Getting Started

Flux is currently available in InfluxDB 1.7 and 2.0, or through the REPL that can be compiled from this repository.

To build Flux, first install the GNU pkg-config utility on your system, then ensure the pkg-config wrapper is also installed.

# First install GNU pkg-config.
# On Debian/Ubuntu
$ sudo apt-get install -y clang pkg-config

# Or on Mac OS X with Homebrew
$ brew install pkg-config

# Next, install the pkg-config wrapper utility
$ go get github.com/influxdata/pkg-config

# Optionally, add the GOBIN directory to your PATH
$ export PATH=${GOPATH}/bin:${PATH}

If GOBIN is in your PATH, ensure that pkg-config is configured correctly by using which -a.

$ which -a pkg-config
/home/user/go/bin/pkg-config
/usr/bin/pkg-config

To compile and use the REPL, use the following command to run the repl with suggestions enabled:

$ go build ./cmd/flux
$ ./flux --enable-suggestions

Alternatively, because the pkg-config wrapper may not work in all projects you may not want to add the wrapper pkg-config to your PATH. In this case you can set PKG_CONFIG and Go will use it. Eg, to build and install to ${GOPATH}/bin using PKG_CONFIG:

$ PKG_CONFIG=/home/user/go/bin/pkg-config go install ./cmd/flux
$ ${GOPATH}/bin/flux repl

From within the REPL, you can run any Flux expression. You can also load a file directly into the REPL by typing @ followed by the filename.

> @my_file_to_load.flux

Basic Syntax

Here are a few examples of the language to get an idea of the syntax.

// This line is a comment

// Support for traditional math operators
1 + 1

// Several data types are built-in
true                     // a boolean true value
1                        // an int
1.0                      // a float
"this is a string"       // a string literal
1h5m                     // a duration of time representing 1 hour and 5 minutes
2018-10-10               // a time starting at midnight for the default timezone on Oct 10th 2018
2018-10-10T10:05:00      // a time at 10:05 AM for the default timezone on Oct 10th 2018
[1,1,2]                  // an array of integers
{foo: "str", bar: false} // an object with two keys and their values

// Values can be assigned to identifiers
x = 5.0
x + 3.0 // 8.0

// Import libraries
import "math"

// Call functions always using keyword arguments
math.pow(x: 5.0, y: 3.0) // 5^3 = 125

// Functions are defined by assigning them to identifiers
add = (a, b) => a + b

// Call add using keyword arguments
add(a: 5, b: 3) // 8

// Functions are polymorphic
add(a: 5.5, b: 2.5) // 8.0

// And strongly typed
add(a: 5, b: 2.5) // type error

// Access data from a database and store it as an identifier
// This is only possible within the influxdb repl (at the moment).
import "influxdata/influxdb"
data = influxdb.from(bucket:"telegraf/autogen")

// When running inside of influxdb, the import isn't needed.
data = from(bucket:"telegraf/autogen")

// Chain more transformation functions to further specify the desired data
cpu = data
    // only get the last 5m of data
    |> range(start: -5m)
    // only get the "usage_user" data from the _measurement "cpu"
    |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_user")

// Return the data to the client
cpu |> yield()

// Group an aggregate along different dimensions
cpu
    // organize data into groups by host and region
    |> group(columns:["host","region"])
    // compute the mean of each group
    |> mean()
    // yield this result to the client
    |> yield()

// Window an aggregate over time
cpu
    // organize data into groups of 1 minute
    // compute the mean of each group
    |> aggregateWindow(every: 1m, fn: mean)
    // yield this result to the client
    |> yield()

// Gather different data
mem = data
    // only get the last 5m of data
    |> range(start: -5m)
    // only get the "used_percent" data from the _measurement "mem"
    |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent")


// Join data to create wider tables and map a function over the result
join(tables: {cpu:cpu, mem:mem}, on:["_time", "host"])
    // compute the ratio of cpu usage to mem used_percent
    |> map(fn:(r) => {_time: r._time, _value: r._value_cpu / r._value_mem)
    // again yield this result to the client
    |> yield()

The above examples give only a taste of what is possible with Flux. See the complete documentation for more complete examples and instructions for how to use Flux with InfluxDB 2.0.

Contributing

Flux welcomes contributions to the language and the runtime.

If you are interested in contributing, please read the contributing guide for more information.

Development basics

If you modify any Rust code, you will need to force Go to rebuild the library.

$ go generate ./libflux/go/libflux

If you create or change any Flux functions, you will need to rebuild the stdlib and inform Go that it must rebuild libflux:

$ go generate ./stdlib ./libflux/go/libflux

Your new Flux's code should be formatted to coexist nicely with the existing codebase with go fmt. For example, if you add code to stdlib/universe:

$ go fmt ./stdlib/universe/

Don't forget to add your tests and make sure they work. Here is an example showing how to run the tests for the stdlib/universe package:

$ go test ./stdlib/universe/

If you modify the flatbuffer files you need to run make generate to update the generated bindings. To ensure that you have the correct version of flatc you may use ./install_flatc.sh.

$ ./install_flatc.sh
$ make generate

More Repositories

1

influxdb

Scalable datastore for metrics, events, and real-time analytics
Rust
27,320
star
2

telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
Go
13,798
star
3

kapacitor

Open source framework for processing, monitoring, and alerting on time series data
Go
2,279
star
4

influxdb_iox

Pronounced (influxdb eye-ox), short for iron oxide. This is the new core of InfluxDB written in Rust on top of Apache Arrow.
Rust
1,803
star
5

influxdb-python

Python client for InfluxDB
Python
1,678
star
6

chronograf

Open source monitoring and visualization UI for the TICK stack
TypeScript
1,480
star
7

influxdb-java

Java client for InfluxDB
Java
1,156
star
8

influxdb-relay

Service to replicate InfluxDB data for high availability
Python
830
star
9

influxdb-client-python

InfluxDB 2.0 python client
Python
664
star
10

influxdb-client-go

InfluxDB 2 Go Client
Go
572
star
11

sandbox

A sandbox for the full TICK stack
Shell
475
star
12

go-syslog

Blazing fast syslog parser
Go
468
star
13

influxdb-php

influxdb-php: A PHP Client for InfluxDB, a time series database
PHP
430
star
14

influxdb-client-java

InfluxDB 2 JVM Based Clients
Java
412
star
15

influxdb-client-csharp

InfluxDB 2.x C# Client
C#
337
star
16

community-templates

InfluxDB Community Templates: Quickly collect & analyze time series data from a range of sources: Kubernetes, MySQL, Postgres, AWS, Nginx, Jenkins, and more.
Python
332
star
17

influxdb-client-js

InfluxDB 2.0 JavaScript client
TypeScript
316
star
18

influxdata-docker

Official docker images for the influxdata stack
Shell
314
star
19

influxdb-comparisons

Code for comparison write ups of InfluxDB and other solutions
Go
306
star
20

rskafka

A minimal Rust client for Apache Kafka
Rust
282
star
21

docs.influxdata.com-ARCHIVE

ARCHIVE - 1.x docs for InfluxData
Less
253
star
22

helm-charts

Official Helm Chart Repository for InfluxData Applications
Mustache
212
star
23

influxdb-rails

Ruby on Rails bindings to automatically write metrics into InfluxDB
Ruby
205
star
24

influxdb-csharp

A .NET library for efficiently sending points to InfluxDB 1.x
C#
198
star
25

influxdb1-client

The old clientv2 for InfluxDB 1.x
Go
187
star
26

giraffe

A foundation for visualizations in the InfluxDB UI
TypeScript
178
star
27

influxql

Package influxql implements a parser for the InfluxDB query language.
Go
163
star
28

influxdb-client-php

InfluxDB (v2+) Client Library for PHP
PHP
140
star
29

tdigest

An implementation of Ted Dunning's t-digest in Go.
Go
126
star
30

influx-stress

New tool for generating artificial load on InfluxDB
Go
118
star
31

tick-charts

A repository for Helm Charts for the full TICK Stack
Smarty
90
star
32

ui

UI for InfluxDB
TypeScript
86
star
33

telegraf-operator

telegraf-operator helps monitor application on Kubernetes with Telegraf
Go
79
star
34

pbjson

Auto-generate serde implementations for prost types
Rust
79
star
35

inch

An InfluxDB benchmarking tool.
Go
78
star
36

influxdata-operator

A k8s operator for InfluxDB
Go
76
star
37

docs-v2

InfluxData Documentation that covers InfluxDB Cloud, InfluxDB OSS 2.x, InfluxDB OSS 1.x, InfluxDB Enterprise, Telegraf, Chronograf, Kapacitor, and Flux.
SCSS
66
star
38

wirey

Manage local wireguard interfaces in a distributed system
Go
62
star
39

influxdb-go

61
star
40

influx-cli

CLI for managing resources in InfluxDB v2
Go
58
star
41

terraform-aws-influx

Reusable infrastructure modules for running TICK stack on AWS
HCL
50
star
42

grade

Track Go benchmark performance over time by storing results in InfluxDB
Go
43
star
43

influxdb-r

R library for InfluxDB
R
43
star
44

influxdb-observability

Go
43
star
45

clockface

UI Kit for building Chronograf
TypeScript
43
star
46

nginx-influxdb-module

C
40
star
47

influxdb2-sample-data

Sample data for InfluxDB 2.0
JavaScript
40
star
48

influxdb-client-ruby

InfluxDB 2.0 Ruby Client
Ruby
40
star
49

tensorflow-influxdb

Jupyter Notebook
34
star
50

nifi-influxdb-bundle

InfluxDB Processors For Apache NiFi
Java
33
star
51

line-protocol

Go
33
star
52

whisper-migrator

A tool for migrating data from Graphite Whisper files to InfluxDB TSM files (version 0.10.0).
Go
33
star
53

iot-center-flutter

InlfuxDB 2.0 dart client flutter demo
Dart
31
star
54

kube-influxdb

Configuration to monitor Kubernetes with the TICK stack
Shell
30
star
55

k8s-kapacitor-autoscale

Demonstration of using Kapacitor to autoscale a k8s deployment
Go
30
star
56

terraform-aws-influxdb

Deploys InfluxDB Enterprise to AWS
HCL
29
star
57

catslack

Shell -> Slack the easy way
Go
28
star
58

influxdb-operator

The Kubernetes operator for InfluxDB and the TICK stack.
Go
27
star
59

flux-lsp

Implementation of Language Server Protocol for the flux language
Rust
26
star
60

influxdb-client-swift

InfluxDB (v2+) Client Library for Swift
Swift
26
star
61

flightsql-dbapi

DB API 2 interface for Flight SQL with SQLAlchemy extras.
Python
26
star
62

influxdb-c

C
25
star
63

influxdb-client-dart

InfluxDB (v2+) Client Library for Dart and Flutter
Dart
24
star
64

ansible-chrony

A role to manage chrony on Linux systems
Ruby
24
star
65

kapacitor-course

24
star
66

vsflux

Flux language extension for VSCode
TypeScript
24
star
67

grafana-flightsql-datasource

Grafana plugin for Flight SQL APIs.
TypeScript
24
star
68

influxdb-scala

Scala client for InfluxDB
Scala
22
star
69

cron

A fast, zero-allocation cron parser in ragel and golang
Go
21
star
70

influxdb-plugin-fluent

A buffered output plugin for Fluentd and InfluxDB 2
Ruby
21
star
71

terraform-google-influx

Reusable infrastructure modules for running TICK stack on GCP
Shell
20
star
72

influxdb3_core

InfluxData's core functionality for InfluxDB Edge and IOx
Rust
18
star
73

openapi

An OpenAPI specification for influx (cloud/oss) apis.
Shell
17
star
74

influxdb-university

InfluxDB University
Python
16
star
75

influxdb-client-r

InfluxDB (v2+) Client R Package
R
14
star
76

cd-gitops-reference-architecture

Details of the CD/GitOps architecture in use at InfluxData
Shell
13
star
77

kafka-connect-influxdb

InfluxDB 2 Connector for Kafka
Scala
13
star
78

oats

An OpenAPI to TypeScript generator.
TypeScript
12
star
79

awesome

SCSS
12
star
80

windows-packager

Create a windows installer
Shell
12
star
81

iot-api-ui

Common React UI for iot-api-<js, python, etc.> example apps designed for InfluxDB client library tutorials.
TypeScript
12
star
82

promql

Go
11
star
83

yarpc

Yet Another RPC for Go
Go
11
star
84

iot-api-python

Python
11
star
85

influxdb-gds-connector

Google Data Studio Connector for InfluxDB.
JavaScript
11
star
86

object_store_rs

Rust
10
star
87

ansible-influxdb-enterprise

Ansible role for deploying InfluxDB Enterprise.
10
star
88

influxdb-sample-data

Sample time series data used to test InfluxDB
9
star
89

ingen

ingen is a tool for directly generating TSM data
Go
8
star
90

ansible-kapacitor

Official Kapacitor Ansible Role for Linux
Jinja
7
star
91

wlog

Simple log level based Go logger.
Go
7
star
92

iot-api-js

An example IoT app built with NextJS (NodeJS + React) and the InfluxDB API client library for Javascript.
JavaScript
7
star
93

influxdb-iox-client-go

InfluxDB/IOx Client for Go
Go
7
star
94

k8s-jsonnet-libs

Jsonnet Libs repo - mostly generated with jsonnet-libs/k8s project
Jsonnet
7
star
95

google-deployment-manager-influxdb-enterprise

GCP Deployment Manager templates for InfluxDB Enterprise.
HTML
6
star
96

jaeger-influxdb

Go
6
star
97

influxdb-action

A GitHub action for setting up and configuring InfluxDB and the InfluxDB Cloud CLI
Shell
6
star
98

influxdb-fsharp

A F# client library for InfluxDB, a time series database http://influxdb.com
F#
6
star
99

qprof

A tool for profiling the performance of InfluxQL queries
Go
6
star
100

influxdb-nodejs

InfluxDB client library for NodeJS
5
star