eiiches/jackson-jq eiiches/jackson-jq

  • Stars
    star
    279
  • Rank 147,967 (Top 3 %)
  • Language
    Java
  • License
    Other
  • Created about 9 years ago
  • Updated about 2 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
eiiches/jackson-jq
github

eiiches

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

jq for Jackson Java JSON Processor

jackson-jq

Pure Java jq Implementation for Jackson JSON Processor

GitHub Actions

Usage

First, you need Java 8 or later.

If you use Maven, add the following snippet to the <dependencies> section of your POM. For instructions for other build tools (Gradle, etc.), visit jackson-jq on search.maven.org.

<dependency>
	<groupId>net.thisptr</groupId>
	<artifactId>jackson-jq</artifactId>
	<version>1.0.0-preview.20230409</version>
</dependency>

See jackson-jq/src/test/java/examples/Usage.java for the API usage.

Using a jackson-jq command line tool

To test a query quickly, we provide jackson-jq CLI.

Please note that jackson-jq is a Java library and the CLI is provided solely for debugging/testing purpose (and not for production). The command-line options might change without notice.

$ curl -LO https://repo1.maven.org/maven2/net/thisptr/jackson-jq-cli/1.0.0-preview.20230409/jackson-jq-cli-1.0.0-preview.20230409.jar

$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --help
usage: jackson-jq [OPTIONS...] QUERY
 -c,--compact      compact instead of pretty-printed output
 -h,--help         print this message
    --jq <arg>     specify jq version
 -n,--null-input   use `null` as the single input value
 -r,--raw          output raw strings, not JSON texts

$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar '.foo'
{"foo": 42}
42

To test a query with a specific jq version,

$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --jq 1.5 'join("-")'
["1", 2]
jq: error: string ("-") and number (2) cannot be added

$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --jq 1.6 'join("-")' # jq-1.6 can join any values, not only strings
["1", 2]
"1-2"

Homebrew (or Linuxbrew) users can alternatively run brew tap eiiches/jackson-jq && brew install jackson-jq to install the CLI. jackson-jq will be available on your $PATH.

Branches and versioning

There are currently two development branches.

  • develop/1.x: This branch (you are viewing), which is currently under development for the future 1.0 release. You can find preview releases at Releases page (tags: 1.0.0-preview.yyyyMMdd). Although the API is not stable yet, I recommend new users to use these releases insetad of 0.x versions, because these releases have more features, better compatibility, and better performance.
  • develop/0.x: The development branch for 0.x versions. Features that need breaking API changes will no longer be added. Go to Releases and find the latest 0.x.y version.

PRs can be sent to any of the develop/* branches. The patch will be ported to the other branch(es) if necessary.

We use Semantic Versioning 2.0.0 for Java API versioning, 1.0.0 onwards. A jq behavior fix (even if it may possibly affect users) will not be considered a major change if the fix is to make the bahavior compatible with ./jq; these kind of incompatible changes are documented in the release note.

If you get different results between ./jq and jackson-jq, please file an issue. That is a bug on jackson-jq side.

Implementation Status

jackson-jq aims to be a compatible jq implementation. However, not every feature is available; some are intentionally omitted because thay are not relevant as a Java library; some may be incomplete, have bugs or are yet to be implemented.

List of Features

Click to see the list

This table illustrates which features (picked from jq-1.5 manual) are supported and which are not in jackson-jq. We try to keep this list accurate and up to date. If you find something is missing or wrong, please file an issue.

Language Features / Functions jackson-jq
Basic filters ○
    • . ○
    • .foo, .foo.bar ○
    • .foo? ○
    • .[<string>], .[2], .[10:15] ○
    • .[] ○
    • .[]? ○
    • , ○
    • ǀ ○
Types and Values ○
    • Array construction - [] ○
    • Objects - {} ○*4
Builtin operators and functions ○
    • Addition - + ○
    • Subtraction - - ○
    • Multiplication, division, modulo - *, /, and % ○*5
    • length ○
    • keys, keys_unsorted ○
    • has(key) ○
    • in ○
    • path(path_expression) ○*7
    • del(path_expression) ○
    • to_entries, from_entries, with_entries ○
    • select(boolean_expression) ○
    • arrays, objects, iterables, booleans, numbers, normals, finites, strings, nulls, values, scalars ○
    • empty ○
    • error(message) ○
    • $__loc__ ×
    • map(x), map_values(x) ○
    • paths, paths(node_filter), leaf_paths ○
    • add ○
    • any, any(condition), any(generator; condition) ○
    • all, all(condition), all(generator; condition) ○
    • flatten, flatten(depth) ○
    • range(upto), range(from;upto) range(from;upto;by) ○
    • floor ○
    • sqrt ○
    • tonumber ○
    • tostring ○
    • type ○
    • infinite, nan, isinfinite, isnan, isfinite, isnormal ○
    • sort, sort_by(path_expression) ○
    • group_by(path_expression) ○
    • min, max, min_by(path_exp), max_by(path_exp) ○
    • unique, unique_by(path_exp) ○
    • reverse ○
    • contains(element) ○
    • indices(s) ○*9
    • index(s), rindex(s) ○
    • inside ○
    • startswith(str) ○
    • endswith(str) ○
    • combinations, combinations(n) ○
    • ltrimstr(str) ○
    • rtrimstr(str) ○
    • explode ○
    • implode ○
    • split ○
    • join(str) ○
    • ascii_downcase, ascii_upcase ○
    • while(cond; update) ○
    • until(cond; next) ○
    • recurse(f), recurse, recurse(f; condition), recurse_down ○
    • .. ○
    • env ○*6
    • transpose ○
    • bsearch(x) ×
    • String interpolation - \(foo) ○
    • Convert to/from JSON ○
    • Format strings and escaping ○
    • Dates ×
Conditionals and Comparisons ○
    • ==, != ○
    • if-then-else ○
    • >, >=, <=, < ○
    • and/or/not ○
    • Alternative operator - // ○
    • try-catch ○*1
    • Breaking out of control structures ○*2
    • ? operator ○
Regular expressions (PCRE) ○
    • test(val), test(regex; flags) ○
    • match(val), match(regex; flags) ○
    • capture(val), capture(regex; flags) ○
    • scan(regex), scan(regex; flags) ○
    • split(regex; flags) ○
    • splits(regex), splits(regex; flags) ○
    • sub(regex; tostring) sub(regex; string; flags) ○
    • gsub(regex; string), gsub(regex; string; flags) ○
Advanced features ○
    • Variables ○*11
    • Destructuring Alternative Operator: ?// ✕ (#44)
    • Defining Functions ○*3
    • Reduce ○
    • limit(n; exp) ○
    • first(expr), last(expr), nth(n; expr) ○
    • first, last, nth(n) ○
    • foreach ○
    • Recursion ○
    • Generators and iterators ○
Math △
I/O N/A
    • input N/A
    • inputs N/A
    • debug N/A
    • input_filename N/A
    • input_line_number N/A
Streaming N/A
    • truncate_stream(stream_expression) N/A
    • fromstream(stream_expression) N/A
    • tostream N/A
Assignment ○
    • = ○
    • ǀ= ○*8
    • +=, -=, *=, /=, %=, //= ○
    • Complex assignments ○
Modules △
    • import RelativePathString as NAME [<metadata>]; ○
    • include RelativePathString [<metadata>]; ○
    • import RelativePathString as $NAME [<metadata>]; ○
    • module <metadata>; ○
    • modulemeta ×

Known Compatibility Issues / Differences

Category: BUG

(*11) Operator Precedences in 1 + 3 as $a | ($a * 2)
Description

The presence of as $a affects precedence of | and other operators in jq:

$ jq -n '1 + 3 | (. * 2)' # interpreted as (1 + 3) | (. * 2)
8
$ jq -n '1 + 3 as $a | ($a * 2)' # interpreted as 1 + (3 as $a | ($a * 2))
7

whereas jackson-jq consistently interprets them as (1 + 3) whether as $a is used or not:

$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar -n '1 + 3 | (. * 2)' # interpreted as (1 + 3) | (. * 2)
8
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar -n '1 + 3 as $a | ($a * 2)' # interpreted as (1 + 3) as $a | ($a * 2)
8
Examples
$ jq -n '1 + 3 as $a | ($a * 2)' # interpreted as 1 + (3 as $a | ($a * 2))
7
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar -n '1 + 3 as $a | ($a * 2)' # interpreted as (1 + 3) as $a | ($a * 2)
8
Workaround

Use explicit parentheses.

Links
(*3) Multiple functions with the same name in the same scope
Description

If the function with the same is defined more than once at the same scope, jackson-jq uses the last one.

Examples
$ jq -n 'def f: 1; def g: f; def f: 2; g'
1
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar -n 'def f: 1; def g: f; def f: 2; g'
2
Workaround

Avoid using the duplicate function name.

$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar -n 'def f1: 1; def g: f1; def f2: 2; g'
1

Category: BY DESIGN

(*1) Error Message Wording
Description

Error messages differ between jq and jackson-jq and they also tend to change between versions.

Workaround

None. This is by design and will not be fixed.

(*6) env/0 is not available by default.
Description

env/0 is not available by default for security reasons and must be added manually to the scope.

Workaround

Add env/0 manually into the scope:

SCOPE.addFunction("env", 0, new EnvFunction())
(*4) Field Ordering in JSON Object
Description

The order of the keys in JSON is not preserved. It was a design decision but we are slowly trying to fix this in order to improve the compatibility with jq.

Workaround

None. Use array if the order is important.

(*5) 0 / 0 is an error in jackson-jq.
Description

jq evaluates 0 / 0, if hard-coded, to NaN without any errors, whereas 0 | 0 / . results in a zero-division error. jackson-jq always raises an error in both cases.

Examples
$ jq -n '0 / 0'
null
$ jq -n '10 / 0'
jq: error: Division by zero? at <top-level>, line 1:
10 / 0
jq: 1 compile error
$ jq '. / 0' <<< 0
jq: error (at <stdin>:1): number (0) and number (0) cannot be divided because the divisor is zero
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar -n '0 / 0'
jq: error: number (0) and number (0) cannot be divided because the divisor is zero
Workaround

If you need NaN, use nan instead of 0 / 0.

(*8) ... |= empty is an error in jackson-jq.
Description

.foo |= empty always throws an error in jackson-jq instead of producing an unexpected result. jq-1.5 and jq-1.6 respectively produces a different and incorrect result for [1,2,3] | ((.[] | select(. > 1)) |= empty). jq#897 says "empty in the RHS is undefined". You can still use _modify/2 directly if you really want to emulate the exact jq-1.5 or jq-1.6 behavior.

Examples
$ jq-1.6 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
[
  1,
  3
]
$ jq-1.5 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
null
$ jq-1.2 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
[
  1,
  2,
  3
]
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --jq 1.6 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
jq: error: `|= empty` is undefined. See https://github.com/stedolan/jq/issues/897
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --jq 1.5 -n '[1,2,3] | ((.[] | select(. > 1)) |= empty)'
jq: error: `|= empty` is undefined. See https://github.com/stedolan/jq/issues/897
Workaround

You can use _modify/2 if you really want to the original behavior.

$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --jq 1.6 -n '[1,2,3] | _modify((.[] | select(. > 1)); empty)'
[ 1, 3 ]
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --jq 1.5 -n '[1,2,3] | _modify((.[] | select(. > 1)); empty)'
null
(*7) Variables don't carry path information even in jq 1.5 compat mode.
Description

path(.foo as $a | $a) always throws an error as $variables in jackson-jq do not carry path information like jq-1.5 accidentally? did. The behavior is fixed in jq-1.6 whose documentation explicitly states them as "not a valid or useful path expression". So, I dicided not to implement it even in jq-1.5 compatible mode.

Examples

jq 1.5

$ jq-1.5 -c 'path(.foo as $a | $a)' <<< '{"foo": 1}'
["foo"]
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --jq 1.5 -c 'path(.foo as $a | $a)' <<< '{"foo": 1}'
jq: error: Invalid path expression with result 1

jq 1.6

$ jq-1.6 -c 'path(.foo as $a | $a)' <<< '{"foo": 1}'
jq: error (at <stdin>:1): Invalid path expression with result 1
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar --jq 1.6 -c 'path(.foo as $a | $a)' <<< '{"foo": 1}'
jq: error: Invalid path expression with result 1
Workaround

None

(*2) try (break $label) catch . always produces {"__jq": 0}.
Description

try (break $label) catch . always produces {"__jq": 0} in jackson-jq, while __jq should contain the index of the label the break statement jumps to.

Examples
$ jq -n 'label $a | label $b | try (break $b) catch .'
{
  "__jq": 1
}
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar -n 'label $a | label $b | try (break $b) catch .'
{
  "__jq" : 0
}
Workaround

None. Tell us your use case if you need this feature.

Category: BUGFIX

(*9) indices("") returns [] (empty array) in jackson-jq.
Description

indices/1 implementation in jq-1.5 and jq-1.6 had a bug that caused indices("") to end up in infinite loop which eventually leads to OOM. The bug is fixed and likely to be in jq-1.7 (not released yet). jackson-jq chose not to simulate this bug.

Examples
$ jq-1.5 -n '"x" | indices("")' # stuck in infinite loop
^C
$ jq-1.6 -n '"x" | indices("")' # stuck in infinite loop
^C
$ jq-1.6-83-gb52fc10 -n '"x" | indices("")'
[]
$ java -jar jackson-jq-cli-1.0.0-preview.20230409.jar -n '"x" | indices("")'
[ ]

Using jackson-jq/extras module

The jackson-jq/extras module is a jq module that provides some useful functions that do not exist in jq.

To use this module, you need to add the following Maven dependency and set BuiltinModuleLoader (see jackson-jq/src/test/java/examples/Usage.java) to the scope.

<dependency>
	<groupId>net.thisptr</groupId>
	<artifactId>jackson-jq-extra</artifactId>
	<version>1.0.0-preview.20230409</version>
</dependency>

Now, you can import the module in jq:

import "jackson-jq/extras" as extras;

extras::uuid4

For a historical reason, adding the Maven dependency also makes the functions directly available to jq. This behavior is deprecated and will be removed at some point in the future.

List of Functions

uuid4/0

  • jackson-jq -n 'uuid4' #=> "a69cf146-f40e-42e1-ae88-12590bdae947"

random/0

  • jackson-jq -n 'random' #=> 0.43292159535427466

timestamp/0, strptime/{1, 2}, strftime/{1, 2}

  • jackson-jq -n 'timestamp' #=> 1477162056362
  • jackson-jq -n '1477162342372 | strftime("yyyy-MM-dd HH:mm:ss.SSSXXX")' #=> "2016-10-23 03:52:22.372+09:00"
  • jackson-jq -n '1477162342372 | strftime("yyyy-MM-dd HH:mm:ss.SSSXXX"; "UTC")' #=> "2016-10-22 18:52:22.372Z"
  • jackson-jq -n '"2016-10-23 03:52:22.372+09:00" | strptime("yyyy-MM-dd HH:mm:ss.SSSXXX")' #=> 1477162342372
  • jackson-jq -n '"2016-10-22 18:52:22.372" | strptime("yyyy-MM-dd HH:mm:ss.SSS"; "UTC")' #=> 1477162342372

uriparse/0

  • jackson-jq -n '"http://[email protected]:8080/index.html?foo=1&bar=%20#hash" | uriparse' #=>

    {
  "scheme" : "http",
  "user_info" : "user",
  "raw_user_info" : "user",
  "host" : "www.example.com",
  "port" : 8080,
  "authority" : "[email protected]:8080",
  "raw_authority" : "[email protected]:8080",
  "path" : "/index.html",
  "raw_path" : "/index.html",
  "query" : "foo=1&bar= ",
  "raw_query" : "foo=1&bar=%20",
  "query_obj" : {
    "bar" : " ",
    "foo" : "1"
  },
  "fragment" : "hash",
  "raw_fragment" : "hash"
}

uridecode/0

  • jackson-jq -n '"%66%6f%6f" | uridecode' #=> "foo"

hostname/0

  • jackson-jq -n 'hostname' #=> "jenkins-slave01"

Contributing

  • If you are planning to send a PR and the change is not small, please open an issue and discuss it with the authors first.
  • Other than bug reports or patches, documentation improvements (including small grammatical or wording corrections) would be greatly appreciated.

License

This software is licensed under Apache Software License, Version 2.0, with some exceptions:

See COPYING for details.

More Repositories

1

java-procfs-mbeans-agent

Java agent to make /proc metrics available as MXBeans
Java
9
star
2

scriptable-jmx-exporter

A javaagent for scraping and exposing MBeans to Prometheus
Java
7
star
3

java-influxdb-metrics-agent

[⚠️DEPRECATED This repository is no longer maintained. A Prometheus alternative to this tool would be https://github.com/eiiches/scriptable-jmx-exporter] -- Java agent for collecting and reporting JMX metrics to InfluxDB
Java
7
star
4

go-gen-proxy

Proxy code generation for Go interfaces
Go
6
star
5

vim-ref-info

vim-ref source for info manuals.
Vim Script
4
star
6

mining-core

A data mining library in java.
Java
3
star
7

phoenix-jq-udf

jq for Apache Phoenix
Java
2
star
8

unite-tselect

:tselect using unite.vim interface.
Vim Script
2
star
9

vim-ref-gtkdoc

vim-ref addon for gtkdoc
Vim Script
2
star
10

vim-rainbowbrackets

Display brackets in different colors according to their nesting levels.
Vim Script
2
star
11

specialize-java

[⚠️ DEPRECATED -- I'm not maintaining this anymore. Moreover, this probably won't work with Java 8. Hopefully, in the future version of Java, we might be able to do similar things without any of these cranky hacks. See Project Valhalla. ] Generics with primitive types in Java.
Java
2
star
12

vim-colorconvert

Functions for converting colors in vim.
Python
1
star
13

vim-colored-colorscheme

Python
1
star
14

go-hadoop-io

SequenceFile reader for Go (still in early development)
Go
1
star
15

docker-jqplay-jackson-jq

Dockerfile for jqplay with jq replaced by jackson-jq
Dockerfile
1
star
16

gtkdoc

cui gtk-doc viewer
C
1
star
17

vimconfig

my vim configurations
Vim Script
1
star
18

gladeviewer

A GTK+ UI file viewer. Planning to implement auto-reloading, allowing users to see the resulting UI while editing XML manually.
C++
1
star
19

clb.ko

[WORK IN PROGRESS!!!!] Load-balance connect(2)ions transparently to the application, with zero overhead after establishment.
C
1
star