• Stars
    star
    125
  • Rank 285,458 (Top 6 %)
  • Language
    Scala
  • License
    BSD 3-Clause "New...
  • Created almost 9 years ago
  • Updated about 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

SBT Plugin to ease working with JNI

CI Maven Central Snapshots

sbt version group id plugin version
0.13.x ch.jodersky 1.2.6
1.x ch.jodersky 1.4.1
1.x com.github.sbt Maven Central

SBT-JNI

A suite of sbt plugins for simplifying creation and distribution of JNI programs.

Setup

Add sbt-jni as a dependency to project/plugins.sbt:

addSbtPlugin("com.github.sbt" % "sbt-jni" % "<latest version>")

where <latest version> refers to the version indicated by the badge above.

Note: We changed the organization from ch.jodersky to com.github.sbt

Motivation

Java Native Interface (JNI), is a framework that enables programs written in a JVM language to interact with native code and vice-versa. Such programs can be divided into two logical parts: the JVM part, consisting of sources that will be compiled to bytecode (e.g. Scala or Java), and the native part, consisting of sources that will be compiled to machine-native code (e.g. C, C++ or assembly).

Using native code can be beneficial in some situations: it can, for example, provide raw performance boosts or enable otherwise infeasable features such as interaction with peripherals. However, it also adds a few layers of complexities, most notably:

  • Compilation: the project is divided into two parts, each of which require separate compilation.
  • Portability: native binaries only run on the platform on which they were compiled.
  • Distribution: native binaries must be made available and packaged for every supported platform.

The second point, portability, is inherent to JNI and thus unavoidable. However the first and last points can be greatly simplified with the help of build tools.

Plugin Summary

Plugin Description
JniJavah Adds support for generating headers from classfiles that have @native methods.
JniLoad Makes @nativeLoader annotation available, that injects code to transparently load native libraries.
JniNative Adds sbt wrapper tasks around native build tools to ease building and integrating native libraries.
JniPackage Packages native libraries into multi-platform fat jars. No more manual library installation!

Note that most plugins are enabled in projects by default. Disabling their functionality can be achieved by adding disablePlugins(<plugin>) to the corresponding project definition (for example, should you wish to disable packaging of native libraries).

Plugin Details

JniJavah

Enabled Source
automatic, for all projects JniJavah.scala

This plugin wraps the JDK javah command 1.

Run sbt-javah to generate C header files with prototypes for any methods marked as native. E.g. the following scala class

package org.example
class Adder(val base: Int) {
  @native def plus(term: Int): Int // implemented in a native library
}

will yield this prototype

/*
 * Class:     org_example_Adder
 * Method:    plus
 * Signature: (I)I
 */
JNIEXPORT jint JNICALL Java_org_example_Adder_plus
  (JNIEnv *, jobject, jint);

The header output directory can be configured

javah / target := <dir> // defaults to target/native/include

Note that native methods declared both in Scala and Java are supported. Whereas Scala uses the @native annotation, Java uses the native keyword.

JniLoad

Enabled Source
automatic, for all projects JniLoad.scala

This plugin enables loading native libraries in a safe and transparent manner to the developer (no more explicit, static System.load("library") calls required). It does so by providing a class annotation which injects native loading code to all its annottees. Furthermore, in case a native library is not available on the current java.library.path, the code injected by the annotation will fall back to loading native libraries packaged according to the rules of JniPackage.

Usage example (Scala 2.x):

import com.github.sbt.jni.nativeLoader

// By adding this annotation, there is no need to call
// System.load("adder0") before accessing native methods.
@nativeLoader("adder0")
class Adder(val base: Int) {
  @native def plus(term: Int): Int // implemented in libadder0.so
}

// The application feels like a pure Scala app.
object Main extends App {
  (new Adder(0)).plus(1)
}

Note: this plugin is just a shorthand for adding sbt-jni-core (the project in core/) and the scala-macros-paradise (on Scala <= 2.13) projects as provided dependencies.

See the annotation's implementation for details about the injected code.

Usage example (Scala 3.x / Scala 2.x):

Scala 3 has no macro annotations support. As a solution we don't need this to be a macro function anymore. As the result, this option requires to have an explicit dependency on the sbt-jni-core library.

Note that if you want to run or test the project from sbt and have ThisBuild / turbo := true, you have to change the classLoaderLayeringStrategy to ClassLoaderLayeringStrategy.Flat, otherwise you will get UnsatisfiedLinkError, like java.lang.UnsatisfiedLinkError: 'int simple.Adder.plus(int)'.

This plugin behavior is configurable via:

// set to `Provided` by default, `Compile` is needed to use syntax (`extends NativeLoader`)
sbtJniCoreScope := Compile
// to make the code below work the core project should be included as a dependency via
// sbtJniCoreScope := Compile
import com.github.sbt.jni.syntax.NativeLoader

// By adding this annotation, there is no need to call
// System.load("adder0") before accessing native methods.
class Adder(val base: Int) extends NativeLoader("adder0"):
  @native def plus(term: Int): Int // implemented in libadder0.so

// The application feels like a pure Scala app.
@main def main: Unit = (new Adder(0)).plus(1)

JniNative

Enabled Source
manual JniNative.scala

JniNative adds the capability of building native code (compiling and linking) to sbt, by interfacing with commonly used build tools.

Since this plugin is basically a command-line wrapper, native build tools must follow certain calling conventions to be compatible. The supported build tools are currently:

An initial, compatible build template can be obtained by running sbt nativeInit <tool>. Once the native build tool initialised, projects are built by calling the sbt nativeCompile task.

Source and output directories are configurable

nativeCompile / sourceDirectory := sourceDirectory.value / "native"
nativeCompile / target := target.value / "native" / nativePlatform.value

CMake

A regular CMake native project definition usually looks this following way:

lazy val native = project
  // sourceDirectory = <project_root>/native/src
  .settings(nativeCompile / sourceDirectory := sourceDirectory.value)
  .enablePlugins(JniNative)

Source directory is set to sourceDirectory.value since the CMake project structure is of the following shape:

β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ CMakeLists.txt
β”‚   β”œβ”€β”€ lib.cpp

By default, CMake build is launched the following flags:

  • -DCMAKE_BUILD_TYPE=Release
  • -DSBT:BOOLEAN=true

It is possible to configure CMake by overriding the nativeBuildTool setting:

// default 
nativeBuildTool := CMake.make(Seq("-DCMAKE_BUILD_TYPE=Release", "-DSBT:BOOLEAN=true"))
// debug mode
nativeBuildTool := CMake.make(Seq("-DCMAKE_BUILD_TYPE=Debug", "-DSBT:BOOLEAN=true"))
// no flags passed
nativeBuildTool := CMake.make(Nil)

Cargo

A regular Cargo native project definition usually looks this following way:

lazy val native = project
  // baseDirectory = <project_root>/native
  .settings(nativeCompile / sourceDirectory := baseDirectory.value)
  .enablePlugins(JniNative)

Source directory is set to baseDirectory.value since the Cargo project structure is of the following shape:

β”œβ”€β”€ Cargo.toml
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ lib.rs

By default, Cargo build is launched with the --release flag. It is possible to configure Cargo profile by overriding the nativeBuildTool setting:

// default
nativeBuildTool := Cargo.make(Seq("--release"))
// extra flags passed
nativeBuildTool := Cargo.make(Seq("--release", "--ignore-rust-version"))
// no flags passed, debug mode
nativeBuildTool := Cargo.make(Nil)

Meson

A regular Meson native project definition usually looks this following way:

lazy val native = project
  // baseDirectory = <project_root>/native
  .settings(nativeCompile / sourceDirectory := baseDirectory.value)
  .enablePlugins(JniNative)

Source directory is set to baseDirectory.value since the Meson project structure is of the following shape:

β”œβ”€β”€ meson.build
β”œβ”€β”€ meson.options
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ library.c

By default, Meson build is launched with the --buildtype=release flag. It is possible to configure Meson by overriding the nativeBuildTool setting:

// default
nativeBuildTool := Meson.make(Seq("--buildtype=release"))
// extra flags passed
nativeBuildTool := Meson.make(Seq("--buildtype=release", "--fatal-meson-warnings"))
// no flags passed, debug mode
nativeBuildTool := Meson.make(Nil)

JniPackage

Enabled Source
automatic, when JniNative enabled JniPackage.scala

This plugin packages native libraries produced by JniNative in a way that they can be transparently loaded with JniLoad. It uses the notion of a native "platform", defined as the architecture-kernel values returned by uname -sm. A native binary of a given platform is assumed to be executable on any machines of the same platform.

Canonical Use

Keep in mind that sbt-jni is a suite of plugins, there are many other use cases. This is a just a description of the most common one.

  1. Define separate sub-projects for JVM and native sources. In myproject/build.sbt:

    lazy val core = (project in file("myproject-core")) // regular scala code with @native methods
      .dependsOn(native % Runtime) // remove this if `core` is a library, leave choice to end-user
    
    lazy val native = (project in file("myproject-native")) // native code and build script
      .enablePlugins(JniNative) // JniNative needs to be explicitly enabled

    Note that separate projects are not strictly required. They are strongly recommended nevertheless, as a portability-convenience tradeoff: programs written in a JVM language are expected to run anywhere without recompilation, but including native libraries in jars limits this portability to only platforms of the packaged libraries. Having a separate native project enables the users to easily swap out the native library with their own implementation.

  2. Initialize the native build tool from a template:

    Run sbt "nativeInit cmake <libname>"

  3. Implement core project:

    This step is identical to building a regular scala project, with the addition that some classes will also contain @native methods.

  4. Generate native headers:

    Run sbt javah

  5. Implement native headers:

    The function prototypes in the header files must be implemented in native code (such as C, C++) and built into a shared library. Run sbt nativeCompile to call the native build tool and build a shared library.

  6. Build/run/test:

    At this point, the project can be tested and run as any standard sbt project. For example, you can publish your project as a library (sbt publish), run it (sbt core/run) or simply run unit tests (sbt test). Packaging and recompiling of the native library will happen transparently.

  7. Develop:

    The usual iterative development process. Nothing speial needs to be done, except in case any @native methods are added/removed or their signature changed, then sbt javah needs to be run again.

Examples

The plugins' unit tests offer some simple examples. They can be run individually through these steps:

  1. Publish the core library locally sbt publishLocal.
  2. Change to the test's directory and run sbt -Dplugin.version=<version>.
  3. Follow the instructions in the test file (only enter the lines that start with ">" into sbt).

Real-world use-cases of sbt-jni include:

Requirements and Dependencies

  • projects using JniLoad must use Scala versions 2.11, 2.12, 2.13 or 3.2
    • projects using JniLoad with Scala 3 should use it with the sbtJniCoreScope := Compile SBT key set
  • only POSIX platforms are supported (actually, any platform that has the uname command available)

The goal of sbt-jni is to be the least intrusive possible. No transitive dependencies are added to projects using any plugin (some dependencies are added to the provided configuration, however these do not affect any downstream projects).

Building

Both the core (former macros) library (sbt-jni-core) and the sbt plugins (sbt-jni) are published. Cross-building happens on a per-project basis:

  • sbt-jni-core is built against Scala 2.11, 2.12, 2.13, and 3.2
  • sbt-jni is built against Scala 2.12 (the Scala version that sbt 1.x uses)

The differing Scala versions make it necessary to always cross-compile and cross-publish this project, i.e. append a "+" before every task.

Run sbt +publishLocal to build and use this plugin locally.

Copying

This project is released under the terms of the 3-clause BSD license. See LICENSE for details.

javah is released under the terms of the MIT license since it uses Glavo's gjavah. See LICENSE for details.

Footnotes

  1. Glavo's gjavah is actually used, since javah has been removed from the JDK since version 1.10. If something goes wrong, please open an issue to help us improve it. ↩

More Repositories

1

sbt

sbt, the interactive build tool
Scala
4,685
star
2

sbt-native-packager

sbt Native Packager
Scala
1,594
star
3

sbt-dependency-graph

sbt plugin to create a dependency graph for your project
Scala
1,245
star
4

sbt-jmh

"Trust no one, bench everything." - sbt plugin for JMH (Java Microbenchmark Harness)
Scala
789
star
5

sbt-eclipse

Plugin for sbt to create Eclipse project definitions
Scala
718
star
6

sbt-release

A release plugin for sbt
Scala
642
star
7

sbt-buildinfo

I know this because build.sbt knows this.
Scala
550
star
8

sbt-web

Library for building sbt plugins for the web
Scala
368
star
9

sbt-git

A git plugin for sbt
Scala
342
star
10

zinc

Scala incremental compiler library, used by sbt and other build tools
Scala
333
star
11

docker-sbt

Official sbt docker images
Dockerfile
317
star
12

sbt-dynver

An sbt plugin to dynamically set your version from git
Scala
299
star
13

sbt-ci-release

sbt plugin to automate Sonatype releases from GitHub Actions
Scala
283
star
14

sbt-onejar

Packages your project using One-JARβ„’
Scala
267
star
15

sbt-scalariform

sbt plugin adding support for source code formatting using Scalariform
Scala
258
star
16

sbt-fresh

sbt-plugin to create an opinionated fresh sbt project
Scala
233
star
17

sbt-github-actions

An sbt plugin which makes it easier to build with GitHub Actions
Scala
195
star
18

sbt-header

sbt-header is an sbt plugin for creating file headers, e.g. copyright headers
Scala
186
star
19

sbt-bintray

fresh packages delivered from your sbt console
Scala
179
star
20

sbt-protobuf

sbt plugin for compiling protobuf files
Scala
173
star
21

sbt-site

Site generation for sbt
Scala
173
star
22

sbt-start-script

SBT Plugin to create a "start" script to run the program
Scala
144
star
23

sbt-pgp

PGP plugin for sbt
Scala
142
star
24

sbt-groll

sbt plugin to roll the Git history
Scala
133
star
25

junit-interface

Implementation of sbt's test interface for JUnit
Java
132
star
26

sbt-unidoc

sbt plugin to create a unified Scaladoc or Javadoc API document across multiple subprojects.
Scala
124
star
27

sbt-jacoco

an sbt plugin for JaCoCo Code Coverage
Scala
123
star
28

sbt-projectmatrix

Scala
119
star
29

sbt-boilerplate

sbt plugin for generating scala.Tuple/Function related boilerplate code
Scala
109
star
30

sbt-proguard

Proguard sbt plugin
Scala
99
star
31

sbt-atmos

sbt plugin for running Typesafe Console in development
Scala
98
star
32

sbt-dirty-money

clean Ivy2 cache
Scala
89
star
33

sbt-launcher-package

Packaging for sbt so you can run it.
Scala
89
star
34

sbt-license-report

Report on licenses used in an sbt project.
Scala
85
star
35

website

The source for scala-sbt.org
Scala
77
star
36

sbt-doge

sbt plugin to aggregate tasks across subprojects and their crossScalaVersions
Scala
77
star
37

sbt-pom-reader

Translates xml -> awesome. Maven-ish support for sbt.
Scala
76
star
38

sbt-remote-control

Create and manage sbt process using unicorns and forks
Scala
74
star
39

sbt-aspectj

AspectJ sbt plugin
Scala
72
star
40

sbt-scalabuff

SBT plugin which generate case classes and support for serialization from Google Protocol Buffer definitions using ScalaBuff
Scala
72
star
41

contraband

http://www.scala-sbt.org/contraband/
Scala
67
star
42

sbt-s3

sbt-s3 is a simple sbt plugin to manipulate objects on Amazon S3
Scala
63
star
43

sbt-javaagent

sbt plugin for adding java agents to projects
Scala
56
star
44

sbt-multi-jvm

Multi-JVM testing in sbt
Scala
56
star
45

sbt-paradox-material-theme

Material Design theme for Paradox
StringTemplate
53
star
46

sbt-cpd

Copy & Paste Detector plugin using PMD for sbt.
Scala
48
star
47

sbt-osgi

sbt plugin for creating OSGi bundles
Scala
47
star
48

sbt-man

Looks up scaladoc.
Scala
46
star
49

sbt-findbugs

FindBugs static analysis plugin for sbt.
Scala
46
star
50

librarymanagement

librarymanagement module for sbt
Scala
46
star
51

ipcsocket

IPC: Unix Domain Socket and Windows Named Pipes for Java
Java
45
star
52

sbt-less

Scala
42
star
53

io

IO module for sbt
Scala
40
star
54

sbt-js-engine

Support for sbt plugins that use JavaScript
Scala
40
star
55

launcher

The sbt launcher as its own project. Can launch any ivy/maven published project with a main class, with some fancy features.
Scala
40
star
56

sbt-autoversion

Scala
37
star
57

sbt-avro

sbt plugin for compiling Avro schemas, similar to sbt-protobuf
Scala
33
star
58

sbt-jupiter-interface

Implementation of sbt's test interface for JUnit 5's Jupiter module
Java
32
star
59

sbt-digest

sbt-web plugin for checksum files
Scala
32
star
60

sbt-slash

unified slash syntax for both shell and build.sbt
Scala
29
star
61

sbt-java-formatter

An sbt plugin for formating Java code
Scala
28
star
62

sbt-gzip

sbt-web plugin for gzipping assets
Scala
26
star
63

sbt-pull-request-validator

Plugin that optimizes pull request validation to only validate sub projects that have changed
Scala
24
star
64

sbt-unique-version

emulates Maven's uniqueVersion snapshots
Scala
24
star
65

sbt-duplicates-finder

Find classes and resources conflicts in your build
Scala
22
star
66

sbt.github.com

See https://github.com/sbt/website for the source
HTML
22
star
67

sbt-autoplugin.g8

giter8 template for sbt 0.13.5+ AutoPlugin
Scala
20
star
68

sbt-cucumber

Cucumber plugin for SBT.
Scala
20
star
69

sbt-jcstress

Trust no-one, and especially not memory visibility.
HTML
19
star
70

adept

adept helps you find, declare, and download dependencies. http://groups.google.com/group/adept-dev/
18
star
71

sbt-sriracha

Scala
17
star
72

sbt-mocha

SBT plugin for running mocha JavaScript unit tests on node
Scala
17
star
73

sbt-multi-release-jar

Support for JDK9's Multi Release JAR Files (JEP 238)
Scala
17
star
74

sbt-xjc

SBT plugin to compile an XML Schema with XJC
Scala
15
star
75

util

util modules for sbt
Scala
15
star
76

sbt-export-repo

exports your dependency graph to a preloaded local repository
Scala
15
star
77

sbt-nocomma

sbt-nocomma reduces commas from your build.sbt.
Scala
13
star
78

serialization

serialization facility for sbt
Scala
13
star
79

sbt-maven-resolver

An sbt plugin to resolve dependencies using Aether
Scala
12
star
80

sbt-core-next

sbt APIs targeted for eventual inclusion in sbt core
Scala
12
star
81

sbt-houserules

House rules for sbt modules.
Scala
12
star
82

sbt-pamflet

sbt plugin to run Pamflet (and Pamflet plugin to run sbt)
Scala
11
star
83

sbt-sdlc

Scaladoc link checker for sbt
Scala
11
star
84

sbt-appbundle

A plugin for the simple-build-tool to create an OS X application bundle.
Scala
10
star
85

bintry

your packages, delivered fresh
Scala
10
star
86

sbt-fmpp

FreeMarker Scala/Java Templating Plugin for SBT
Scala
9
star
87

sbt-ynolub

Scala
9
star
88

sbt-testng

Implementation of the sbt testing interface for TestNG, bundled with an sbt plug-in for convenience.
Scala
9
star
89

sbt-concat

sbt-web plugin for concatenating web assets
Scala
8
star
90

sbt-ant

SBT plug-in to call Ant targets from within SBT builds
Scala
7
star
91

sbtn-dist

Shell
6
star
92

sbt-community-plugins

All community plugins that opt into an uber-build
Scala
6
star
93

sbt-vimquit

an sbt plugin that adds :q command.
Scala
5
star
94

helloworld-one

An example build for sbt 1.0.0.
Scala
5
star
95

sbt-giter8-resolver

Scala
5
star
96

sbt-sequential

adds sequential tasks to sbt
Scala
4
star
97

sbt-scalashim

generates sys.error.
Scala
4
star
98

sbt-experimental

Experimental APIs to fix rough edges in sbt
Scala
3
star
99

sbt-web-build-base

Scala
3
star
100

sbt-validator

Builds sbt 1.0.x against recent versions of the sbt modules
Shell
3
star