• Stars
    star
    132
  • Rank 264,111 (Top 6 %)
  • Language
    Scala
  • License
    Apache License 2.0
  • Created about 9 years ago
  • Updated 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

sbt plugin for generating Scala sources for Apache Avro schemas and protocols.

sbt-avrohugger

Scala CI Maven Central

sbt plugin for generating Scala case classes and ADTs from Apache Avro schemas, datafiles, and protocols.

Install the plugin (compatible with sbt 1.3+)

Add the following lines to the file myproject/project/plugins.sbt in your project directory:

addSbtPlugin("com.julianpeeters" % "sbt-avrohugger" % "2.5.1")

NOTE: On Nexus, please set nexus proxy layout to permissive in to resolve artifacts with a sbt-version suffixes

Usage

The following tasks and settings are automatically imported to your build:

Tasks:

Name Description
avroScalaGenerate Compiles the Avro files into Scala case classes.
avroScalaGenerateScavro (@deprecated) Compiles the Avro files into Scala case class Scavro wrapper classes.
avroScalaGenerateSpecific Compiles the Avro files into Scala case classes implementing SpecificRecord.

Compile

Wire the tasks into compile in your build.sbt:

e.g.: Compile / sourceGenerators += (Compile / avroScalaGenerate).taskValue

By default, the plugin looks Avro files in src/main/avro and generates Scala files in $sourceManaged, e.g., target/scala-2.13/src_managed/main/compiled_avro/ (to choose different locations, please see Changing Settings).

Test

And/Or wire the tasks into the Test config, putting Avro files in src/test/avro:

e.g.: Test / sourceGenerators += (Test / avroScalaGenerate).taskValue

Manually

To run the tasks manually, please see Changing Settings or the sbt docs in order to ensure the compiler will be able to find the generated files.

Watch Avro Files

To enable file-watching for avro files, e.g. in ~compile, use:

e.g.: watchSources ++= ((Compile / avroSourceDirectory).value ** "*.avdl").get

Settings:

Standard Settings

Name Default Description
avroSourceDirectories Seq("src/main/avro") List of paths containing *.avsc, *.avdl, and/or *.avro files.
avroScalaSource $sourceManaged/main/compiled_avro Path for the generated *.scala or *.java files.
avroScalaCustomTypes Standard.defaultTypes Customizable Type Mapping.
avroScalaCustomNamespace Map.empty[String, String] Map for reassigning namespaces.

Scavro Settings (@deprecated)

Name Default Description
avroScavroSourceDirectories Seq("src/main/avro") List of paths containing *.avsc, *.avdl, and/or *.avro files.
avroScavroScalaSource $sourceManaged/main/compiled_avro Path for the generated *.scala or *.java files.
avroScalaScavroCustomTypes Scavro.defaultTypes Customizable Type Mapping.
avroScalaScavroCustomNamespace Map.empty[String, String] Map for reassigning namespaces.

SpecificRecord Settings

Name Default Description
avroSpecificSourceDirectories Seq("src/main/avro") Path containing *.avsc, *.avdl, and/or *.avro files.
avroSpecificScalaSource $sourceManaged/main/compiled_avro Path for the generated *.scala or *.java files.
avroScalaSpecificCustomTypes SpecificRecord.defaultTypes Customizable Type Mapping.
avroScalaSpecificCustomNamespace Map.empty[String, String] Map for reassigning namespaces.

Changing Settings

Settings for each format's task can be extended/overridden by adding lines to your build.sbt file.

E.g., to change how classes of SpecificRecords format are generated, use:

Compile / avroSpecificSourceDirectories += (Compile / sourceDirectory).value / "myavro"

Compile / avroSpecificScalaSource := new java.io.File("myScalaSource")

Compile / avroScalaSpecificCustomNamespace := Map("my.example"->"my.overridden.ex", "test.*" -> "wildcarded")

Compile / avroScalaSpecificCustomTypes := {
  avrohugger.format.SpecificRecord.defaultTypes.copy(
    array = avrohugger.types.ScalaVector)
}
  • record can be assigned to ScalaCaseClass and ScalaCaseClassWithSchema (with schema in a companion object)
  • array can be assigned to ScalaSeq, ScalaArray, ScalaList, and ScalaVector
  • enum can be assigned to JavaEnum, ScalaCaseObjectEnum, EnumAsScalaString, and ScalaEnumeration
  • fixed can be assigned to , ScalaCaseClassWrapper and ScalaCaseClassWrapperWithSchema(with schema in a companion object)
  • union can be assigned to OptionEitherShapelessCoproduct and OptionalShapelessCoproduct
  • int, long, float, double can be assigned to ScalaInt, ScalaLong, ScalaFloat, ScalaDouble
  • date logical type can be assigned to JavaTimeLocalDate and JavaSqlDate
  • timestamp-millis logical type can be assigned to JavaTimeInstant and JavaSqlTimestamp
  • uuid logical type can be assigned to UUID
  • decimal can be assigned to e.g. ScalaBigDecimal(Some(BigDecimal.RoundingMode.HALF_EVEN)) and ScalaBigDecimalWithPrecision(None) (via Shapeless Tagged Types)
  • protocol can be assigned to ScalaADT and NoTypeGenerated (see Protocol Support)

Datatypes

Supports generating case classes with arbitrary fields of the following datatypes: see avrohugger docs - supported datatypes

Testing

Please run unit tests in src/test/scala with ^ test, and integration tests in src/sbt-test with ^ scripted sbt-avrohugger/*, or, e.g. scripted sbt-avrohugger/specific.

Credits

sbt-avrohugger is based on sbt-avro originally developed by Juan Manuel Caicedo, and depends on avrohugger.

Contributors

Marius Soutier
Vince Tse
Saket
Raúl Raja Martínez
Marco Stefani
Sacha Barber
sullis
Alexandre Bertails
Alexander Khotyanov
Brennan Saeta
Jerome Wacongne
Jon Morra
Paul Snively
Diego E. Alonso Blas
Andrew Gustafson
Jonas Grabber
mcenkar
Daniel Lundin
Ryan Koval
Simonas Gelazevicius
Zach Cox
Chris Albright
Fede Fernández
natefitzgerald
Angel Sanadinov

Fork away, just make sure the tests pass before you send a pull request.

Criticism is appreciated.

More Repositories

1

avrohugger

Generate Scala case class definitions from Avro schemas
Scala
199
star
2

avro-scala-macro-annotations

Compile-time tools for working with Avros in Scala
Scala
55
star
3

case-class-generator

Dynamically defines and loads Scala classes at runtime. Useful for turning JSON schemas into Scala case classes on the fly.
Scala
44
star
4

avro2caseclass

Generate Scala case class definitions from Avro schemas
Scala
5
star
5

artisanal-pickle-maker

Make your own Scala pickled signatures. Takes class info as strings and gives back a pickled Scala signatures. For use with Java bytecode engineering libraries.
Scala
4
star
6

avro-scala-macro-annotation-examples

EXAMPLES HAVE MOVED TO THE PROJECT'S REPOSITORY:
Scala
2
star
7

hello-kafka-streams-twitter

Java
2
star
8

dc10

Code generation tools for Scala
Scala
2
star
9

polynomial

The category of Poly, simply typed.
Scala
2
star
10

dynamical

Mode-dependent dynamical systems
Scala
2
star
11

asm-example

A few examples of objectweb's asm, a java bytecode library. Trying out the ASMifier to run HelloWorlds from bytecode in both Scala and Java.
Java
1
star
12

asm-salat-example

An example of how to use Salat with a dynamically generated case class as the model.
Java
1
star
13

schemabuilder4cats

A referentially transparent FFI to `org.apache.avro.SchemaBuilder`
Scala
1
star
14

dc10-cats-effect

Library for use with the `dc10-scala` code generator
Scala
1
star
15

toolbox-type-provider

A runtime type-provider that gives case classes from strings, using standard (but experimental) Scala reflection
Scala
1
star
16

dc10-scala

A definitional compiler for generating Scala code.
Scala
1
star
17

programming-language-foundations

Programming Language Foundations (Stump 2013)
Haskell
1
star
18

dc10-scalaq

Scala
1
star
19

dc10-scalaq-twelf

Render to Twelf as a target lang. Library for use with the dc10-scalaq code generator.
Scala
1
star
20

destructured

Typeclasses of a lower kind.
Scala
1
star
21

sbt-buildlevel

Which build is currently loaded, meta-build or proper build?
Scala
1
star
22

scalassist-example

*Under Construction* Scalassist is a tool to make Java bytecode-engineering libraries more usable with Scala. It updates a pickled Scala signature to reflect changes made to the bytecode.
Scala
1
star
23

scalasig-caseclass-experiments

generating a bunch of case class varieties, and comparing their scala sigs to get a sense of how to make one myself.
Scala
1
star