🌎 Travelm Agency

Table of Contents

🔍 Feature Overview
📖 How to use
- ⌨️ Script usage
- 📦 Usage as a node module
Explanation on the generated Elm file
🌯 Embedding the output in your application
▶️ Inline vs Dynamic
Detail on supported formats
Internal structure
🙄 Why another i18n solution

This package is a library and an executable to generate Elm Modules from different common I18n formats. You can see what the package does and how to use it in the interactive tutorial.

🔍 Feature Overview

✔️ Supports .json, .ftl (with access to the Browsers Intl API with Elm 0.19!) and .properties files as input
🕵️‍♂️ Errors out if i18n keys are inconsistent across languages (unless you explicitely specify a fallback language)
🚀 Key length and placeholder length do not impact resource size
✨ Generates correctly typed Elm functions for values with placeholders
✍ Generates HTML if your translations contain HTML tags
🔑 Generated Elm module exposes all your translation keys as functions
🎌 Simple runtime switching of languages
🎚️ Choose between inline code generation and dynamic loading based code generation
♾️ Optional hashing of generated filenames for infinite browser caching
🖋️ Node API is written in Typescript

📖 How to use

Install this package from npm with npm install --save-dev travelm-agency.
Put your translations in files of one of the supported formats and bundle them in a folder. The filenames should follow the pattern [identifier].[language].[extension].
Choose a filepath for the generated Elm file.
Choose a folder for the JSON output
Run the script with one of the methods described below
Follow the output documentation to actually run the code

⌨️ Script usage

Run

npx travelm-agency --elm_path=src/I18n.elm --json_path=dist/i18n [folder with translation files]

Note	`--json_path` is not necessary when you use `--inline` mode.

Alternatively, add a script to your package.json with the content

travelm-agency --elm_path=src/I18n.elm --json_path=dist/i18n [folder with translation files]

and then run that script with npm run [scriptname].

For more information on command line options, run npx travelm-agency --help.

📦 Usage as a node module

If you need more customizable behaviour, or need to embed this inside a more complex script or plugin, you can import this module in a js or ts file and go from there.

Let the types guide you 🦮.

Explanation on the generated Elm file

We are going to look at the type signatures of a generated modules exposed functions to explain how this tool is meant to be used.

type Language = De | En
languageFromString : String -> Maybe Language
languageToString : Language -> String

Let us start with the Language type. Travelm-Agency generates an Enum containing all given languages. So if you give it two files messages.de.json and messages.en.json, this is what would come out. For now, the generated functions on the Language type are very basic. In the future, languageFromString will probably select the closest match, so that en-US would result in Just En and not Nothing.

init : I18n
init : Language -> I18n
init : Intl -> Language -> I18n

Next, the init function. I18n is a opaque type that differs in its implementation depending on which features your translations need. Therefore, the type signature of init and its implementation varies. This is the only way to get an instance of the I18n type.

load : Language -> I18n -> I18n
loadMessages : { language : Language, path : String, onLoad : Result Http.Error (I18n -> I18n) -> msg } -> Cmd msg

The load family of functions modifies your I18n instance with the translations you want to load. Again the type signatures and their implementations vary, but the idea is the same. In the dynamic case, you can load seperate message bundles seperately, so the functions are named accordingly. In this case loadMessages is the result of a translation file named messages.[language].[extension].

yourTranslationKey : I18n -> String
yourTranslationKeyWithPlaceholder : String -> I18n -> String
yourTranslationKeyWithMultiplePlaceholders : { name: String, profession: String } -> I18n -> String
yourTranslationKeyUsingNumberFormat : Float -> I18n -> String

Travelm-Agency generates a function for each of your keys in your translation files. The functions will always take your I18n instance as the last argument and return a String. Depending on the number of variables in your translations, the function will take additional arguments.

🌯 Embedding the output in your application

To use the output generated by this package in your application, the general idea is to store the active translations inside your Model and load translations on init and on demand. To do that, you call the generated load[translation-file-identifier] function (e.g. loadMessages for messages.en.json), sending an HTTP request to get the generated JSON file from your server. The update to your translations will then go in your main update function, where you can update your Model.

In your view, you can access your translations by using the exposed accessor functions of the generated Elm module.

View the /demo directory for working code that builds the interactive tutorial.

▶️ Inline vs Dynamic

For the example applications, the inline variant results in a smaller bundle. However, this is mostly the case because of non-needed elm/http and elm/parser. In many webapps, these packages will end up in the bundle regardless.

I introduced this package in one of my webapps and with 15 key/value pairs and 2 languages, the dynamic variant started winning slightly.

For more detail and thoughts on optimization and how this package works internally, take a look here.

Detail on supported formats

JSON

Needs to be a top level object with strings as keys and strings or objects of the same format as values. Example:

✔️ { "my": { "json": {"object": "value" } } }

❌ "top level string"

❌ { "no": ["arrays"], "or": { "numbers": 42 } }

Comments are not allowed. Placeholders use {curly-bracket} syntax, if you want a literal "{" use "\{" to escape it. No multiline support.

The '<' symbol is interpreted as a start to an HTML expression. If you want to escape this behaviour, use "\\<".

To disable missing key detection and default missing keys to some other of your languages, you can use { "--fallback-language": "{your language key, i.e. 'en'}" } as a top-level key.

Properties

Needs to be a newline seperated list of key value pairs (seperated by "="). Whitespace before and after the "=" is ignored. You may break your value into multiple lines by ending every line but the last with "\". Example:

✔️ my.property = test
✔️ my.multiline = test \
    extra \
    lines

❌ key.without.value
❌ multiline = without
      backslash

Lines leading with "#" are treated as comments.

Placeholders use {curly-bracket} syntax, if you want a literal "{", you can use "{" or '{', similarily use "'" for the literal single quote and '"' for the literal double quote.

The '<' symbol is interpreted as a start to an HTML expression. If you want to escape this behaviour, use quotes ('<').

To disable missing key detection and default missing keys to some other of your languages, you can add a comment to the respective file:

# fallback-language: en

^ this will use the key value pairs of your .en.properties file if any of the current file are missing.

Fluent

See Fluent Homepage for documentation. Most of the syntax should be supported:

Straight up texts
Interpolation ({$var}) of runtime variables (also referred to as placeholders in this README)
References to terms ({-term-name})
Term arguments ({ -term-name(name: "Andy") })
Text placeables ({ "…" })
Multiline Texts
Attributes
Comments
NUMBER and DATETIME function (only with explicit usage)
Runtime matching on variables (numbers, gender)

The '<' symbol is interpreted as a start to an HTML expression. If you want to escape this behaviour, use a text placeable.

To disable missing key detection and default missing keys to some other of your languages, use the same approach as for .properties files:

# fallback-language: en

Internal structure

This section is for people who are interested in contributing to help you get started quicker.

Overview

Travelm-Agency is a classic two stage compiler. In the first stage, the given file (like .json for example) is parsed and transformed into an AST (Abstract Syntax Tree). This is done by the code in the ContentTypes folder. The AST pieces are in the Types folder.

For example, the string { "key": "value" } becomes an Elm data type ("key", (Text "value", [])).

In the second stage, we generate Elm (and possibly other) files from the AST. This is done by the code in the Generators folder.

Design considerations

Most of the time, the less passes you have to do, the faster/less resource-intensive a compiler is. We still chose the two split phases for several reasons:

It is a lot easier to test, since we can test the two stages seperately.
We have to write less tests, since we do not have to cover the cartesian product of ContentTypes x GenerationModes but instead test ContentTypes → AST and AST → GenerationModes.
We can do some nice optimizations with some full AST analysis (i.e. do not generate some parsing code for some interpolation feature if it won’t be used)

Testing

As mentioned in the previous section, most tests are of the kind ContentTypes → AST or AST → GenerationModes. The first kind is rather straight forward, using Elm multiline strings, we can just have "inlined" files in the tests from which we generate ASTs.

The second kind is more involved. We could regression test the generated code using string comparison, but the tests would fail a lot because of minor, uninteresting changes, a lot of which would not even have any runtime impact.

Therefore, we import the generated files in the tests themselves so that we can confirm that the generated code typechecks and does "the right thing". To do that, there is a seperate folder gen_test_cases and an associated JS script generate_test_cases.js, which calls the generator elm code via the node-elm-compiler for each file in gen_test_cases ending with …Case.elm. The resulting generated files live in the gen_test_cases Inline and Dynamic subdirectories and can be imported in tests just like any other file. The directories with generated files are gitignored and thus you will need to run generate_test_cases.js once for tests to compile.

🙄 Why another i18n solution

Here are some other i18n solutions with their differences:

elm-i18next-gen	Allows you to access your translations object in unsafe ways via the Translation API, but also more freedom. I like the approach of using Dict internally and not storing functions inside of the model. It made me switch my internal dynamic representation from a custom record into an Array. Also generates a lot of modules instead of one module with all translations.
elm-i18n	Generates a whole extra js bundle for each language. This makes initial load time optimal, but language switching during runtime more difficult. I like the approach because the user usually does not switch languages very often. I might write a frontend using this technique as well. The main issue here is that I have no idea how to use this together with a bundler like webpack.
i18n-to-elm	This chooses the --inline approach of this module. I like to be flexible and have an option to switch to/benchmark runtime loading
elm-i18n-module-generator	Also an inline approach, this time using a language union type.

Interestingly enough, none of these seem to have explored the possibility of optimizing the i18n .json files. More importantly, none of these can access the browsers Intl API with Elm 0.19. I think this is the first package to do so. As far as I know, this is also the first package to combine this feature set with HTML generation.

Also, I really enjoy metaprogramming Elm using Elm itself, that is why I started building this.

anmolitor/travelm-agency

anmolitor

Reviews

Repository Details