• Stars
    star
    446
  • Rank 94,513 (Top 2 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created about 10 years ago
  • Updated 5 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Parse your code to extract translation keys/values and manage your catalog files

i18next Parser Build Status codecov

NPM

When translating an application, maintaining the translation catalog by hand is painful. This package parses your code and automates this process.

Finally, if you want to make this process even less painful, I invite you to check Locize. They are a sponsor of this project. Actually, if you use this package and like it, supporting me on Patreon would mean a great deal!

Become a Patreon

Features

  • Choose your weapon: A CLI, a standalone parser or a stream transform
  • 6 built in lexers: Javascript, JSX, HTML, Handlebars, TypeScript+tsx and Vue
  • Creates one catalog file per locale and per namespace
  • Backs up the old keys your code doesn't use anymore in namespace_old.json catalog
  • Restores keys from the _old file if the one in the translation file is empty
  • Parses comments for static keys to support dynamic key translations.
  • Supports i18next features:
    • Context: keys of the form key_context
    • Plural: keys of the form key_zero, key_one, key_two, key_few, key_many and key_other as described here

Versions

You can find information about major releases on the dedicated page. The migration documentation will help you figure out the breaking changes between versions.

  • 8.x is tested on Node 16, 18 and 20.
  • 7.x is tested on Node 14, 16 and 18.
  • 6.x is tested on Node 14 and 16.

Usage

CLI

You can use the CLI with the package installed locally but if you want to use it from anywhere, you better install it globally:

yarn global add i18next-parser
npm install -g i18next-parser
i18next 'app/**/*.{js,hbs}' 'lib/**/*.{js,hbs}' [-oc]

Multiple globbing patterns are supported to specify complex file selections. You can learn how to write globs here. Note that glob must be wrapped with single quotes when passed as arguments.

IMPORTANT NOTE: If you pass the globs as CLI argument, they must be relative to where you run the command (aka relative to process.cwd()). If you pass the globs via the input option of the config file, they must be relative to the config file.

  • -c, --config : Path to the config file (default: i18next-parser.config.{js,mjs,json,ts,yaml,yml}).
  • -o, --output : Path to the output directory (default: locales/$LOCALE/$NAMESPACE.json).
  • -s, --silent: Disable logging to stdout.
  • --fail-on-warnings: Exit with an exit code of 1 on warnings
  • --fail-on-update: Exit with an exit code of 1 when translations are updated (for CI purpose)

Gulp

Save the package to your devDependencies:

yarn add -D i18next-parser
npm install --save-dev i18next-parser

Gulp defines itself as the streaming build system. Put simply, it is like Grunt, but performant and elegant.

import { gulp as i18nextParser } from 'i18next-parser'

gulp.task('i18next', function () {
  gulp
    .src('app/**')
    .pipe(
      new i18nextParser({
        locales: ['en', 'de'],
        output: 'locales/$LOCALE/$NAMESPACE.json',
      })
    )
    .pipe(gulp.dest('./'))
})

IMPORTANT: output is required to know where to read the catalog from. You might think that gulp.dest() is enough though it does not inform the transform where to read the existing catalog from.

Broccoli

Save the package to your devDependencies:

yarn add -D i18next-parser
npm install --save-dev i18next-parser

Broccoli.js defines itself as a fast, reliable asset pipeline, supporting constant-time rebuilds and compact build definitions.

import Funnel from 'broccoli-funnel'
import { broccoli as i18nextParser } from 'i18next-parser'

const appRoot = 'broccoli'

let i18n = new Funnel(appRoot, {
  files: ['handlebars.hbs', 'javascript.js'],
  annotation: 'i18next-parser',
})

i18n = new i18nextParser([i18n], {
  output: 'broccoli/locales/$LOCALE/$NAMESPACE.json',
})

export default i18n

Options

Using a config file gives you fine-grained control over how i18next-parser treats your files. Here's an example config showing all config options with their defaults.

// i18next-parser.config.js

export default {
  contextSeparator: '_',
  // Key separator used in your translation keys

  createOldCatalogs: true,
  // Save the \_old files

  defaultNamespace: 'translation',
  // Default namespace used in your i18next config

  defaultValue: '',
  // Default value to give to keys with no value
  // You may also specify a function accepting the locale, namespace, key, and value as arguments

  indentation: 2,
  // Indentation of the catalog files

  keepRemoved: false,
  // Keep keys from the catalog that are no longer in code
  // You may either specify a boolean to keep or discard all removed keys.
  // You may also specify an array of patterns: the keys from the catalog that are no long in the code but match one of the patterns will be kept.
  // The patterns are applied to the full key including the namespace, the parent keys and the separators.

  keySeparator: '.',
  // Key separator used in your translation keys
  // If you want to use plain english keys, separators such as `.` and `:` will conflict. You might want to set `keySeparator: false` and `namespaceSeparator: false`. That way, `t('Status: Loading...')` will not think that there are a namespace and three separator dots for instance.

  // see below for more details
  lexers: {
    hbs: ['HandlebarsLexer'],
    handlebars: ['HandlebarsLexer'],

    htm: ['HTMLLexer'],
    html: ['HTMLLexer'],

    mjs: ['JavascriptLexer'],
    js: ['JavascriptLexer'], // if you're writing jsx inside .js files, change this to JsxLexer
    ts: ['JavascriptLexer'],
    jsx: ['JsxLexer'],
    tsx: ['JsxLexer'],

    default: ['JavascriptLexer'],
  },

  lineEnding: 'auto',
  // Control the line ending. See options at https://github.com/ryanve/eol

  locales: ['en', 'fr'],
  // An array of the locales in your applications

  namespaceSeparator: ':',
  // Namespace separator used in your translation keys
  // If you want to use plain english keys, separators such as `.` and `:` will conflict. You might want to set `keySeparator: false` and `namespaceSeparator: false`. That way, `t('Status: Loading...')` will not think that there are a namespace and three separator dots for instance.

  output: 'locales/$LOCALE/$NAMESPACE.json',
  // Supports $LOCALE and $NAMESPACE injection
  // Supports JSON (.json) and YAML (.yml) file formats
  // Where to write the locale files relative to process.cwd()

  pluralSeparator: '_',
  // Plural separator used in your translation keys
  // If you want to use plain english keys, separators such as `_` might conflict. You might want to set `pluralSeparator` to a different string that does not occur in your keys.
  // If you don't want to generate keys for plurals (for example, in case you are using ICU format), set `pluralSeparator: false`.

  input: undefined,
  // An array of globs that describe where to look for source files
  // relative to the location of the configuration file

  sort: false,
  // Whether or not to sort the catalog. Can also be a [compareFunction](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#parameters)

  verbose: false,
  // Display info about the parsing including some stats

  failOnWarnings: false,
  // Exit with an exit code of 1 on warnings

  failOnUpdate: false,
  // Exit with an exit code of 1 when translations are updated (for CI purpose)

  customValueTemplate: null,
  // If you wish to customize the value output the value as an object, you can set your own format.
  // ${defaultValue} is the default value you set in your translation function.
  // Any other custom property will be automatically extracted.
  //
  // Example:
  // {
  //   message: "${defaultValue}",
  //   description: "${maxLength}", // t('my-key', {maxLength: 150})
  // }

  resetDefaultValueLocale: null,
  // The locale to compare with default values to determine whether a default value has been changed.
  // If this is set and a default value differs from a translation in the specified locale, all entries
  // for that key across locales are reset to the default value, and existing translations are moved to
  // the `_old` file.

  i18nextOptions: null,
  // If you wish to customize options in internally used i18next instance, you can define an object with any
  // configuration property supported by i18next (https://www.i18next.com/overview/configuration-options).
  // { compatibilityJSON: 'v3' } can be used to generate v3 compatible plurals.

  yamlOptions: null,
  // If you wish to customize options for yaml output, you can define an object here.
  // Configuration options are here (https://github.com/nodeca/js-yaml#dump-object---options-).
  // Example:
  // {
  //   lineWidth: -1,
  // }
}

Lexers

The lexers option let you configure which Lexer to use for which extension. Here is the default:

Note the presence of a default which will catch any extension that is not listed. There are 4 lexers available: HandlebarsLexer, HTMLLexer, JavascriptLexer and JsxLexer. Each has configurations of its own. Typescript is supported via JavascriptLexer and JsxLexer. If you need to change the defaults, you can do it like so:

Javascript

The Javascript lexer uses Typescript compiler to walk through your code and extract translation functions.

The default configuration is below:

{
  // JavascriptLexer default config (js, mjs)
  js: [{
    lexer: 'JavascriptLexer',
    functions: ['t'], // Array of functions to match
    namespaceFunctions: ['useTranslation', 'withTranslation'], // Array of functions to match for namespace
  }],
}

Jsx

The JSX lexer builds off of the Javascript lexer and extends it with support for JSX syntax.

Default configuration:

{
  // JsxLexer default config (jsx)
  // JsxLexer can take all the options of the JavascriptLexer plus the following
  jsx: [{
    lexer: 'JsxLexer',
    attr: 'i18nKey', // Attribute for the keys
    componentFunctions: ['Trans'], // Array of components to match
  }],
}

If your JSX files have .js extension (e.g. create-react-app projects) you should override the default js lexer with JsxLexer to enable jsx parsing from js files:

{
  js: [{
    lexer: 'JsxLexer'
  }],
}
Supporting helper functions in Trans tags

If you're working with i18next in Typescript, you might be using a helper function to make sure that objects in components pass the typechecker: e.g.,

const SomeComponent = (props) => (
  <Trans>
    Visit
    <Link to='/user/john'>{castAsString({ name: props.name })}'s profile</Link>
    {/* Equivalent to, but resolves typechecker errors with */}
    <Link to='/user/john'>{{ name: props.name }}'s profile</Link>
  </Trans>
)

function castAsString(record: Record<string, unknown>): string {
  return record as unknown as string
}

In order for the parser to extract variables properly, you can add a list of such functions to the lexer options:

{
  js: [{
    lexer: 'JsxLexer',
    transIdentityFunctionsToIgnore: ['castAsString']
  }],
}

Ts(x)

Typescript is supported via Javascript and Jsx lexers. If you are using Javascript syntax (e.g. with React), follow the steps in Jsx section, otherwise Javascript section.

Handlebars

{
  // HandlebarsLexer default config (hbs, handlebars)
  handlebars: [
    {
      lexer: 'HandlebarsLexer',
      functions: ['t'], // Array of functions to match
    },
  ]
}

Html

{
  // HtmlLexer default config (htm, html)
  html: [{
    lexer: 'HtmlLexer',
    attr: 'data-i18n' // Attribute for the keys
    optionAttr: 'data-i18n-options' // Attribute for the options
  }]
}

Custom lexers

You can provide function instead of string as a custom lexer.

import CustomJsLexer from './CustomJsLexer';

// ...
{
  js: [CustomJsLexer],
  jsx: [{
    lexer: CustomJsLexer,
    customOption: true // Custom attribute passed to CustomJsLexer class constructor
  }]
}
// ...

Caveats

While i18next extracts translation keys in runtime, i18next-parser doesn't run the code, so it can't interpolate values in these expressions:

t(key)
t('key' + id)
t(`key${id}`)

As a workaround you should specify possible static values in comments anywhere in your file:

// t('key_1')
// t('key_2')
t(key)

/*
t('key1')
t('key2')
*/
t('key' + id)

Events

The transform emits a reading event for each file it parses:

.pipe( i18next().on('reading', (file) => {}) )

The transform emits a error:json event if the JSON.parse on json files fail:

.pipe( i18next().on('error:json', (path, error) => {}) )

The transform emits a warning event if the file has a key that is not a string litteral or an option object with a spread operator:

.pipe( i18next().on('warning', (path, key) => {}) )

Here is a list of the warnings:

  • Key is not a string literal: the parser cannot parse variables, only literals. If your code contains something like t(variable), the parser will throw a warning.
  • Found same keys with different values: if you use different default values for the same key, you'll get this error. For example, having t('key', {defaultValue: 'foo'}) and t('key', {defaultValue: bar'}). The parser will select the latest one.
  • Found translation key already mapped to a map or parent of new key already mapped to a string: happens in this kind of situation: t('parent', {defaultValue: 'foo'}) and t('parent.child', {defaultValue: 'bar'}). parent is both a translation and an object for child.

Contribute

Any contribution is welcome. Please read the guidelines first.

Thanks a lot to all the previous contributors.

If you use this package and like it, supporting me on Patreon is another great way to contribute!

Become a Patreon


Gold Sponsors

More Repositories

1

react-i18next

Internationalization for react done right. Using the i18next i18n ecosystem.
JavaScript
8,979
star
2

i18next

i18next: learn once - translate everywhere
JavaScript
7,433
star
3

next-i18next

The easiest way to translate your NextJs apps.
TypeScript
5,289
star
4

i18next-browser-languageDetector

language detector used in browser environment for i18next
JavaScript
821
star
5

i18next-scanner

Scan your code, extract translation keys/values, and merge them into i18n resource files.
JavaScript
532
star
6

i18next-http-backend

i18next-http-backend is a backend layer for i18next using in Node.js, in the browser and for Deno.
JavaScript
416
star
7

i18next-node

[deprecated] can be replaced with v2 of i18next
JavaScript
261
star
8

i18next-xhr-backend

[deprecated] can be replaced with i18next-http-backend
JavaScript
253
star
9

i18next-express-middleware

[deprecated] can be replaced with i18next-http-middleware
JavaScript
206
star
10

i18next-gettext-converter

converts gettext .mo or .po to 18next json format and vice versa
JavaScript
190
star
11

jquery-i18next

jQuery-i18next is a jQuery based Javascript internationalization library on top of i18next. It helps you to easily internationalize your web applications.
HTML
166
star
12

i18next-gitbook

HTML
161
star
13

ng-i18next

translation for AngularJS using i18next
JavaScript
160
star
14

next-app-dir-i18next-example

Next.js 13/14 app directory feature in combination with i18next
JavaScript
158
star
15

i18next-http-middleware

i18next-http-middleware is a middleware to be used with Node.js web frameworks like express or Fastify and also for Deno.
JavaScript
146
star
16

react-i18next-gitbook

CSS
93
star
17

next-app-dir-i18next-example-ts

Next.js 13/14 app directory feature in combination with i18next
TypeScript
91
star
18

i18next-fs-backend

i18next-fs-backend is a backend layer for i18next using in Node.js and for Deno to load translations from the filesystem.
JavaScript
87
star
19

i18next-localstorage-backend

This is a i18next cache layer to be used in the browser. It will load and cache resources from localStorage and can be used in combination with the chained backend.
JavaScript
80
star
20

i18next-icu

i18nFormat plugin to use ICU format with i18next
JavaScript
75
star
21

i18next-node-fs-backend

[deprecated] can be replaced with i18next-fs-backend
JavaScript
65
star
22

i18next-vue

Internationalization for Vue 2 & 3 using the i18next ecosystem
TypeScript
64
star
23

i18next-chained-backend

An i18next backend to chain multiple backends (add fallbacks, caches, ...)
JavaScript
62
star
24

i18nextify

enables localization of any page with zero effort.
JavaScript
61
star
25

next-language-detector

This package helps to handle language detection in next.js when using static servers only.
JavaScript
55
star
26

i18next-resources-to-backend

This package helps to transform resources to an i18next backend
JavaScript
49
star
27

i18next-android

i18next internationalization library for Android
Java
41
star
28

i18next-webtranslate

[deprecated] Translation User Interface for i18next - successor locize.com
JavaScript
41
star
29

i18next-ios

i18next internationalization library for iOS
Objective-C
28
star
30

i18next-sprintf-postProcessor

sprintf post processor for i18next
JavaScript
28
star
31

i18next-resources-for-ts

This package helps to transform resources to be used in a typesafe i18next project.
JavaScript
28
star
32

i18next-localStorage-cache

[deprecated] caching layer for i18next using browsers localStorage
JavaScript
25
star
33

i18next-intervalPlural-postProcessor

post processor for i18next enabling interval plurals
JavaScript
24
star
34

i18next-fluent

i18nFormat plugin to use mozilla fluent format with i18next
JavaScript
22
star
35

i18next-multiload-backend-adapter

This is a i18next backend to enable another backend's multiload behaviour of loading multiple lng-ns combinations with one request.
JavaScript
15
star
36

next-app-dir-i18next-no-locale-path-example

JavaScript
11
star
37

i18next-emoji-postprocessor

This is a postProcessor plugin for i18next using in Node.js and in the browser that replaces all words with emojis.
JavaScript
10
star
38

i18next-v4-format-converter

This package helps to convert old i18next translation resources to the new i18next v4 json format.
JavaScript
9
star
39

i18next.com

[obsolete] was replaced by i18next-gitbook! i18next.com website
JavaScript
7
star
40

grunt-i18next

Bundle language resource files for i18next.
JavaScript
6
star
41

i18next-translation-parser

parses i18next translations to AST
JavaScript
6
star
42

i18next-node-remote-backend

[deprecated] can be replaced with i18next-http-backend
JavaScript
6
star
43

omi-i18n

omi-i18n solution for omi.js using i18next ecosystem
JavaScript
5
star
44

i18next-fluent-backend

i18next backend to load fluent formatted .ftl files via xhr
JavaScript
4
star
45

i18next-polyglot

i18nFormat plugin to use airbnb/polyglot.js format with i18next
JavaScript
4
star
46

i18next-cli-app-example

i18next in a cli app
JavaScript
2
star
47

bs-react-i18next

Bucklescript + ReasonReact binding for react-i18next components.
OCaml
1
star