contentful-migration - content model migration tool

Describe and execute changes to your content model and transform entry content.

What is Contentful?

Contentful provides content infrastructure for digital teams to power websites, apps, and devices. Unlike a CMS, Contentful was built to integrate with the modern software stack. It offers a central hub for structured content, powerful management and delivery APIs, and a customizable web app that enable developers and content creators to ship their products faster.

Core Features

• Content type
  • Edit Content type
  • Create a Content type
• Entries
  • Transform Entries for a Given Content type
  • Derives a new entry and sets up a reference to it on the source entry
  • Updates tags on entries for a given Content Type
• Fields
  • Create a field
  • Edit a field
  • Delete a field
  • Rename a field
  • Change a field's control
  • Reset a field's control
  • Copy a field's control
  • Move field
• Tags
  • Create a Tag
  • Rename a Tag
  • Delete a Tag

Pre-requisites && Installation

Pre-requisites

• Node LTS

Installation

npm install contentful-migration

Usage

❗ Usage as CLI

We moved the CLI version of this tool into our Contentful CLI. This allows our users to use and install only one single CLI tool to get the full Contentful experience.

Please have a look at the Contentful CLI migration command documentation to learn more about how to use this as command line tool.

Usage as a library

const { runMigration } = require('contentful-migration')
const options = {
  filePath: '<migration-file-path>',
  spaceId: '<space-id>',
  accessToken: '<access-token>'
}
runMigration(options)
  .then(() => console.log('Migration Done!'))
  .catch((e) => console.error(e))

In your migration description file, export a function that accepts the migration object as its argument. For example:

module.exports = function (migration, context) {
  const dog = migration.createContentType('dog')
  const name = dog.createField('name')
  name.type('Symbol').required(true)
}

You can also pass the function directly. For example:

const { runMigration } = require('contentful-migration')

function migrationFunction(migration, context) {
  const dog = migration.createContentType('dog')
  const name = dog.createField('name')
  name.type('Symbol').required(true)
}

const options = {
  migrationFunction,
  spaceId: '<space-id>',
  accessToken: '<access-token>'
}

runMigration(options)
  .then(() => console.log('Migration Done!'))
  .catch((e) => console.error(e))

Documentation & References

Configuration

Chaining vs Object notation

All methods described below can be used in two flavors:

1. The chained approach:

   const author = migration
     .createContentType('author')
     .name('Author')
     .description('Author of blog posts or pages')

2. The object approach:

   const author = migration.createContentType('author', {
     name: 'Author',
     description: 'Author of blog posts or pages'
   })

   While both approaches work, it is recommended to use the chained approach since validation errors will display context information whenever an error is detected, along with a line number. The object notation will lead the validation error to only show the line where the object is described, whereas the chained notation will show precisely where the error is located.

migration

The main interface for creating and editing content types and tags.

createContentType(id[, opts]) : ContentType

Creates a content type with provided id and returns a reference to the newly created content type.

id : string – The ID of the content type.

opts : Object – Content type definition, with the following options:

• name : string – Name of the content type.
• description : string – Description of the content type.
• displayField : string – ID of the field to use as the display field for the content type. This is referred to as the "Entry title" in the web application.

editContentType(id[, opts]) : ContentType

Edits an existing content type of provided id and returns a reference to the content type. Uses the same options as createContentType.

deleteContentType(id)

Deletes the content type with the provided id and returns undefined. Note that the content type must not have any entries.

transformEntries(config)

For the given content type, transforms all its entries according to the user-provided transformEntryForLocale function. For each entry, the CLI will call this function once per locale in the space, passing in the from fields and the locale as arguments. The transform function is expected to return an object with the desired target fields. If it returns undefined, this entry locale will be left untouched.

config : Object – Content transformation definition, with the following properties:

• contentType : string (required) – Content type ID
• from : array (required) – Array of the source field IDs
• to : array (required) – Array of the target field IDs
• transformEntryForLocale : function (fields, locale, {id}): object (required) – Transformation function to be applied.
  • fields is an object containing each of the from fields. Each field will contain their current localized values (i.e. fields == {myField: {'en-US': 'my field value'}})
  • locale one of the locales in the space being transformed
  • id id of the current entry in scope
    The return value must be an object with the same keys as specified in to. Their values will be written to the respective entry fields for the current locale (i.e. {nameField: 'myNewValue'}). If it returns undefined, this the values for this locale on the entry will be left untouched.
• shouldPublish : bool | 'preserve' (optional) – Flag that specifies publishing of target entries, preserve will keep current states of the source entries (default 'preserve')

transformEntries Example

migration.transformEntries({
  contentType: 'newsArticle',
  from: ['author', 'authorCity'],
  to: ['byline'],
  transformEntryForLocale: function (fromFields, currentLocale, { id }) {
    if (currentLocale === 'de-DE') {
      return
    }
    const newByline = `${fromFields.author[currentLocale]} ${fromFields.authorCity[currentLocale]}`
    return { byline: newByline }
  }
})

For the complete version, please refer to this example.

deriveLinkedEntries(config)

For each entry of the given content type (source entry), derives a new entry and sets up a reference to it on the source entry. The content of the new entry is generated by the user-provided deriveEntryForLocale function. For each source entry, this function will be called as many times as there are locales in the space. Each time, it will be called with the from fields and one of the locales as arguments. The derive function is expected to return an object with the desired target fields. If it returns undefined, the new entry will have no values for the current locale.

config : Object – Entry derivation definition, with the following properties:

• contentType : string (required) – Source content type ID

• derivedContentType : string (required) – Target content type ID

• from : array (required) – Array of the source field IDs

• toReferenceField : string (required) – ID of the field on the source content type in which to insert the reference

• derivedFields : array (required) – Array of the field IDs on the target content type

• identityKey: function (fields): string (required) - Called once per source entry. Returns the ID used for the derived entry, which is also used for de-duplication so that multiple source entries can link to the same derived entry.

  • fields is an object containing each of the from fields. Each field will contain their current localized values (i.e. fields == {myField: {'en-US': 'my field value'}})

• deriveEntryForLocale : function (fields, locale, {id}): object (required) – Function that generates the field values for the derived entry.

  • fields is an object containing each of the from fields. Each field will contain their current localized values (i.e. fields == {myField: {'en-US': 'my field value'}})
  • locale one of the locales in the space being transformed
  • id id of the current entry in scope

  The return value must be an object with the same keys as specified in derivedFields. Their values will be written to the respective new entry fields for the current locale (i.e. {nameField: 'myNewValue'})

• shouldPublish : bool|'preserve' (optional) – If true, both the source and the derived entries will be published. If false, both will remain in draft state. If preserve, will keep current states of the source entries (default true)

deriveLinkedEntries(config) Example

migration.deriveLinkedEntries({
  contentType: 'dog',
  derivedContentType: 'owner',
  from: ['owner'],
  toReferenceField: 'ownerRef',
  derivedFields: ['firstName', 'lastName'],
  identityKey: async (fromFields) => {
    return fromFields.owner['en-US'].toLowerCase().replace(' ', '-')
  },
  shouldPublish: true,
  deriveEntryForLocale: async (inputFields, locale, { id }) => {
    if (locale !== 'en-US') {
      return
    }
    const [firstName, lastName] = inputFields.owner[locale].split(' ')
    return {
      firstName,
      lastName
    }
  }
})

For the complete version of this migration, please refer to this example.

transformEntriesToType(config)

For the given (source) content type, transforms all its entries according to the user-provided transformEntryForLocale function into a new entry of a specific different (target) content type. For each entry, the CLI will call the function transformEntryForLocale once per locale in the space, passing in the from fields and the locale as arguments. The transform function is expected to return an object with the desired target fields. If it returns undefined, this entry locale will be left untouched.

config : Object – Content transformation definition, with the following properties:

• sourceContentType : string (required) – Content type ID of source entries

• targetContentType : string (required) – Targeted Content type ID

• from : array (optional) – Array of the source field IDs, returns complete list of fields if not configured

• identityKey: function (fields): string (required) - Function to create a new entry ID for the target entry

• shouldPublish : bool | 'preserve' (optional) – Flag that specifies publishing of target entries, preserve will keep current states of the source entries (default false)

• updateReferences : bool (optional) – Flag that specifies if linking entries should be updated with target entries (default false). Note that this flag does not support Rich Text Fields references.

• removeOldEntries : bool (optional) – Flag that specifies if source entries should be deleted (default false)

• transformEntryForLocale : function (fields, locale, {id}): object (required) – Transformation function to be applied.

  • fields is an object containing each of the from fields. Each field will contain their current localized values (i.e. fields == {myField: {'en-US': 'my field value'}})
  • locale one of the locales in the space being transformed
  • id id of the current entry in scope

  The return value must be an object with the same keys as specified in the targetContentType. Their values will be written to the respective entry fields for the current locale (i.e. {nameField: 'myNewValue'}). If it returns undefined, this the values for this locale on the entry will be left untouched.

transformEntriesToType Example

const MurmurHash3 = require('imurmurhash')

migration.transformEntriesToType({
  sourceContentType: 'dog',
  targetContentType: 'copycat',
  from: ['woofs'],
  shouldPublish: false,
  updateReferences: false,
  removeOldEntries: false,
  identityKey: function (fields) {
    const value = fields.woofs['en-US'].toString()
    return MurmurHash3(value).result().toString()
  },
  transformEntryForLocale: function (fromFields, currentLocale, { id }) {
    return {
      woofs: `copy - ${fromFields.woofs[currentLocale]}`
    }
  }
})

For the complete version of this migration, please refer to this example.

createTag(id[, opts, visibility])

Creates a tag with provided id and returns a reference to the newly created tag.

• id : string – The ID of the tag.

• opts : Object – Tag definition, with the following options:

  • name : string – Name of the tag.

• visibility : 'private' | 'public' Tag visibility - defaults to private.

editTag(id[, opts])

Edits an existing tag of provided id and returns a reference to the tag. Uses the same options as createTag.

deleteTag(id)

Deletes the tag with the provided id and returns undefined. Note that this deletes the tag even if it is still attached to entries or assets.

setTagsForEntries(config)

For the given content type, updates the tags that are attached to its entries according to the user-provided setTagsForEntry function. For each entry, the CLI will call this function once, passing in the from fields, link objects of all tags that already are attached to the entry and link objects of all tags available in the environment. The setTagsForEntry function is expected to return an array with link objects for all tags that are to be added to the entry. If it returns undefined, the entry will be left untouched.

config : Object – Content transformation definition, with the following properties:

• contentType : string (required) – Content type ID
• from : array (required) – Array of the source field IDs
• setTagsForEntry : function (entryFields, entryTags, apiTags): array (required) – Transformation function to be applied. - entryFields is an object containing each of the from fields. - entryTags is an array containing link objects of all tags already attached to the entry. - apiTags is an array containing link objects of all tags available in the environment.

setTagsForEntries Example

migration.createTag('department-sf').name('Department: San Francisco')
migration.createTag('department-ldn').name('Department: London')

const departmentMapping = {
  'san-francisco': 'department-sf',
  london: 'department-ldn'
}

migration.setTagsForEntries({
  contentType: 'news-article',
  from: ['department'],
  setTagsForEntry: (entryFields, entryTags, apiTags) => {
    const departmentField = entryFields.department['en-US']
    const newTag = apiTags.find((tag) => tag.sys.id === departmentMapping[departmentField])

    return [...entryTags, newTag]
  }
})

context

There may be cases where you want to use Contentful API features that are not supported by the migration object. For these cases you have access to the internal configuration of the running migration in a context object.

module.exports = async function (migration, { makeRequest, spaceId, accessToken }) {
  const contentType = await makeRequest({
    method: 'GET',
    url: `/content_types?sys.id[in]=foo`
  })

  const anyOtherTool = new AnyOtherTool({ spaceId, accessToken })
}

makeRequest(config)

The function used by the migration object to talk to the Contentful Management API. This can be useful if you want to use API features that may not be supported by the migration object.

config : Object - Configuration for the request based on the Contentful management SDK

• method : string – HTTP method
• url : string - HTTP endpoint

module.exports = async function (migration, { makeRequest }) {
  const contentType = await makeRequest({
    method: 'GET',
    url: `/content_types?sys.id[in]=foo`
  })
}

spaceId : string

The space ID that was set for the current migration.

accessToken : string

The access token that was set for the current migration.

Content type

For a comprehensive guide to content modelling, please refer to this guide.

createField(id[, opts]) : Field

Creates a field with provided id.

id : string – The ID of the field.

opts : Object – Field definition, with the following options:

• name : string (required) – Field name.

• type : string (required) – Field type, amongst the following values:

  • Symbol (Short text)
  • Text (Long text)
  • Integer
  • Number
  • Date
  • Boolean
  • Object
  • Location
  • RichText
  • Array (requires items)
  • Link (requires linkType)
  • ResourceLink (requires allowedResources)

• items : Object (required for type 'Array') – Defines the items of an Array field. Example:

  items: {
    type: 'Link',
    linkType: 'Entry',
    validations: [
      { linkContentType: [ 'my-content-type' ] }
    ]
  }

• linkType : string (required for type 'Link') – Type of the referenced entry. Can take the same values as the ones listed for type above.

• allowedResources (required for type 'ResourceLink') - Defines which resources can be linked through the field.

• required : boolean – Sets the field as required.

• validations : Array – Validations for the field. Example:

  validations: [{ in: ['Web', 'iOS', 'Android'] }]

  See The CMA documentation for the list of available validations.

• localized : boolean – Sets the field as localized.

• disabled : boolean – Sets the field as disabled, hence not editable by authors.

• omitted : boolean – Sets the field as omitted, hence not sent in response.

• deleted : boolean – Sets the field as deleted. Requires to have been omitted first. You may prefer using the deleteField method.

• defaultValue : Object – Sets the default value for the field. Example:

  defaultValue: {
    "en-US": false,
    "de-DE": true
  }

editField(id[, opts]) : Field

Edits the field of provided id.

id : string – The ID of the field to edit.

opts : Object – Same as createField listed above.

deleteField(id) : void

Shorthand method to omit a field, publish its content type, and then delete the field. This implies that associated content for the field will be lost.

id : string – The ID of the field to delete.

changeFieldId (currentId, newId) : void

Changes the field's ID.

currentId : string – The current ID of the field.

newId : string – The new ID for the field.

moveField (id) : MovableField

Move the field (position of the field in the web editor)

id: string - The ID of the field to move

.moveField(id) returns a movable field type which must be called with a direction function:

• .toTheTop()
• .toTheBottom()
• .beforeField(fieldId)
• .afterField(fieldId)

Example:

module.exports = function (migration) {
  const food = migration.editContentType('food')

  food.createField('calories').type('Number').name('How many calories does it have?')

  food.createField('sugar').type('Number').name('Amount of sugar')

  food.createField('vegan').type('Boolean').name('Vegan friendly')

  food.createField('producer').type('Symbol').name('Food producer')

  food.createField('gmo').type('Boolean').name('Genetically modified food')

  food.moveField('calories').toTheTop()
  food.moveField('sugar').toTheBottom()
  food.moveField('producer').beforeField('vegan')
  food.moveField('gmo').afterField('vegan')
}

changeFieldControl (fieldId, widgetNamespace, widgetId[, settings]) : void

Changes control interface of given field's ID.

fieldId : string – The ID of the field.

widgetNamespace : string – The namespace of the widget, one of the following values:

• builtin (Standard widget)
• app (Custom App)
• extension (Custom UI extension)
• app (Custom app widget)

widgetId : string – The new widget ID for the field. See the editor interface documentation for a list of available widgets.

settings : Object – Widget settings and extension instance parameters. Key-value pairs of type (string, number | boolean | string). For builtin widgets, the the following options are available:

• helpText : string – This help text will show up below the field.
• trueLabel : string (only for fields of type boolean) – Shows this text next to the radio button that sets this value to true. Defaults to “Yes”.
• falseLabel : string (only for fields of type boolean) – Shows this text next to the radio button that sets this value to false. Defaults to “No”.
• stars : number (only for fields of type rating) – Number of stars to select from. Defaults to 5.
• format : string (only for fields of type datePicker) – One of “dateonly”, “time”, “timeZ” (default). Specifies whether to show the clock and/or timezone inputs.
• ampm : string (only for fields of type datePicker) – Specifies which type of clock to use. Must be one of the strings “12” or “24” (default).
• bulkEditing : boolean (only for fields of type Array) – Specifies whether bulk editing of linked entries is possible.
• trackingFieldId : string (only for fields of type slugEditor) – Specifies the ID of the field that will be used to generate the slug value.
• showCreateEntityAction : boolean (only for fields of type Link) - specifies whether creation of new entries from the field is enabled.
• showLinkEntityAction : boolean (only for fields of type Link) - specifies whether linking to existing entries from the field is enabled.

resetFieldControl (fieldId) : void

fieldId : string – The ID of the field.

copyFieldControl (sourceFieldId, destinationFieldId) : void

sourceFieldId : string – The ID of the field to copy the control setting from. destinationFieldId : string – The ID of the field to apply the copied control setting to.

addSidebarWidget (widgetNamespace, widgetId[, settings, insertBeforeWidgetId]) : void

Adds a builtin or custom widget to the sidebar of the content type.

widgetNamespace: string – The namespace of the widget, one of the following values:

• sidebar-builtin (Standard widget, default)
• extension (Custom UI extension)

widgetId : string – The ID of the builtin or extension widget to add.

settings : Object – Instance settings for the widget. Key-value pairs of type (string, number | boolean | string)

insertBeforeWidgetId : Object – Insert widget above this widget in the sidebar. If null, the widget will be added to the end.

updateSidebarWidget (widgetNamespace, widgetId, settings) : void

Updates the configuration of a widget in the sidebar of the content type.

widgetNamespace: string – The namespace of the widget, one of the following values:

• sidebar-builtin (Standard widget, default)
• extension (Custom UI extension)

widgetId : string – The ID of the builtin or extension widget to add.

settings : Object – Instance settings for the widget. Key-value pairs of type (string, number | boolean | string)

removeSidebarWidget (widgetNamespace, widgetId) : void

Removes a widget from the sidebar of the content type.

widgetNamespace: string – The namespace of the widget, one of the following values:

• sidebar-builtin (Standard widget, default)
• extension (Custom UI extension)

widgetId : string – The ID of the builtin or extension widget to remove.

resetSidebarToDefault () : void

Resets the sidebar of the content type to default.

configureEntryEditor (widgetNamespace, widgetId[, settings]) : void

Sets the entry editor to specified widget.

widgetNamespace: string – The namespace of the widget. widgetId : string – The ID of the builtin or extension widget to add. settings : Object – Instance settings for the widget. Key-value pairs of type (string, number | boolean | string). Optional.

configureEntryEditors (EntryEditor[]) : void

As opposed to configureEntryEditor which only sets one editor, this sets a list of editors to the current editor interface of a content-type.

Each EntryEditor has the following properties:

• widgetNamespace: string – The namespace of the widget (i.e: app, extension or builtin-editor).
• widgetId : string – The ID of the builtin, extension or app widget to add.
• settings : Object – Instance settings for the widget. Key-value pairs of type (string, number | boolean | string). Optional.

resetEntryEditorToDefault () : void

Resets the entry editor of the content type to default.

createEditorLayout () : EditorLayout

Creates an empty editor layout for this content type.

editEditorLayout () : EditorLayout

Edits the editor layout for this content type.

deleteEditorLayout () : void

Deletes the editor layout for this content type.

setAnnotations(AnnotationId[])

Configure the annotations assigned to this content type. See annotations documentation for more details on valid AnnotationId.

clearAnnotations()

Remove all assigned annotations from this content type

Field

The field object has the same methods as the properties listed in the ContentType.createField method.

In addition the following methods allow to manage field annotations.

setAnnotations(AnnotationId[])

Configure the annotations assigned to this field. See annotations documentation for more details on valid AnnotationId.

clearAnnotations()

Remove all assigned annotations from this field.

Editor Layout

moveField(id) : MovableEditorLayoutItem

Moves the field with the provided id.

moveField(id) returns a movable editor layout item type which must be called with a direction function:

• .toTheTopOfFieldGroup(groupId)
  • • if no groupId is provided, the field will be moved within its group
• .toTheBottomOfFieldGroup(groupId)
  • • if no groupId is provided, the field will be moved within its group
• .beforeFieldGroup(groupId)
• .afterFieldGroup(groupId)
• .beforeField(fieldId)
• .afterField(fieldId)

createFieldGroup(id[, opts]) : EditorLayoutFieldGroup

Creates a tab with the provided id.

id : string – The ID of the group.

opts : Object – Group settings, with the following options:

• name : string (required) – Group name.

deleteFieldGroup (id) : void

Deletes the group with the provided id from the editor layout, moving its contents to the parent if the group to delete is a field set or to the default tab if it’s a tab.

changeFieldGroupId (currentId, newId)

Changes the group’s ID.

currentId : string – The current ID of the group.

newId : string – The new ID for the group.

editFieldGroup (id[, opts]) : EditorLayoutFieldGroup

Editor Layout Field Group

createFieldGroup (id[, opts]) : EditorLayoutFieldGroup

Creates a field set with the provided id.

id : string – The ID of the group.

opts : Object – Group settings, with the following options:

• name : string (required) – Group name.

changeFieldGroupControl (id, widgetNamespace, widgetId[, settings]) : void

Sets the group control for a field group.

widgetNamespace : string – The namespace for the group control. Currently allowed: builtin. widgetId : string - The widget ID for the group control. Allowed values: fieldset, topLevelTab. settings : Object – Field set settings, with the following properties:

• helpText : string – Help text for the field set. Displayed when editing.
• collapsible : boolean – Whether the field set can be collapsed when editing.
• collapsedByDefault : string – Whether the field set is collapsed when opening the editor.

Validation errors

You can learn more from the possible validation errors here.

Example migrations

You can check out the examples to learn more about the migrations DSL. Each example file is prefixed with a sequence number, specifying the order in which you're supposed to run the migrations, as follows:

const runMigration = require('contentful-migration/built/bin/cli').runMigration

const options = {
  spaceId: '<space-id>',
  accessToken: '<access-token>',
  yes: true
}

const migrations = async () => {
  await runMigration({ ...options, ...{ filePath: '01-angry-dog.js' } })
  await runMigration({ ...options, ...{ filePath: '02-friendly-dog.js' } })
  await runMigration({ ...options, ...{ filePath: '03-long-example.js' } })
  await runMigration({ ...options, ...{ filePath: '04-steps-errors.js' } })
  await runMigration({ ...options, ...{ filePath: '05-plan-errors.js' } })
  await runMigration({ ...options, ...{ filePath: '06-delete-field.js' } })
  await runMigration({ ...options, ...{ filePath: '07-display-field.js' } })
}

migrations()

Writing Migrations in Typescript

You can use Typescript to write your migration files using ts-node! First npm install --save ts-node typescript, then run your migration with ts-node:

node_modules/.bin/ts-node node_modules/.bin/contentful-migration -s $CONTENTFUL_SPACE_ID -a $CONTENTFUL_MANAGEMENT_TOKEN my_migration.ts

An example Typescript migration:

import { MigrationFunction } from 'contentful-migration'

// typecast to 'MigrationFunction' to ensure you get type hints in your editor
export = function (migration, { makeRequest, spaceId, accessToken }) {
  const dog = migration.createContentType('dog', {
    name: 'Dog'
  })

  const name = dog.createField('name')
  name.name('Name').type('Symbol').required(true)
} as MigrationFunction

Here's how it looks inside VS Code:

Troubleshooting

• Unable to connect to Contentful through your Proxy? Try to set the rawProxy option to true.

runMigration({
  proxy: 'https://cat:[email protected]:1234',
  rawProxy: true,
  ...
})

Updating Integration tests fixtures

• To add new/update integration tests, you need to set environment variable NOCK_RECORD=1 which should automatically update fixtures

Reach out to us

You have questions about how to use this library?

• Reach out to our community forum:
• Jump into our community slack channel:

You found a bug or want to propose a feature?

• File an issue here on GitHub: . Make sure to remove any credential from your code before sharing it.

You need to share confidential information or have other questions?

• File a support ticket at our Contentful Customer Support:

Get involved

We appreciate any help on our repositories. For more details about how to contribute see our CONTRIBUTING.md document.

License

This repository is published under the MIT license.

Code of Conduct

We want to provide a safe, inclusive, welcoming, and harassment-free space and experience for all participants, regardless of gender identity and expression, sexual orientation, disability, physical appearance, socioeconomic status, body size, ethnicity, nationality, level of experience, age, religion (or lack thereof), or other identity markers.