SFDX-Git-Delta
SFDX plugin to generate Incremental Salesforce deployments manifests and artifacts
Report Bug
ยท
Request Feature
Table of Contents
- TL;DR
- What is SFDX-Git-Delta?
- Is SGD for you?
- Getting Started
- How to use it?
sfdx sgd:source:delta -f <string> [-t <string>] [-r <filepath>] [-i <filepath>] [-D <filepath>] [-s <filepath>] [-W] [-o <filepath>] [-a <number>] [-d] [-n <filepath>] [-N <filepath>] [--json] [--loglevel trace|debug|info|warn|error|fatal|TRACE|DEBUG|INFO|WARN|ERROR|FATAL]
- Walkthrough
- Advanced use-cases
- Generate a folder containing only the added/modified sources
- Exclude some metadata only from destructiveChanges.xml
- Explicitly including specific files for inclusion or destruction regardless of diff
- Scoping delta generation to a specific folder
- Generate a comma-separated list of the added and modified Apex classes
- Condition deployment on package.xml and destructiveChange content
- Use the module in your own node application
- Changelog
- Built With
- Versioning
- Authors
- Contributing
- License
TL;DR
sfdx plugins:install sfdx-git-delta
sfdx sgd:source:delta --to "HEAD" --from "HEAD~1" --output "."
sfdx force:source:deploy -x package/package.xml --postdestructivechanges destructiveChanges/destructiveChanges.xml
What is SFDX-Git-Delta?
SFDX-Git-Delta (a.k.a. SGD) helps Salesforce Architects and Developers do 2 things with their source deployments:
-
Make deployments faster: identify the changed metadata since a reference commit.
-
Automate destructive deployments: build the destructiveChanges.xml from the deleted (or renamed) metadata
Have a look at this post on the Salesforce Developers Blog to dive into it: Optimizing Unpackaged Deployments Using a Delta Generation Tool.
Is SGD for you?
If you are not a Salesforce Architect or Developer, probably not, sorry.
If you are a Technical Architect or Developer, then itโs a very useful tool for you, when meeting the 3 conditions below:
- Your Salesforce project uses a git repo as the source of truth.
- You use the Source (DX) format in the repo.
- Your metadata is unmanaged (in other words, you are not building a managed or unlocked package).
SGD is designed to be part of a CI/CD pipeline (Jenkins, Bitbucket Pipelines, GitLab CI, GitHub Actions, Azure DevOps...) that handles the deployment of the sources to the Salesforce org(s).
Pro tip: Make sure your pipeline works before implementing incremental deployments. Otherwise it will just make it harder to debug your pipeline. It's also important to implement a way to switch back to full deployment in case the incremental deployment does not behave as expected.
DISCLAIMER:
Getting Started
Prerequisites
The plugin requires git command line on the running environment.
Node v16.20.0 or above is required.
To check if Salesforce CLI runs under a supported node version for SGD, run sfdx --version
. You should see a node version above v.16.20.0 to use SGD.
If you encounter this issue whereas the node version is OK on the running environment, try to install the Salesforce CLI via npm (npm install sfdx-cli --global
).
Installation
SGD is a Salesforce CLI plugin (sfdx sgd:source:delta
). Run the following command to install it:
sfdx plugins:install sfdx-git-delta
Because this plugin is not signed, you will get a warning saying that "This plugin is not digitally signed and its authenticity cannot be verified". This is expected, and you will have to answer y
(yes) to proceed with the installation.
If you run your CI/CD jobs inside a Docker image, you can add the plugin to your image (such as in this example). If you use GitHub Actions, you can find some examples of using SGD here.
โ ๏ธ The Salesforce CLI plugin is now the only supported way to install SGD. There used to be another way to install it using yarn or npm. The legacy sgd
command is now deprecated and decommissioned.
How to use it?
sfdx sgd:source:delta -f <string> [-t <string>] [-r <filepath>] [-i <filepath>] [-D <filepath>] [-s <filepath>] [-W] [-o <filepath>] [-a <number>] [-d] [-n <filepath>] [-N <filepath>] [--json] [--loglevel trace|debug|info|warn|error|fatal|TRACE|DEBUG|INFO|WARN|ERROR|FATAL]
Generate the sfdx content in source format and destructive change from two git commits
USAGE
$ sfdx sgd:source:delta -f <string> [-t <string>] [-r <filepath>] [-i <filepath>] [-D <filepath>] [-s <filepath>] [-W]
[-o <filepath>] [-a <number>] [-d] [-n <filepath>] [-N <filepath>] [--json] [--loglevel
trace|debug|info|warn|error|fatal|TRACE|DEBUG|INFO|WARN|ERROR|FATAL]
OPTIONS
-D, --ignore-destructive=ignore-destructive file listing paths to explicitly
ignore for any destructive actions
-N, --include-destructive=include-destructive file listing paths to explicitly
include for any destructive actions
-W, --ignore-whitespace ignore git diff whitespace (space,
tab, eol) changes
-a, --api-version=api-version salesforce metadata API version,
default to sfdx-project.json
"sourceApiVersion" attribute or
latest version
-d, --generate-delta generate delta files in [--output]
folder
-f, --from=from (required) commit sha from where the
diff is done [git rev-list
--max-parents=0 HEAD]
-i, --ignore=ignore file listing paths to explicitly
ignore for any diff actions
-n, --include=include file listing paths to explicitly
include for any diff actions
-o, --output=output [default: ./output] source package
specific output
-r, --repo=repo [default: ./] git repository
location
-s, --source=source [default: ./] source folder focus
location related to --repo
-t, --to=to [default: HEAD] commit sha to where
the diff is done
--json format output as json
--loglevel=(trace|debug|info|warn|error|fatal|TRACE|DEBUG|INFO|WARN|ERROR|FATAL) [default: warn] logging level for
this command invocation
See code: src/commands/sgd/source/delta.ts
Windows users
If you run SGD on a Windows system, use double quotes to prevent the terminal to interpret parameters
You should also avoid using the "^" character (shorthand for parent commit in git) because it is the escape character in Windows. So instead of:
sfdx sgd:source:delta --from "HEAD^" # wrong git shortcut with windows because it uses "^" syntax
You should write:
sfdx sgd:source:delta --from "HEAD~1" # right git shortcut with windows because it does not use "^", it uses "~n" syntax
CI/CD specificity
In CI/CD pipelines, for most of the CI/CD providers, the checkout operation fetch only the last commit of the branch currently evaluated. You need to fetch all the needed commits, as the plugin needs to have the branch to compare from as well, Example for Github action checkout here.
In CI/CD pipelines, branches are not checked out locally when the repository is cloned, so you must specify the remote prefix.
If you do not specify the remote in CI context, the git pointer check will raise an error (as the branch is not created locally).
This applies to both --from
and --to
parameters as they both accept git pointers.
Example comparing HEAD
with a development
branch when the CI clone the repository with origin
set as reference to the remote:
sfdx sgd:source:delta --to "HEAD" --from "origin/development" --output .
Use global variable when you need to easily switch sgd version (vX.X.X
format) or channel (stable
, latest
, latest-rc
) in your pipeline, without having to commit a new version of your pipeline.
Example with github action, create a variable SGD_VERSION and use it in the plugin installation phase
- name: Install SGD
run: echo y | sfdx plugins:install "sfdx-git-delta@${{ vars.SGD_VERSION }}"
Use cases
Any git sha pointer is supported: commit sha, branch, tag, git expression (HEAD, etc.).
Here are examples of how to compare the content of different branches:
- Comparing between commits in different branches
For example, if you have commit
fbc3ade6
in branchdevelop
and commit61f235b1
in branchmain
:
sfdx sgd:source:delta --to fbc3ade6 --from 61f235b1 --output .
- Comparing branches (all changes)
Comparing all changes between the
develop
branch and themain
branch:
sfdx sgd:source:delta --to develop --from main --output .
- Comparing branches (from a common ancestor)
To compare the
develop
branch since its common ancestor with themain
branch (i.e. ignoring the changes performed in themain
branch afterdevelop
creation):
sfdx sgd:source:delta --to develop --from $(git merge-base develop main) --output .
Walkthrough
Letโs take a look at the following scenario:
The CI pipelines deploys the sources to Production anytime there is a new commit in the main branch.
In our example, the latest commit to main is composed of:
- Apex Class added: TriggerHandler
- Apex Class added: TriggerHandler_Test
- Apex Class modified: TestDataFactory
- Apex Class deleted: AnotherTriggerFramework
In this situation, we would expect the CI pipeline to:
- Deploy to Production only 3 classes (no matter how much metadata is present in the force-app folder):
TriggerHandler
,TriggerHandler_Test
, andTestDataFactory
- Delete from Production 1 class:
AnotherTriggerFramework
So letโs do it!
Execute sgd
From the project repo folder, the CI pipeline will run the following command:
sfdx sgd:source:delta --to "HEAD" --from "HEAD~1" --output .
which means:
Analyze the difference between HEAD (latest commit) and HEAD~1 (previous commit), and output the result in the current folder.
The sfdx sgd:source:delta
command produces 2 useful artifacts:
1) A package.xml
file, inside a package
folder. This package.xml
file contains just the added/changed metadata to deploy to the target org.
Content of the package.xml
file in our scenario:
2) A destructiveChanges.xml
file, inside a destructiveChanges
folder. This destructiveChanges.xml
file contains just the removed/renamed metadata to delete from the target org. Note: the destructiveChanges
folder also contains a minimal package.xml file, because deploying destructive changes requires a package.xml (even an empty one).
Content of the destructiveChanges.xml
file in our scenario:
Note: it is also possible to generate a source folder containing added/changed metadata with the --generate-delta (-d)
parameter. See the "Advanced use-cases" section for more examples.
Deploy the delta metadata
The simplest option to deploy the incremental changes is to use force:source:deploy
command with -x
parameter:
sfdx force:source:deploy -x package/package.xml --postdestructivechanges destructiveChanges/destructiveChanges.xml
And voilร ! ๐ฅณ
However, keep in mind that the above command will fail if the destructive change was supposed to be executed before the deployment (i.e. as --predestructivechanges
), or if a warning occurs during deployment. Make sure to protect your CI/CD pipeline from those scenarios, so that it doesn't get stuck by a failed destructive change.
If needed, you can also split the added/modified metadata deployment from the deleted/renamed metadata deployment, as in the below examples:
Use the package/package.xml
file to deploy only the added/modified metadata:
echo "--- package.xml generated with added and modified metadata ---"
cat package/package.xml
echo
echo "---- Deploying added and modified metadata ----"
sfdx force:source:deploy -x package/package.xml
Use the destructiveChanges
folder to deploy only the destructive changes:
echo "--- destructiveChanges.xml generated with deleted metadata ---"
cat destructiveChanges/destructiveChanges.xml
echo
echo "--- Deleting removed metadata ---"
sfdx force:mdapi:deploy -d destructiveChanges --ignorewarnings
Advanced use-cases
Generate a folder containing only the added/modified sources
Using a package.xml for deployment is the simplest approach to delta deployments. But in some cases you may want to have only the actual recently changed source files.
One example is to speed up object deployments: the package.xml approach will deploy the entire sub-folder for a given object. Having a copy of the actual sources added/modified allows you to deploy only those components.
This is where the --generate-delta (-d)
option comes handy!
Let's use this option with our previous example:
mkdir changed-sources
sfdx sgd:source:delta --to "HEAD" --from "HEAD~1" --output changed-sources/ --generate-delta
It generates the package
and destructiveChanges
folders, and copies added/changed files in the output folder.
Content of the output folder when using the --generate-delta option, with the same scenario as above:
โ ๏ธ Use--generate-delta (-d)
when--to (-t)
value is set to "HEAD" or to the "HEAD commit SHA". If you need to use it with--to (-t)
pointing to another commit than "HEAD", checkout that commit first. Example:# move HEAD to the wanted past commit $ git checkout <not-HEAD-commit-sha> # You can omit --to, it will take "HEAD" as default value $ sfdx sgd:source:delta --from "HEAD~1" --output changed-sources/ --generate-delta
Then it is possible to deploy the change-sources
folder using force:source:deploy
command with -p
parameter:
sfdx force:source:deploy -p change-sources
Exclude some metadata only from destructiveChanges.xml
The --ignore [-i]
parameter allows you to specify an ignore file to filter the
element on the diff to ignore. SGD ignores every diff line matching the pattern from the ignore file specified in the --ignore [-i]
. package.xml
generation, destructiveChanges.xml
generation and --delta-generate
will ignore those lines.
Sometimes you may need to have two different ignore policies. One for the package.xml
and another one for destructiveChanges.xml
files. This is where the --ignore-destructive [-D]
option comes handy!
Use the --ignore-destructive
parameter to specify a dedicated ignore file to handle deletions. It will apply to metadata listed in the destructiveChanges.xml
. In other words, this will override the --ignore [-i]
parameter for deleted items.
Consider the following:
- a repository containing many sub-folders (force-app/main, force-app/sample, etc)
- a commit deleting the Custom__c object from one folder and modifying the Custom__c object from another folder. This is a Modification and a Deletion events.
The Custom__c object appears in the package.xml
and in destructiveChanges.xml
and fail the deployment. This is a situation where your may want to use the --ignore-destructive [-D]
parameter! Add the Custom__c object pattern in an ignore file and pass it in the CLI parameter:
# destructiveignore
*Custom\_\_c.object-meta.xml
$ sfdx sgd:source:delta --from commit --ignore-destructive destructiveignore
Note: when only using the --ignore [-i]
parameter (and not --ignore-destructive [-D]
) the plugin will apply it to added/changed/deleted elements.
Explicitly including specific files for inclusion or destruction regardless of diff
The --include [-n]
parameter allows you to specify a file based on micromatch glob matching to include specific files. Regardless whether they appears in the diff or not.
Like the --ignore
flag, this file defines a list of glob file matchers to always include git
aware files in the package.xml
package.
SGD will include every line matching the pattern from the include file specified in the --include [-n]
.
As with --ignore
, you may need different policies for the package.xml
and destructiveChanges.xml
files. This is where the --include-destructive [-N]
option comes handy!
Use the --include-destructive
parameter to specify a dedicated include file to handle deletions. Related metadata will appear in the destructiveChanges.xml
output. Here, you will show which files should the destructiveChanges.xml
should include .
Consider the following:
- a repository containing many sub-folders (force-app/main,force-app/sample, etc)
- a CI/CD platform generating a
force-app/generated/foo
file thesource:deploy
command should not include. You can create a file with a line matching this new file and specify this file using the--include-destructive [-N]
parameter.
# .destructiveinclude
*generated/foo
$ sfdx sgd:source:delta --from commit --include-destructive .destructiveinclude
The path matchers in includes file must follow gitignore
spec and accept only unix path separator /
(even for windows system).
Scoping delta generation to a specific folder
The --source [-s]
parameter allows you to specify a folder to focus on, making any other folder ignored.
It means the delta generation will only focus on the dedicated folder.
For example, consider a repository containing many sub-folders (force-app/package, force-app/unpackaged, etc). This repository contains packaged (deployed via package) and unpackaged (deployed via CLI) sources. You only want to apply delta generation for the unpackaged sources.
$ tree
.
โโโ force-app
โโโ packaged
โ โโโ classes
โ โโโ PackagedClass.cls
โโโ unpackaged
โโโ classes
โโโ UnpackagedClass.cls
โโโ ...
# scope the delta generation only to the unpackaged folder
$ sfdx sgd:source:delta --from commit --source force-app/unpackaged
The ignored patterns specified using
--ignore [-i]
and--ignore-destructive [-D]
still apply. The--source
path must be relative to the--repo
path
Generate a comma-separated list of the added and modified Apex classes
Depending on your testing strategy, you may want to generate a comma-separated list of the added and modified Apex classes. This list can feed the sfdx force:source:deploy --testlevel RunSpecifiedTests
command, for example.
To cover this need, parse the content of the package.xml file produced by SGD using yq:
xq . < package/package.xml | jq '.Package.types | [.] | flatten | map(select(.name=="ApexClass")) | .[] | .members | [.] | flatten | map(select(. | index("*") | not)) | unique | join(",")'
Condition deployment on package.xml and destructiveChange content
SGD does not always generate content in the package.xml (or destructiveChanges.xml). Sometimes the commit range contains changes only within files to ignore (using .sgdignore and --i
parameter).
Deploying empty package.xml can lead to deployment errors.
To avoid starting a failing deployment, test files content before execution:
# run deploy command only if the generated package contains metadata
if grep -q '<types>' ./package/package.xml ; then
echo "---- Deploying added and modified metadata ----"
sfdx force:source:deploy -x package/package.xml
else
echo "---- No changes to deploy ----"
fi
Use the module in your own node application
If you want to embed sgd in your node application, install it as a dependency for your application
yarn add sfdx-git-delta
Then use the JavaScript module
// sample/app.js
const sgd = require('sfdx-git-delta')
const work = await sgd({
to: '', // commit sha to where the diff is done. [default : "HEAD"]
from: '', // (required) commit sha from where the diff is done. [default : git rev-list --max-parents=0 HEAD]
output: '', // source package specific output. [default : "./output"]
apiVersion: '', // salesforce API version. [default : latest]
repo: '', // git repository location. [default : "."]
})
console.log(JSON.stringify(work))
/* {
* config: config,
* diffs: { package: {...}, destructiveChanges: {...} },
* warnings: []
* }
*/
Changelog
changelog.md is available for consultation.
Built With
- fast-xml-parser - Validate XML, Parse XML to JS/JSON and vise versa, or parse XML to Nimn rapidly without C/C++ based libraries and no callback
- fs-extra - Node.js: extra methods for the fs object like copy(), remove(), mkdirs().
- ignore - is a manager, filter and parser which implemented in pure JavaScript according to the .gitignore spec 2.22.1.
- lodash - A modern JavaScript utility library delivering modularity, performance & extras.
- micromatch - a file glob matcher utility
- xmlbuilder2 - An XML builder for node.js.
- MegaLinter - Open-Source tool for CI/CD workflows that analyzes the consistency of your code, IAC, configuration, and scripts
Versioning
Versioning follows SemVer specification.
Authors
Contributing
Contributions are what make the trailblazer community such an amazing place. I regard this component as a way to inspire and learn from others. Any contributions you make are appreciated.
See contributing.md for sgd contribution principles.
License
This project license is MIT - see the LICENSE.md file for details