compiling-unreal

A man's notes as he discovers how Unreal Engine compiles things

Google Slides

This is available in google slides form here, which might either contain more information or less information than this repo because honestly I forgot how I took these notes.

https://docs.google.com/presentation/d/1jRY2raIiX2AlBRhm5b8I98a93de6lihYSmm99asAMfo/edit?usp=sharing

Support

If you have gotten any use out of this, please consider becoming one of my GitHub Sponsors. Even a $1 is very appreciated.

No Warranty

I make no claims that any of this information is accurate, correct, or relevant in any way. It definitely won't ever be completed.

Licensing

MIT, please contribute knowledge if this helps you discover any. Please.

Build Pipeline Tools

• Build Scripts
  • Build.bat
• DotNETUtilities
• UnrealBuildTool
• UnrealHeaderTool

Build Scripts — Build.bat

• Wrapper for UnrealBuildTool.exe
• Ensures that the current working directory is set to the Engine’s Source folder
• Checks to see if UnrealBuildTool.exe exists and errors out if it doesn’t
• Otherwise just passes all command line args to UnrealBuildTool.exe

DotNETUtilities

• Provides general utilities for all DotNET tools for Unreal Engine
• Contains things like:
  • A JSON library
  • A library for working with Perforce
  • File Handling
  • Logging
  • Process management and queuing
  • Command Line Arg Processing

Visual Studio Notes

• Project files generated in Engine\Intermediate\ProjectFiles
• This also appears to be the working directory for NMake/Build commands

How To Debug UnrealBuildTool in the Development Editor Configuration

• Why would you want to debug while using the Development Editor configuration?
  • I am hella lazy and I don’t want to keep toggling that Debug box
• Do the following to DotNETUtilities and UnrealBuildTool:
  • In Project Build Properties: Define DEBUG Constant, Disable Optimize Code
  • Set Debugging Information in the Output Advanced window to Full
• Use Entrian Attach to attach to Engine\Binaries\DotNET\UnrealBuildTool.exe
  • Otherwise start up the UnrealBuildTool project using command line args like: UnrealHeaderTool Win64 Development -WaitMutex -FromMsBuild

UnrealBuildTool

The following section is centered on the mechanisms specific to the UnrealBuildTool.

Building a Target

When you right-click a project in Visual Studio and click Build, this is ultimately what you're doing.

Visual Studio calls the Build.bat script with command-line arguments representing your desired build target. It does this in the form of:

Build.bat TargetName TargetPlatform Configuration OptionalAdditionalArgs

For example, when building the UnrealHeaderTool directly, you invoke:

Build.bat UnrealHeaderTool Win64 Development -WaitMutex -FromMsBuild

Fun fact: -FromMsBuild means use MSBuild Message Format for warnings and errors, which is a format that Visual Studio understands allowing for navigating directly to the error. That format takes the form of the following DotNet Regular Expression:

(?'FILENAME'.+):(?'LINE'\d+):(?'COLUMN'\d+): (?'CATEGORY'error|warning): (?'TEXT'.*)

Important Objects To Know About

Feel free to skip this list and come back to it when you come across an object term or type you don't know. It'll hopefully be in here.

#TODO: Make a table of contents that links to the right shit

• SingleInstanceMutexes

SingleInstanceMutexes

UnrealBuildTool uses SingleInstanceMutexes to ensure that no two UnrealBuildTool instances are going to stomp each other and compete for the same resources. This uses DotNet Mutexes that are named based on your Engine instance, so you can compile two different Engines at once, but you can't build two targets within the same Engine simultaneously.

• SingleInstanceMutex(string MutexName, bool bWaitMutex)
  • Creates Mutexes with formatted string: Global\\{0}_{1}
    • {0}: Mutex Name
    • {1}: DotNet String GetHashCode()
    • i.e. Global\UnrealBuildTool_Mutex_XmlConfig_-2108754181
• Used Mutex Names:
  • UnrealBuildTool_Mutex_XmlConfig, Assembly.GetExecutingAssembly().CodeBase
    • Used when reading XmlConfig files on UnrealBuildTool startup
  • UnrealBuildTool_Mutex, Assembly.GetExecutingAssembly().CodeBase
    • Used when actually building
  • UnrealBuildTool_WriteMetadata, UnrealBuildTool.RootDirectory.FullName
    • Used when UnrealBuildTool runs UnrealBuildTool in WriteMetaData mode as part of a build so that DebugGame and DevelopmentEditor .module files can be written in parallel

XMLConfigFiles

The UnrealBuildTool can be configured in many ways. One of them is through configuration files in .xml form. These are generally global configuration options that affect the entire build pipeline as opposed to settings you might want to be passed to tools via command-line. Like Unreal's .ini configuration system, there are multiple sources and layers to XMLConfigFiles.

XML Config Cache Validation

1. Skipped if XMLConfigCache= is passed
2. Non-Installed Engine, DefaultConfigFile is Engine\Saved\UnrealBuildTool\BuildConfiguration.xml
3. Installed Engine DefaultConfigFile is %localappdata%\UnrealEngine\XmlConfigCache-(EnginePath).bin
4. Global AppData override of UnrealBuildtool Configuration: %APPDATA%\Unreal Engine\UnrealBuildTool\BuildConfiguration.XML
5. Global Documents override of UnrealBuildtool Configuration?: My Documents\Unreal Engine\UnrealBuildTool\BuildConfiguration.XML
6. Reads XML Schema: Engine\Saved\UnrealBuildTool\BuildConfiguration.Schema.xsd
7. Checks to see if XMLConfigCache is up to date (UnrealBuildTool_Mutex_XmlConfig)
   1. Checks to see if UnrealBuildTool’s executable is newer than cache, if so, rebuild cache
   2. Checks to see if config files in cache’s timestamps are newer than cache timestamp as well as a schema validation check
   3. If not, generates a Dictionary<string (CategoryName), Dictionary<string (FieldName), FieldInfo>> of all UnrealBuildTool C# types that have a field marked XMLConfigFileAttribute
      1. Non-installed engines will write out schema
      2. All XML files in the config cache listed above are read and validated against the schema
      3. All config values saved in the config cache file

UnrealBuildTool's Config Hierarchy / Config Layers

Practical Knowledge

• {TYPE} refers to a Config Type Name that is passed in when merging config files. The “Engine” or “Game” or “Editor” or “Input” in DefaultEngine/DefaultGame/DefaultEditor/DefaultInput.ini
• Platform becomes a config suitable name like “Windows” and is not the same as a Target Platform name which we’ll see later “Win64”
• Note: All paths with Platform in them will first check to see if Engine\Platforms\PlatformName exists before looking into the normal Engine/Config/Platform folder. This allows for easier NDA and “add on” platform extensions, both config wise and code/content wise.
• Projects with Platform can also be extended in this way with their extensions in Project\Platforms instead of Engine\Platforms

1. Non-User Specific Settings

   Path	Notes
   Engine/Base.ini	Empty, but required to exist
   Engine/Base{TYPE}.ini
   Engine/Platform/BasePlatform{TYPE}.ini	i.e. Engine/Windows/BaseWindows*.ini
   Project/Default{TYPE}.ini	i.e. see image to the right
   Engine/Platform/Platform{TYPE}.ini	i.e. Engine/Windows/Windows*.ini
   Project/Platform/Platform{TYPE}.ini	i.e. Project/Windows/Windows*.ini

2. User Specific Settings

   Path	Notes
   UserSettings/.../User{TYPE}.ini	This no longer exists? i.e. AppData/Local/Unreal Engine/Engine/Config/UserEngine.ini
   UserDir/.../User{TYPE}.ini	This no longer exists? i.e. Documents/Unreal Engine/Engine/Config/UserEngine.ini
   Project/User{TYPE}.ini	i.e. Project/User*.ini

Technical Knowledge

This is how the Config paths are generated.

Tokens

• {ENGINE} = Absolute path to the Engine
  • D:\UE425\Engine
• {PROJECT} = Absolute path to Project
• {PLATFORM} = See slide on Platform
• {TYPE} = Requested .ini type
  • Engine, Game, Input, etc.
• {EXTENGINE} = Extension folder
  • Used if exists, otherwise Engine
• {USERSETTINGS} = LocalAppData/Roaming
• {USER} = User My Documents folder
  • Expansion Tags (out of the box stock engine):
• {ED} = Directory Prefix
  • Note the /
• {EF} = File Prefix

Any path with {ED} or {EF} gets thrown into an ConfigLayerExpansion loop. Within that loop, any path with {PLATFORM} gets thrown into a DataDerivedPlatformInfo loop.

1. Non-User Specific Settings

   Path	Notes
   {ENGINE}/Base.ini
   {ENGINE}/{ED}{EF}Base{TYPE}.ini
   {ENGINE}/{ED}{PLATFORM}/{EF}Base{PLATFORM}{TYPE}.ini	But use extension folder if it exists: {EXTENGINE}/{ED}{EF}Base{PLATFORM}{TYPE}.ini
   {PROJECT}/{ED}{EF}Default{TYPE}.ini
   {ENGINE}/{ED}{PLATFORM}/{EF}{PLATFORM}{TYPE}.ini	But use extension folder if it exists: {EXTENGINE}/{ED}{EF}{PLATFORM}{TYPE}.ini
   {PROJECT}/{ED}{PLATFORM}/{EF}{PLATFORM}{TYPE}.ini	But use extension folder if it exists: {EXTPROJECT}/{ED}{EF}{PLATFORM}{TYPE}.ini

2. User Specific Settings

   Path	Notes
   {USERSETTINGS}/Unreal Engine/Engine/Config/User{TYPE}.ini
   {USER}/Unreal Engine/Engine/Config/User{TYPE}.ini
   {PROJECT}/User{TYPE}.in

Expansion Loop

1. If config format has either {ED} or {EF} in it, it runs through ConfigLayerExpansions which allow for additional overrides
2. All config files with expansion tags run through entire array of the following ED (Directory), EF (File) pairs. i.e. {ENGINE}/Config/{ED}{EF}BaseEngine.ini

   {ED}	{EF}	Example
   		Engine/Config/BaseEngine.ini
   	Shippable	Engine/Config/ShippableBaseEngine.ini
   NotForLicensees/		Engine/Config/NotForLicensees/BaseEngine.ini
   NotForLicensees/	Shippable	Engine/Config/NotForLicensees/ShippableBaseEngine.ini
   NoRedist/		Engine/Config/NoRedist/BaseEngine.ini
   NoRedist/	Shippable	Engine/Config/NoRedist/ShippableBaseEngine.ini

3. If config path has a {PLATFORM} tag within it, for every Expansion iteration we also perform Platform lookup.
   1. Fetch DataDrivenPlatformInfo
      1. In non-extension platform folders, i.e. Engine/Config/Windows/, there exists a DataDrivenPlatformInfo.ini which is used to defining various platform configurations and targets. That platform name the .ini is loaded for is the file’s parent folder: Engine/Config/Windows/DataDrivenPlatformInfo.ini, platform = Windows
      2. In extension platform folders, same logic, platform name is two folders up: Engine/Platforms/Windows/Config/DataDrivenPlatformInfi.ini, platform = Windows
   2. UnrealBuildTool only loads the following information per platform:
      1. bIsConfidential
      2. AdditionalRestrictedFolders (an array of strings)
      3. IniParentChain (an array of strings)
         1. If IniParentChain is not null, platform ini’s loaded from most generic to most specific, with most specific being our current active platform
         2. Unix then Linux i.e. Engine/Unix/BaseUnixEngine.ini, then Engine/BaseLinuxEngine.ini
         3. Unix then Arch64Linux
         4. Android then Lumin
         5. IOS then TVOS i.e. Engine/IOS/BaseIOSEngine.ini, then Engine/TVOS/BaseTVOSEngine.ini

Full List of Config Files

These are all the config files loaded for Linux for config type Engine. This process is repeated for Game, Input, etc...

Linux does not use the platform extension folder, which would change Engine config paths from Engine\Config\PlatformName to Engine\Platforms\PlatformName\Config.

1. Engine/Base.ini
2. Engine/BaseEngine.ini
3. Engine/ShippableBaseEngine.ini
4. Engine/NotForLicensees/BaseEngine.ini
5. Engine/NotForLicensees/ShippableBaseEngine.ini
6. Engine/NoRedist/BaseEngine.ini
7. Engine/NoRedist/ShippableBaseEngine.ini
8. Engine/Unix/BaseUnixEngine.ini
9. Engine/Unix/ShippableBaseUnixEngine.ini
10. Engine/Unix/NotForLicensees/BaseUnixEngine.ini
11. Engine/Unix/NotForLicensees/ShippableBaseUnixEngine.ini
12. Engine/Unix/NoRedist/BaseUnixEngine.ini
13. Engine/Unix/NoRedist/ShippableBaseUnixEngine.ini
14. Engine/Linux/BaseLinuxEngine.ini
15. Engine/Linux/ShippableBaseLinuxEngine.ini
16. Engine/Linux/NotForLicensees/BaseLinuxEngine.ini
17. Engine/Linux/NotForLicensees/ShippableBaseLinuxEngine.ini
18. Engine/Linux/NoRedist/BaseLinuxEngine.ini
19. Engine/Linux/NoRedist/ShippableBaseLinuxEngine.ini
20. Project/DefaultEngine.ini
21. Project/ShippableDefaultEngine.ini
22. Project/NotForLicensees/DefaultEngine.ini
23. Project/NotForLicensees/ShippableDefaultEngine.ini
24. Project/NoRedist/DefaultEngine.ini
25. Project/NoRedist/ShippableDefaultEngine.ini
26. Engine/Unix/UnixEngine.ini
27. Engine/Unix/ShippableUnixEngine.ini
28. Engine/NotForLicensees/Unix/UnixEngine.ini
29. Engine/NotForLicensees/Unix/ShippableUnixEngine.ini
30. Engine/NoRedist/Unix/UnixEngine.ini
31. Engine/NoRedist/Unix/ShippableUnixEngine.ini
32. Project/Unix/Unix/Engine.ini
33. Project/Unix/ShippableUnixEngine.ini
34. Project/NotForLicensees/Unix/UnixEngine.ini
35. Project/NotForLicensees/Unix/ShippableUnixEngine.ini
36. Project/NoRedist/Unix/UnixEngine.ini
37. Project/NoRedist/Unix/ShippableUnixEngine.ini
38. AppData/Local/Unreal Engine/Engine/Config/UserEngine.ini*
39. Documents/Unreal Engine/Engine/Config/UserEngine.ini*
40. Project/UserEngine.ini

I believe the two crossed out config files should have been updated to use a newer config method that Unreal Engine uses in which it places config files in versioned folders inside AppData. Maybe this is a bug in the UnrealBuildTool but no one has build settings in their User Data folders so it doesn't matter?

Global BuildConfiguration Settings

When building, the UnrealBuildTool creates a global BuildConfiguration object that configures all builds. Unless otherwise noted, default is false/null. It can be configured in various ways using both XMLConfigFiles and command-line options.

#TODO Write a script to parse this shit from .xml schema

XMLConfigFile Fields

Command-line Options

Target Descriptors

A TargetDescriptor describes all of the information needed to initialize a UEBuildTarget object. It is made up of the following:

• ProjectFile: (Can be Null) A file path to a .uproject file that describes project level information about are target.
• Name: A string representing the name of the target
• Platform: This target’s UnrealTargetPlatform i.e. Win32, Win64, LinuxAArch64, Mac, IOS, etc.
• Configuration: This target’s UnrealTargetConfiguration, i.e. Unknown, Debug, DebugGame, Development, Shipping, Test
• Architecture: A string representing this target’s architecture, usually just an empty string. Appears to be only used on Linux/HoloLens/iOS.
• AdditionalArguments: Command line arguments to use when building this target; aggregate of initial UBT command line args as well as any additional args specified for a target when a TargetList is specified

  Field Name	Command-line Arg	Description
  ForeignPlugin	-ForeignPlugin	Foreign plugin to compile against this target
  OnlyModuleNames	-Module=	List of modules to only build
  SingleFileToCompile	-SingleFile=	File path to the only file that should be compiled
  HotReloadMode	-NoHotReload, -ForceHotReload, -LiveCoding	Sets mode for Hot Reloading
  WriteActionFiles	-WriteActions=	Export the actions for the target to a file
  LiveCodingModules	-LiveCodingModules=	Path to a file containing a list of modules that may be modified for live coding
  LiveCodingManifest	-LiveCodingManifest=	Path to the manifest for passing info about the output to live coding
  bQuiet	-Quiet	Suppress messages about building this target

Example: Building UnrealHeaderTool Target Descriptor

Building the UHT, this is the initial UBT command line UnrealHeaderTool Win64 Development -WaitMutex -FromMsBuild

Goal is to build a TargetDescriptor, which requires: ProjectFile (Can be Null), TargetName, Platform, Configuration, Architecture, Arguments

1. Skips Target List processing as -TargetList/-Target are not provided, go right into parsing this command line as a TargetDescriptor
2. ProjectFile set as null
3. Determines Platform is Win64
4. Determines Configuration is Development
5. Determines UnrealHeaderTool as the TargetName
6. Uses default Architecture for Win64, which is an empty string “”
7. Scan for any ProjectFiles that match this target, which there are none of, so ProjectFile remains null
8. -WaitMutex and -FromMSBuild added as additional commandline arguments for this target
9. TargetDescriptor fully built

Engine BuildVersion

A BuildVersion class describes all the versioning info for any given UE4 Engine.

Always located in Engine\Build\Build.version in JSON format.

Includes the following information:

UEBuildTarget Object

A UEBuildTarget contains all the information needed to build a target. This is required to generate a Makefile.

ModuleRulesContext

Stores information about where a module rules object came from, and how it can be used.

RulesAssembly Object

The most practical way to think of a RulesAssembly Object is to imagine a .dll that simply consists of classes defined by a group of Module .Build.cs files. DotNet C# allows dynamic compilation and loading of .dlls (Assemblies). The UnrealBuiildTool uses this mechanism so that Module authors can use C# to programmatically define the build rules for their Modules but not worry about building the UnrealBuildTool assemblies that are made up of all engine and project module build rules. The RulesAssembly Object is essentially a bunch of metadata about these .dlls as well as a reference to a .dll.

The Engine makes use of two RulesAssembly Objects.

A RulesAssembly stores information about a compiled DotNet C# .dll assembly that contains our assembly’s build rules and the types it contains. When a RulesAssembly Object is constructed, it attempts to compile and load a DotNet C# Assembly.

Compiling and Loading a RulesAssembly Object

A RulesAssembly stores information about a compiled DotNet C# .dll assembly that contains our assembly’s build rules and the types it contains.

1. When constructed, consumes a Scope, list of base directories (Engine + EngineExt), list of PluginInfos, a dictionary mapping module files to ModuleRulesContext objects, list of TargetFiles to compile, an assembly name, whether this assembly contains engine modules, a BuildSettingsVersion (V1, V2, Latest), whether assemblies readonly, whether to skip compile, and our parent RulesAssembly
2. Builds list of Assembly SourceFileNames (union between all module and target files)
3. Grab all Preprocessor Definitions (this is the entire list)
   • WITH_FORWARDED_MODULE_RULES_CTOR
   • WITH_FORWARDED_TARGET_RULES_CTOR
   • UE_4_{MINORVERSION}_OR_LATER, i.e. UE_4_25_OR_LATER; includes all defines MinorVersion 17 and upwards to current
4. DynamicCompilation module compiles an Assembly consuming OutputAssemblyPath, HashSet of SourceFileNames, a list of ReferencedAssemblies, list of preprocessor definitions, whether to not compile, and whether to treat warnings as errors
   1. Load the Assembly manifest, same path as Assembly, just swap out .dll with Manifest.json
      1. i.e. D:\UE425\Engine\Intermediate\Build\BuildRules\UE4RulesManifest.json
      2. This file appears to be generated with Generating Project Files (GPF)
      3. Contains a list of all Engine module files (*.Build.cs``) and the Engine version that it was generated with
   2. If not skipping compiling, check to see whether we require compilation
      1. Consumes SourceFileNames, the Assembly manifest, and the Assembly output path
      2. Require compilation if Assembly output file does not exist
      3. Require compilation if UnrealBuildTool is newer than Assembly
      4. Require compilation if Assembly manifest does not exist (it is generally expected to always exist)
      5. Require compilation if Engine version has changed from the manifest Engine version
      6. Require compilation if any Module source files were added or removed or have a newer timestamp than Assembly
   3. If we do not need to compile our assembly, we try to load it
      1. DotNet C# has a built in way of loading Assemblies, simply Assembly.LoadFile(FilePath), if fails, mark for recompile
   4. If we need to compile our assembly, use the DynamicCompilation class to CompileAssembly
      1. Build DotNet CompileParameters

         Parameter	Description
         GenerateInMemory = false	Always compile the assembly to a file on disk
         OutputAssembly = Output Path	This is the full path to the assembly file we're generating
         GenerateExecutable = false	We always want to generate a class library, not an executable
         Treat Warnings as Errors	Supplied as function arg to CompileAssembly
         WarningLevel = 4	Set the warning level so that we will actually receive warnings
         IncludeDebugInformation = true	Always generate debug information as it takes minimal time

      2. Add compiler flag /optimize if we aren’t in DEBUG
      3. If not supplied any ReferencedAssemblies, force referencing System.dll and System.Core.dll
      4. Otherwise just use our supplied ReferencedAssemblies
      5. Always add UnrealBuildTool and DotNETUtilities as ReferencedAssemblies
      6. Add compiler args for preprocessor definitions /define:
      7. Use DotNet C# CSharpCodeProvider with CompilerVersion 4.0 to compile our Assembly using our CompileParameters
      8. Display any warnings or errors, throwing if any errors.
      9. Clean up temporary files.
5. Save out new Assembly Manifest file containing all the source files we used to compile our Assembly
6. Load Assembly into Application memory
7. Map Module Names to Module Files and Target Names to Target Files
8. Print any Obsolete or Deprecated code in our RulesAssembly module build classes

Creating the Engine RulesAssembly Object

A RulesAssembly object representing the rules of assembling the Engine.

1. Consume Engine RulesScope, a list of all Engine and Engine Platform Extension directories, the Assembly Prefix (UE4), list of included Plugins, whether to use precompiled engine libraries (forced to if using an Engine build), whether to skip compiling for this assembly, and any RulesAssembly parent
2. Add to RulesScope: Engine -> Engine Plugins -> Engine Programs
3. Create a default ModuleRulesContext to associate with all modules in this assembly
   • Scoped to Engine, uses Engine directory as DefaultOutputBaseDir, shared PCH enabled
4. Builds a dictionary of all shared engine module files excluding the Programs directory mapped to a ModuleRulesContext
   1. Iterates all consumed root directories, should be the Engine directory as well as Engine Platform Extensions i.e. Engine\Platforms\XXX
      1. Adds all Engine modules within Runtime, Developer, Editor, and ThirdParty subdirectories within Source subdirectory
      2. All modules have a UHTModuleType associated with their Runtime, Developer, Editor, or ThirdParty location
      3. Grabs the .Build.cs files of these modules to use when mapping to a ModuleRulesContext
      4. 558 total out of the box with UE4.25
   2. Iterates consumed list of Plugin directories
      1. Grabs all Plugin .Build.cs files and maps them to a new ModuleRulesContext the same as default but scoped to Engine Plugins
   3. Determine OutputAssemblyPath, default to Engine folder i.e. D:\UE425\Engine
      1. If Engine is installed, use %LocalAppData%\UnrealEngine\{MajorVersion}.{MinorVersion} i.e. ...\UnrealEngine\4.25
   4. Determine RulesAssembly and ProgramRulesAssembly`` file paths
      1. OutputAssemblyPath\Intermediate\Build\BuildRules\{AssemblyPrefix}{Program}Rules.{AssemblyFrameworkExtension}
      2. Example: D:\UE425\Engine\Intermediate\Build\BuildRules\UE4Rules.dll and ...\UE4ProgramRules.dll
   5. Create RulesAssembly as well as a ProgramRulesAssembly``
   6. Return ProgramsRulesAssembly`` as Engine assembly as it contains build rules for both `Engine` Programs and `Engine` Modules

TargetRules Object

Created from a RulesAssembly object. TargetRules is a data structure that contains the rules for defining a target (application/executable). RequiresUniqueBuildEnvironment attributes must be shared between all targets when building in a shared build environment.

#TODO: Write a script to generate this shit

All Bold Fields are RequiresUniqueBuildEnvironment attributes and must have identical values between all targets when using shared environments.

Creating a TargetRules Object

1. RulesAssembly.CreateTargetRules consumes a TargetDescriptor
2. Find RulesAssembly that contains the TargetRules Object Type for our Target
   1. If TargetNameToTargetFileMap doesn’t contain this Target, bubble up this call to the parent RulesAssembly (ProjectRules -> Rules)
   2. If no RulesAssembly contains the target, we throw a build exception
3. Assemble TargetRules Type Name, which is just our Target’s name and the string “Target”, i.e. UnrealHeaderToolTarget
   1. You should note here that this matches the enforced TargetRules objects we define in our .target.cs files, because that what those are
4. Grab DotNet C# Type with TargetRules Type Name through crazy built in C# DotNet Assembly magic, throws if this fails
5. Create an uninitialized instance of this Type, basically an instance of a TargetRules Object that has not ran its constructor
6. Set DefaultBuildSettings on our new TargetRules Object if DefaultBuildSettings has a value
7. Then run the TargetRules Object Constructor, this is what we’re defining when we create our Target’s .target.cs file
8. Sets the TargetFile (path to the .target.cs file this TargetRules represents) for the new TargetRules Object
9. Sets up default definitions based on Target Type, i.e. UE_GAME=1, UE_CLIENT=1, UE_EDITOR=1, UE_SERVER=1, USE_NULL_RHI=1
10. Make sure we have a LinkType set by this point, for Program targets this is manually set in the .target.cs constructor. Monolithic or Modular.
11. Make sure that if we are not using a shared build environment we are also not using an installed version of the Engine
12. Make sure bCompileAgainstCoreUObject is true if bCompileAgainstEngine is true
13. Make sure bBuildWithEditorOnlyData is true if is true
14. Make sure bDisableDebugInfo and bOmitPCDebugInfoInDevelopment are both false if bForceDebugInfo is true
15. Make sure bOmitFramePointers is false and USE_MALLOC_PROFILER=1 defined if bUseMallocProfiler is true
16. Make sure DISABLE_GENERATED_INI_WHEN_COOKED=1 is defined if bAllowGeneratedIniWhenCooked is true
17. Make sure DISABLE_NONUFS_INI_WHEN_COOKED=1 is defined if bAllowNonUFSIniWhenCooked is true
18. Make sure DISABLE_UNVERIFIED_CERTIFICATE_LOADING=1 is defined if bDisableUnverifiedCertificates is true
19. Platform then validates this TargetRules object
    1. For example, Android forces bCompilePhysX = true and bCompileAPEX = false, presumably because APEX isn’t Android supported

UEBuildModule Object

A UEBuildModule represents a unit of code compilation and linking. Requires a RulesObject (.Target.cs) to construct. Represented by our .Build.cs files. Requires a Readonly TargetRules to be constructed. Our UEBuildTarget uses its RulesAssembly (represents our compiled .Build.cs and .Target.cs) to create our UEBuildModule

• #TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore
• A UEBuildModule is basically a wrapper around our ModuleRules, which is an instance what we define in our .Build.cs file
• After wrapping this ModuleRules file, a UEBuildModule also adds all the default include folders, i.e. Classes, Public, Private

Constructing a UEBuildModule

Requires a RulesObject (.Target.cs). Creates a ModuleRules object which represents our .Build.cs files. Requires a Readonly TargetRules to be constructed. UEBuildTarget uses its RulesAssembly to make our UEBuildModule.

1. Consumes Module Name, a readonly instance of our TargetRules instantiated from our .Target.cs, and a file path to our .Target.cs
2. RulesAssembly uses ModuleName to grab the .Build.cs file path for this module using its cached list of .Build.cs paths to modules
   1. If this fails, we ask the RulesAssembly’s parent RulesAssembly recursively for the .Build.cs file path, throwing if never found
   2. Consumed .Target.cs file path is only used for logging in this step if the ModuleRules type can not be found
   3. Does this so that we know that we have loaded a ModuleRules with typed with our Module Name somewhere
3. Determine the .DotNet C# Type of our UEBuildModule
   1. Uses .DotNet shenanigans to get the Type of our ModuleRules using ModuleName or UnrealBuildTool.Rules.ModuleName namespace
      1. // Temporary hack to avoid System namespace collisions, // @todo projectfiles: Make rules assemblies require namespaces.
      2. This is the UnrealHeaderTool in public class UnrealHeaderTool : ModuleRules
   2. Look for a ModuleRules Type that overrides this module for our platform
      1. First try our supplied Platform, i.e. Win64 (would be UnrealHeaderTool_Win64)
      2. Then try the names of the Platform Groups that contain our Platform (i.e. UnrealHeaderTool_Desktop, _Microsoft, etc.)
         1. There can only be one valid Platform Group module that works for us, otherwise we throw an error
         2. If a Platform ModuleRules class exists, it basically stomps the base ModuleRules transparently
4. Create an uninitialized instance of our ModuleRules class with our determined Type
5. Disables Precompiling if Precompiling is set and we aren’t an Engine module or UE4Game
6. Sets bUsePrecompiled if this RulesAssembly has marked this module or target read-only
7. If our ModuleRules type is overridden (i.e. a Platform override _Win64), iterate our parent ModuleRules
   1. // go up the type hierarchy (if there is a hierarchy), looking for any extra directories for the module
8. Finally grab and invoke our ModuleRules Constructor passing in our ReadOnlyTargetRules object
   1. The constructor here actually is very trivial and just stores the passed in ReadOnlyTargetRules in a field named TargetRules
9. Then set a bunch of default values for our ModuleRules
   1. If we are a C++ module that isn’t named Launch, mark that we require an IMPLEMENT_MODULE macro (bRequiresImplementModule)
   2. If ProjectFile supplied, add all Module names listed in the ProjectFile to PrivateDependencyModuleNames
   3. Remove trailing slash from Include and Library paths, as the ending slash can escape enclosing quotes for command line ops
   4. Validate RulesObject
      1. Ensure that no Privately/Publicly depended Modules are also DynamicallyLoadedModules
      2. Ensure a PrivatePCHHeaderFile is set if not using shared PCHs.
         1. If no PCH is set, we will grab the first C++ source file we come across and use it’s first include as our PCH
   5. // Clear bUsePrecompiled flag if compiling a foreign plugin; as it's treated like an engine module, it will default to true in installed builds.
   6. // Get the base directory for paths referenced by the module. If the module's under the UProject source directory use that, otherwise leave it relative to the Engine source directory.
   7. // Get the generated code directory. Plugins always write to their own intermediate directory so they can be copied between projects, shared engine intermediates go in the engine intermediate folder, and anything else goes in the project folder.
      1. For example, D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Inc\UnrealHeaderTool
   8. Allow all UEBuildPlatforms to modify our rules via UEBuildPlatform.ModifyModuleRulesForOtherPlatform
      1. This allows things like undisclosed platforms and being able to compile Android shaders on Windows for mobile previewing
10. Finally call InstantiateModule passing our RulesObject and GeneratedCodeDirectory to create, cache, and return our UEBuildModule

UEBuildBinary

Represents a binary built by UBT.

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore

CPPCompileEnvironment

Encapsulates the environment that a C++ file is compiled in.

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore

LinkEnvironment

Encapsulates the environment that is used to link object files.

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore

UEToolChain

#TODO: This is really important to cover.

Encapsulates the environment that a C++ file is compiled in.

#TODO: Make some sort of tool to pull all the members of these classes from the .xml files because I don’t want to do this anymore

• On Windows, VCToolChain (child of ISPCToolChain) is used unless you are using the PVS Studio Static Code Analyzer
• UEToolChain defines things like building the command-line args to pass to C++ compilers and linkers and such

UHTModuleType

Type of module. Mirrored in UHT as EBuildModuleType. This should be sorted by the order in which we expect modules to be built.

1. Program
2. EngineRuntime
3. EngineUncooked
4. EngineDeveloper
5. EngineEditor
6. EngineThirdParty
7. GameRuntime
8. GameUncooked
9. GameDeveloper
10. GameEditor
11. GameThirdParty

UHTManifest

Information to be sent to the UnrealHeaderTool.

UHTManifest.Module

Information about a Module stored in a UHTManifest.

Actually Building A Target

1. Startup and Determining Build Mode

1. Sets current working directory to the Engine Source directory
2. Determine GlobalOptions using Command Line Args
   • By default, “Mode” is a null string
   • Different modes can be invoked using the -Mode=[Name] argument on the command line, where [Name] is determined by the ToolModeAttribute on a ToolMode derived class. The log system will be initialized before calling the mode, but little else.
   • All valid ToolMode classes are tagged with a ToolModeAttribute, which defines their “Mode” lookup string
     • ToolModeAttribute can define options that configure UnrealBuildTool, defined as an enum with BitField/BitMask values
       1. None
       2. StartPrefetchingEngine
       3. XMLConfig
       4. BuildPlatforms
       5. BuildPlatformsHostOnly
       6. BuildPlatformsForValidation
       7. SingleInstance
       8. ShowExecutionTime
   • If “Mode” remains null after processing args, default mode is BuildMode
   • BuildMode has the following options set:
     1. XmlConfig
     2. BuildPlatforms
     3. SingleInstance
     4. StartPrefetchingEngine
     5. ShowExecutionTime

2. Startup and Configuration Loading

1. Engine content prefetching
   1. Stores metadata about every file in the form of a map that stores the UnrealBuildTool DirectoryItem and System.IO.FileItem info of a file
   2. System.IO.FileItem knows file directory, path, length, existence
2. Fetch XML Configuration Data (UnrealBuildTool_Mutex_XmlConfig)
   1. XML Config files can be specified by -XmlConfigCache=
      1. Force use of the cached XML config without checking if it's valid (useful for remote builds)
   2. If no config cache specified, default is Engine\Intermediate\Build\XMLConfigCache.bin, validate cache (see slide on Config Validation)
3. Locks SingleInstance Mutex (UnrealBuildTool_Mutex)
4. Register All Build Platforms, not including not installed platforms, not limited to host only platform
   1. Initializes Installed Platform (Invokes a Static Constructor)
      1. BuildPlatformInfo.Current is a Singleton initialized to the current build platform child class via GetRuntimePlatform() determined by DotNet PlatformID Environment.OSVersion.Platform
      2. Linux Vs. Mac determined by File.Exists("/System/Library/CoreServices/SystemVersion.plist");
   2. Builds Config Hierarchy for InstalledPlatform with TYPE set to “Engine” (Puts the Engine in DefaultEngine.ini / BaseEngine.ini)
      1. Creates a list of ConfigFiles which are just a list of ConfigFileSections, which are just a name and a list of ConfigLines:
         1. ConfigLines are a Name, Value, and an Action
         2. Available Config Actions are Set, Add, RemoveKey, RemoveKeyValue
   3. Looks for Config Command Line Arg “-ini:Engine:”
      1. If found, this simply just adds the given .ini file as the final config on the Config Hierarchy
5. Also attempts to parse InstalledPlatforms if configuration value InstalledPlatforms.HasInstalledPlatformInfo exists and has a meaningful value
   1. I don’t know what this is used for, appears to only affect Installed Builds which I currently am not interested in diving into

3. BuildPlatformFactory Discovery

Register all tool chains and build platforms that are present. Iterate through all C# classes filtering for non-abstract UEBuildPlatformFactorys

1. Creates a temp instance of iterated UEBuildPlatformFactory
2. If bHostPlatformOnly == true, filter for only platforms that match host platform. This appears to be usually false
3. InstalledPlatform determines whether iterated BuildPlatform is valid, always valid if including all non-installed platforms
   1. Does a Platform config check
   2. Does a Project config check, either desired build project type is Any, BuildPlatform is marked as Any, or types match
   3. If valid, calls Factory’s RegisterBuildPlatforms()
      1. Factories generally initialize and validate their SDKs in this step, either setting up a AutoSDK or ManualSDK
         1. For example, LinuxPlatformFactory tries to verify that it has Clang, GCC, or other toolchain
4. Example: WindowsPlatformFactory
   1. Windows simply always returns true that it has a valid manual sdk installed
   2. Registers Win32 and Win64 build targets
   3. Adds both targets to Windows, Microsoft, and Desktop platform groups
5. BuildPlatforms loaded “out of the box” with stock Unreal Engine 4
   1. XXXPlatformFactory
   2. MacPlatformFactory
   3. TVOSPlatformFactory
   4. AndroidPlatformFactory
   5. HoloLensPlatformFactory
   6. IOSPlatformFactory
   7. LinuxPlatformFactory
   8. LuminPlatformFactory
   9. WindowsPlatformFactory

4. Executing The BuildTool ToolMode

Now that our UnrealBuildTool has loaded all configuration and platform info, it will now run the BuildMode ToolMode’s Execute function which appears to be the real meat and potatoes of building a target in the Unreal Engine ecosystem.

1. BuildMode applies any Command Line Arg options to itself. These are the available options, all default to false

   Args	Description
   -IgnoreJunk	Whether to skip checking for files identified by the junk manifest.
   -SkipBuild	Skip building; just do setup and terminate.
   -XGEExport	Whether we should just export the XGE XML and pretend it succeeded
   -NoEngineChanges	Do not allow any engine files to be output (used by compile on startup functionality)
   -WriteOutdatedActions=	Whether we should just export the outdated actions list (requires a file path)

2. Initializes Trace Logging
3. Sets up UnrealBuildTool.InitialEnvironment, grabbing ALL system Environment variables
4. Applies XML Config data to itself. Out of the box, there appears to be zero config values on an initial UnrealHeaderTool build.
5. If not configured otherwise, initializes a logger to Engine\Programs\UnrealBuildTool\Log.txt
   1. Can be disabled with -NoLog
   2. Can have suffixes applied to path by using -LogSuffixes= which consumes an array of strings.
      1. -LogSuffixes="Ass, Foo" would output to Engine\Programs\UnrealBuildTool\Log_Ass_Foo.txt
6. Creates BuildConfiguration object, a long list of settings that applies to all builds, see slides about BuildConfiguration configuration
7. Check to see if root path isn’t too long, but only on Windows, because Windows is an inferior platform
8. Deletes Junk, Loads JunkManifest.txt file and removes all junk files/folders defined in it. Currently don’t know what generates a JunkManifest?
9. Parse all Target Descriptors
10. Codepath separates if needing to handle remote building if remote building for Mac
11. Processes and builds all Target Descriptors

5. Determining Target Descriptors

A TargetDescriptor describes all of the information needed to initialize a UEBuildTarget object. Uses CommandLineArguments struct in DotNetUtilities.

1. List of TargetDescriptors generated by parsing the command line
   1. If Target List Args are present (-TargetList, -Target):
      1. Looks for a -TargetList= arg which accepts a list of file paths that represent a list of targets
         1. Each line in a target list represents a build target and is processed through the same command line parser
         2. When re-ran through the command line parser, it adds the target list line args to the args initially passed into UnrealBuildTool but with the -TargetList and -Target args stripped
      2. Looks for a -Target= arg which accepts a list of build targets and command line args, basically an inlined TargetList
      3. This parses the command line recursively until it has a command line for building that includes only a single target
         1. All ParseCommandLine calls filter into ParseSingleCommandLine which does the actual TargetDescriptor parsing
   2. Otherwise treat the entire initial command line argument string as build target info and go right into ParseSingleCommandLine
2. ParseSingleCommandLine (Build a TargetDescriptor based off of command line arg info or target string from target list:
   1. First tries to get Project file path by checking for -Project=, can be null, can only specify a single project file per target, if specified must exist
   2. Looks for arguments without ‘-’ prefix
      1. Also looks for a .uproject file reference as an unnamed arg value by seeing if arg ends with text “.uproject”
      2. If not a project file, tries to parse args as Platform references
         1. Arg must match a platform that the UnrealBuildTool is aware of during startup configuration
         2. Multiple platforms can be specified by separating with the “+” character
      3. If not a Platform reference, tries to parse args as an UnrealTargetConfiguration
         1. Valid UnrealTargetConfigurations are Unknown, Debug, DebugGame, Development, Shipping, Test
         2. Multiple configurations also specifiable with the “+” character
      4. If not an UnrealTargetConfiguration, assume arg without a ‘-’ prefix is a Target name
   3. Must have at least one Platform and one Configuration to be able to move forward
   4. Iterate and Validate all parsed Platforms, but only if using an installed build of the Engine, otherwise all Platforms assumed valid
      1. Determine Architecture for the platform (An Architecture is really just a Platform modifier? Usually an Empty String)
         1. If no Architecture is specified with -Architecture=, grabs the default Architecture from configured Platform info
            1. Multiple Architectures supported if using the ‘+’ character, similar to Platforms and Configurations
            2. If you manually specify an Architecture, you are required to also specify a -TargetType list?
      2. Iterate all parsed Configurations, goal is to ultimately build a list of TargetNames which will then allow for making TargetDescriptors
         1. Build list of Target Names
            1. If ProjectFile is specified, iterate all -TargetType args
               1. A valid TargetType is one of Game, Editor, Client, Server, Program
               2. Determines TargetName from ProjectFile and TargetType and adds this to our list of targets we want to build
                  1. Does some crazy shit with C#/DotNet and actually compiles a new DotNet assembly with our Target .cs classes
                  2. Loads the newly compiled RulesAssembly DotNet assembly and uses RTTI stuff to grab the right target name
         2. If ProjectFile is null, we should already have a TargetName, likely from our initial UBT command line args, i.e. UnrealHeaderTool
            1. Remember, args not ‘-’ prefixed in any target command line iteration that aren’t a Platform/Configuration are treated as TargetNames
         3. Build list of TargetDescriptors using list of TargetNames
            1. Attempt to find a ProjectFile for all TargetNames if ProjectFile still null, iterates all TargetNames:
               1. Determine BaseDirectories by enumerating all .uprojectdirs files in Engine Root folder (i.e. D:\UE425), restricted to subdirs
               2. Enumerates through all .uproject files found in all BaseDirectories recursively
               3. Looks for ProjectFile with matching Target file located in ProjectFile’s Source and Intermediate\Source paths
            2. Now have all required information for a TargetDescriptor to be added to our list of targets to build!
               1. ProjectFile (Can be Null), TargetName, Platform, Configuration, Architecture, Arguments
   5. All TargetDescriptors built for all targets for all desired target types for all desired configurations for all desired architectures for all desired platforms

6. Building our SourceFileWorkingSet

1. Iterate every target and prefetch metadata about all Source and Plugin\Source folders defined by any target ProjectFiles
2. Create a SourceFileWorkingSet from the Root Engine folder as well as a list of all project paths found in step 1
   1. Behavior dependent on current source file provider, which could be one of the following: None, Default, Perforce, or Git.
      1. If provider is None, or we’re set to generate project files, we return an empty SourceFileWorkingSet
      2. If provider is Git, we try to create a GitSourceFileWorkingSet, consuming root engine path and all relevant project paths
         1. Checks for a .git file in the root directory, if found, initializes a GitSourceFileWorkingSet from this .git file
            1. Runs git status --porcelain to get all modified source files to exclude from a Unity build
         2. If a root .git isn’t found, iterate .git files in all relevant project folders, creates GitSourceFileWorkingSets
            1. GitSourceFileWorkingSets can have sub-GitSourceFileWorkingSets that represent Git submodules
      3. If provider is Perforce, we try to create a PerforceSourceFileWorkingSet
         1. Filter out all source files that end in .gen.cpp, as well as filter out all files that are marked read-only
            1. This creates a SourceFileWorkingSet that only has writable source files without actually contacting Perforce
      4. If provider is Default, we attempt to create a GitSourceFileWorkingSet, and if that fails, a PerforceSourceFileWorkingSet
3. Finally invokes the main Build function defined in our BuildMode ToolMode
   1. Consumes all of our TargetDescriptors, our global BuildConfiguration object, our SourceFileWorkingSet, any additional build command line arg configurations, a optional file path to write outdated actions to

7. Loading a Makefile if Present

Finally invoking the main Build function defined in our BuildMode ToolMode. This function attempts to load or create a Makefile per target.

1. If we’re using UBT Makefiles and we’re not compiling a single file, generate the location to a potentially pre-existing Makefile
   1. Base directory is our relevant Project directory, but if passed in a null ProjectFile, our base directory is the Engine folder
   2. BaseDirectory\Intermediate\Build{TARGETPLATFORM}{TARGETNAME}{TARGETCONFIGURATION}\Makefile.bin
      1. I.e. D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Development\Makefile.bin
2. Attempt to load a pre-existing Target Makefile at the determined Makefile file location
   1. Returns load failure if Makefile does not exist
   2. Returns load failure if Makefile timestamp older than Engine BuildVersion timestamp
   3. Returns load failure in non-installed Engines if any Engine ProjectFile (i.e. UnrealHeaderTool) is newer than Target Makefile
   4. If Target ProjectFile is not null:
      1. Returns load failure if Makefile is older than Project’s .uproject file
      2. Returns load failure if Makefile is older than Project’s generated C++ ProjectFile in Project\Intermediate\ProjectFiles
   5. Returns load failure if Makefile is older than timestamp of the UnrealBuildTool executable
   6. Returns load failure if any of the BuildConfiguration XMLConfigFile files are newer than our Makefile
   7. Returns load failure if Makefile TargetMakefile version does not match the UnrealBuildTool TargetMakefile version
   8. Returns load failure if Makefile arguments do not match command line arguments passed in from the UnrealBuildTool
   9. Returns load failure if Makefile’s referenced config files are outdated or changed in any way as Project .inis can affect building
   10. Returns load failure if build platform metadata has changed
       1. Allows platforms to define data that may or may not invalidate makefiles
       2. Currently only used on a out of the box Engine on Windows platforms for discovering changes to a Project’s .exe Icon
3. If Makefile is loaded, run pre-build scripts and additional checks to really fully ensure that our Makefile is still valid and up-to-date
   1. If any pre-build scripts defined, run them, and immediately invalidate our Makefile because we don’t know what may have changed
   2. Check to see if all source files are still valid for this Makefile
      1. Check to see if any source files or directories containing source files for our target have been added, removed, or modified
      2. Check to see if any External or Internal Dependencies have changed @TODO What is a Makefile dependency?
      3. Check to see that no new plugins have been added
      4. Check to see that no source header files with reflection markup have changed
         1. Regular Expression for Reflection Markup: ^\s*U(CLASS|STRUCT|ENUM|INTERFACE|DELEGATE)\b
      5. If adaptive unity build enabled, check to see if any files have been added, removed, or modified from our SourceFileWorkingSet
   3. If we have finally determined that we were able to load a valid pre-existing Makefile:
      1. Set all environment variables desired by the Makefile
      2. Determine and run UnrealHeaderTool if required @TODO figure out
      3. Return our pre-existing Makefile
   4. If we have no valid Makefile or our Makefile was invalidated for any reason, we will create a new Makefile now
   5. In order to build a Makefile, we first need to create a UEBuildTarget object...

8. Creating a UEBuildTarget

A UEBuildTarget contains all the information needed to build a target. This is required to generate a Makefile.

1. Consumes a TargetDescriptor, whether to skip rules compiling, and whether to use precompiled static libraries for editor modules
2. Throws if Target Platform is unavailable or unbuildable
3. Create a RulesAssembly
   1. If ProjectFile is supplied, we try to create a Project RulesAssembly @TODO: Document
   2. Otherwise, create Engine RulesAssembly
      1. Return cached Engine RulesAssembly if one was already made prior in this instance of UnrealBuildTool
         1. Build a list of all Engine PluginInfos (304 of them out of the box on 4.25)
         2. Create an Engine scoped RulesScope with no scope parent
            1. A RulesScope is a LinkedList representing a flat hierarchy i.e. Project -> Engine -> GameFramework
            2. All children of a RulesScope are “in scope” or contained within the parent scope
            3. Dependencies of modules can only go deeper in scope, never upward into parent scopes
         3. Grabs the Engine and Engine Platform Extension directories
         4. Create the actual Engine RulesAssembly object (see slides on Creating an Engine RulesAssembly)
      2. If creating an Engine RulesAssembly failed because Target is an Enterprise Target, make an Enterprise RulesAssembly
   3. If ForeignPlugin is supplied, create a Plugin RulesAssembly
4. Create a TargetRules object using our RulesAssembly (see the next large amount of slides)
5. If we can’t generate project files, check to see if TargetRules allows building for desired platform.
   1. Supported platforms defined by SupportedPlatformsAttribute in Module .build.cs or based off of TargetType and PlatformClass
6. Make sure our desired BuildConfiguration is supported, either defined by SupportedConfigurationsAttribute or based off TargetType
   1. Does a bunch of platform checking based on installed Engine builds, i.e. “throw Download support for building targets from the launcher.”
7. If using a shared environment, validate all RequiresUniqueBuildEnvironment attributes are identical
   1. Target fields checked against the vanilla UE4Game, UE4Editor, UE4Client, UE4Server target rules based on target type.
8. If we’re precompiling, generate a list of all the files we depend on
   1. Stored as Engine or Project\Intermediate\DependencyLists\TargetName\Configuration\Platform\DependencyList.txt
   2. Saved as DependencyList-AllModules.txt if building all modules
9. If TargetDescriptor SingleFileToCompile is supplied, disable all Unity build options and disable linking
10. If TargetDescriptor ForeignPlugin is supplied, disable linking of this output in monolithic target builds and disable shared PCH’s
11. Disable linking if we’re just preprocessing only
12. If building a Project with nativization, build the Plugin RulesAssembly object for the plugin with all the nativized code
13. Finally call the UEBuildTarget constructor using our TargetDescriptor, a ReadOnly version of our RulesObject, and the RulesAssembly
    1. Creates a SourceFileMetadataCache, Engine or Project\Intermediate\Build\SourceFileCache.bin
    2. Does general boring initialization of build info and build path determination
    3. Generates Receipt file path Engine or Project\Binaries\Platform\Configuration\TargetName-Platform-Configuration.target
       1. i.e. D:\UE425\Engine\Binaries\Win64\UnrealHeaderTool.target
       2. Configuration folder is skipped if Development, -Platform-Configuration suffix also skipped
    4. Build a ProjectDescriptor if Project file is given
14. Run UEBuiltTarget’s PreBuildSetup, Setup target before build. This method finds dependencies, sets up global environment etc.
    1. Most of this work is based around building UEBuildModules, ModuleRules and Module objects from the .Target.cs files, similar to .build.cs Rules

9. UEBuildTarget - PreBuildSetup

PreBuildSetup is a function that gets called on a UEBuildTarget immediately after a UEBuildTarget is constructed.

1. SetupBinaries
   1. Find or create the Launch UEBuildModule
   2. // Get the intermediate directory for the launch module directory. This can differ from the standard engine intermediate directory because it is always configuration-specific.
   3. OutputDirectory: // Construct the output paths for this target's executable
      1. Determine if this Target is a Platform Extension Target
      2. Determine whether to output to Project directory
      3. Use Platform Extension or Project directories as output if our .Target.cs file resides in either of these directories
   4. Determine if this binary should compile as a .dll (TargetRules.bShouldCompileAsDLL && bCompileMonolithic)
   5. Construct list of Binary OutputPaths (usually just a single Binary output i.e. D:\UE425\Engine\Binaries\Win64\UnrealHeaderTool.exe)
      1. Get Binary Directory (OutputDirectory\Binaries\Platform i.e. D:\UE425\Engine\Binaries\Win64\
      2. Allows for a TargetRules defined ExeBinariesSubFolder (i.e. Datasmith uses this to output in 3dsMax\2017, 3dsMax\2018, etc.)
      3. Create Binary file name
         1. Prefix “lib” if on Linux and compiling a dynamic or static library
         2. If Configuration == TargetRules.UndecoratedConfiguration, just use Module Name i.e. UnrealHeaderTool
         3. Otherwise use the form ModuleName-{Platform}-{Configuration} i.e. UnrealHeaderTool-Win64-Debug
         4. Determine binary extension for platform (i.e. Windows executables = .exe, Windows libraries = .dll)
         5. // Allow the platform to customize the output path (and output several executables at once if necessary)
            1. Default just creates a single item array from input, a single output binary
            2. I.e. Mac, overrides binary path to UnrealHeaderTool/.app/Contents/MacOS/UnrealHeaderTool
   6. Create a UEBuildBinary that represents the Binary that UBT will build
   7. Set our Launch Module’s Binary property to our new Binary
2. SetupPlugins
   1. Does a bunch of stuff to get Plugin module info for building into our Target
   2. #TODO Write this if I ever care about it, a lot of stuff goes on here
   3. If compiling monolithic, mark all these Plugin Modules to output to our first Binary we made in SetupBinaries
   4. If compiling modularly, mark all these Plugin Modules to output to their own Dynamic Library (.dll)
3. AddExtraModules, Creates UEBuildModules for all modules listed in our Rules.ExtraModuleNames list
   1. If compiling monolithic, mark all these Extra Modules to output to our first Binary we made in SetupBinaries
   2. If compiling modularly, mark all these Extra Modules to output to their own Dynamic Library (.dll)
4. Create all of the UEBuildModules that our Binary depends on recursively
   1. #TODO: This is actually kind of neat but all you need to know is that this basically walks the tree of public/private module dependencies
5. Set the Binary for all dependent UEBuildModules we just created, either .exe or .dll in the same fashion as Plugin/Extra modules
6. If TargetRules.bBuildAllModules is true (which appears to only be true with the UE4Editor target)
   1. /// Adds all the precompiled modules into the target. Precompiled modules are compiled alongside the target, but not linked into it unless directly referenced.
   2. #TODO
7. // Add the external and non-C++ referenced modules to the binaries that reference them.
   1. This basically iterates through all of the modules we depend on and if they have any dependent modules that don’t have a Binary set, set them to their parent’s Binary (leave no module behind!)
8. If not compiling monolithically, determine whether binaries create their import libraries separately
   1. On Windows, mark all Binaries
      1. // On Windows create import libraries for all binaries ahead of time, since linking binaries often causes bottlenecks
   2. Otherwise only mark circularly dependent modules
   3. @TODO: Learn what it means for a binary to create their import library separately
9. Our UEBuildTarget is finally created.

10. Making the Makefile

Generating and executing the UBT Makefile is really the crux of the UnrealBuildTool. Now that we have a UEBuildTarget, we can continue making it.

1. After we make our UEBuildTarget, we create our PreBuildStep script files
   1. Simply a list of strings that represent shell commands
   2. /// Specifies a list of steps which should be executed before this target is built, in the context of the host platform's shell.
   3. /// The following variables will be expanded before execution:
   4. $(EngineDir), $(ProjectDir), $(TargetName), $(TargetPlatform), $(TargetConfiguration), $(TargetType), $(ProjectFile), $(PluginDir)
   5. PreBuildSteps in ProjectDescriptors (.uproject) or in Plugin definitions (.uplugin) gathered first
   6. PreBuildSteps in .Target.cs gathered second
   7. PreBuildSteps in .uplugin files are gathered last
   8. Writes a series of build scripts (either .bat or .sh based on platform)
      1. Outputs scripts to Intermediate\Build\Platform\TargetName\Configuration\PreBuild-1.bat, incrementing name with each script
      2. For example: D:\UE425\Engine\Intermediate\Build\Win64\DatasmithFacadeCSharp\Development\PreBuild-1.bat
      3. Inserts “@echo off” on Windows platforms as first line of each batch file
      4. Does a string replace to expand all the variables mentioned above in each script line
      5. Writes all lines to output script file
      6. Adds reference to outputted script file to UEBuildTarget’s ScriptFiles array
2. Execute our PreBuildStep scripts
   1. On Windows call %COMSPEC% /C “ScriptFile” i.e. C:\Windows\System32\cmd.exe /C “PreBuild-1.bat”
   2. Throws build error if any script returns with a non-zero error code
3. We then kick off building our Target, as our UBT Makefile is actually formed after building of the target

11. UEBuildTarget.Build()

In order to make the UBT Makefile to save time for future builds, we must first do a build.

1. Create CppCompileEnvironment and LinkEnvironment objects
2. Create our UEToolChain for our desired platform
3. Set our Environment Variables, this is set by the desired rules for our platform in our TargetRules object
   1. Remember that the TargetRules object also consists of WindowsTargetRules, LinuxTargetRules, etc?
   2. #TODO: Go make slide about WindowsTargetRules when WindowsTargetRules is first mentioned
4. Fetch External Metadata?
5. // Get diagnostic info to be printed before each build
   1. Using Visual Studio 2019 14.16.27031 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023) and Windows 10.0.18362.0 SDK (C:\Program Files (x86)\Windows Kits\10).
   2. @TODO: Find example of TargetRules Diagnostic Info
6. // Setup the hot reload module list
7. // If we're compiling monolithic, make sure the executable knows about all referenced modules
8. // Add global definitions for project-specific binaries. HACK: Also defining for monolithic builds in binary releases. Might be better to set this via command line instead?
   1. // Defines UE_ENGINE_DIRECTORY=EnginePath
9. Setup Toolchain bundle dependencies, // On Mac and Linux we have actions that should be executed after all the binaries are created
10. ExternalExecution.SetupUObjectModules, // Generate headers
    1. Map Modules to their UHTModuleTypes
    2. // Sort modules by type, then by dependency, StableTopologicalSort() #TODO Write about this because it is neat
    3. // Create the info for each module in parallel
       1. Build UHTModuleInfo objects for each Module
       2. Queue each module through SetupUObjectModule
          1. Finds all headers for all modules and classifies them as Classes, Public, or Private headers.
          2. Headers must have a match to the RegEx: "^\s*U(CLASS|STRUCT|ENUM|INTERFACE|DELEGATE)\b"
          3. All headers cached in SourceFileMetadataCache, stores Header’s last write time and whether it contains markup
    4. Iterate through all Modules and if they have any reflection markup…
       1. Determine GeneratedCPPFilenameBase, Path.Combine(Module.GeneratedCodeDirectory.FullName, Info.ModuleName) + ".gen"
          1. I.e. D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Inc\CoreUObject\CoreUObject.gen
       2. If not using precompiled modules, determine Module GeneratedCodeWildcard
          1. I.e. D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Inc\CoreUObject*.gen.cpp
    5. Output list of UObject dependent modules (contains reflection) and a list of UHTModuleHeaderInfos
       1. A UHTModuleHeaderInfo is simply a list of all a module’s source and header files, and whether this module uses precompiled modules
11. ExternalExecution.ExecuteHeaderToolIfNecessary, Runs UnrealHeaderTool for a module if needed
    1. Determine our UHT Manifest file name i.e. D:\UE425\Engine\Intermediate\Build\Win64\UnrealHeaderTool\Development\UnrealHeaderTool.uhtmanifest
    2. Determine UnrealHeaderTool Receipt file name i.e. D:\UE425\Engine\Binaries\Win64\UnrealHeaderTool.target
    3. Check to see if the UnrealHeaderTool (not target specific, always check the UHT) Receipt exists and is up to date
       1. If UHT Receipt (which is a .target in Binaries path) does not exist, UHT must not be built
       2. If UHT is installed, always up to date
       3. If UHT build products do not exist or are newer than UHT Receipt, UHT is out of date
       4. If UHT Receipt is newer than all build products, UHT should be be up to date
    4. Determine if we have the UHT available. We must not be building the UnrealHeaderTool target and an up to date Receipt must exist
    5. Determine whether we should run UnrealHeaderTool
       1. If UHT is not available, we must run it (lol)
       2. If our BuildConfiguration is forcing header generation, we must run it
       3. If AreGeneratedCodeFilesOutOfDate, we must run it
          1. // Get CoreUObject.init.gen.cpp timestamp. If source files older than CoreUObject generated code, we must run it
          2. If any module is missing its generated code directory, we must run it
          3. Checks to see if special Timestamp file is good #TODO what?
          4. If any source files are older than UnrealHeaderTool.exe, we must run it
          5. If any rules files are older than our UHT timestamp, we must run it
          6. If any header was changed, added, or removed, we must run it
       4. If manifest corresponding .uhtpath doesn’t exist, run it. If .uhtpath doesn’t contain path to UHT Receipt, run it.
       5. If manifest corresponding .deps doesn’t exist, run it, If AreExternalDependenciesOutOfDate, run it #TODO
    6. Generate a list of UHTManifest.Modules for all of our Modules that contain UObjects
    7. Create the UHTManifest
    8. If we need to run UnrealHeaderTool and we’re not building UnrealHeaderTool….
       1. #TODO
    9. Update directory timestamps for all UObject Modules, ExternalExecution.UpdateDirectoryTimestamps()
       1. Don’t actually update directory timestamps as that’ll fuck everything up, instead use a special Timestamp file
       2. Timestamp file serves as directory timestamp but it also contains all timestamps of all headers with Reflection Markup (UObjects)