A TypeScript package providing a normalized typed Solidity AST along with the utilities necessary to generate the AST (from Solc) and traverse/manipulate it.
- Various Solidity compiler versions support, starting from 0.4.13.
- Various compiler selection strategies, including compiler auto-detection via
pragma solidity
directives in Solidity sources with possible fallback options. - Interaction with AST nodes, using type definitions.
- Compensation of differences between legacy and compact Solidity AST.
- Referenced AST nodes linking.
- Easy tree traversal with siblings, parent/children relations, recursive walks and node selection via custom predicates.
- XPath AST traversal (based on jSel package).
- Generating the source code from the AST.
Package could be installed globally via following command:
npm install -g solc-typed-ast
Also it can be installed as the dependency:
npm install --save solc-typed-ast
The package introduces easy and universal compiler invocation. Starting with Solidity 0.4.13, source compilation could be done universally:
import { CompileFailedError, CompileResult, compileSol } from "solc-typed-ast";
let result: CompileResult;
try {
result = await compileSol("sample.sol", "auto", []);
} catch (e) {
if (e instanceof CompileFailedError) {
console.error("Compile errors encountered:");
for (const failure of e.failures) {
console.error(`Solc ${failure.compilerVersion}:`);
for (const error of failure.errors) {
console.error(error);
}
}
} else {
console.error(e.message);
}
}
console.log(result);
The second argument with the "auto"
value specifies a compiler selection strategy. If "auto"
is specified and source code contains valid pragma solidity
directive, then compiler version will be automatically picked from it. If compile process will not succeed, the execution will fall back to "compiler guessing": trying to compile source with a few different versions of the new and old Solidity compilers. The other option would be to specify a concrete supported compiler version string, like "0.7.0"
for example. There is also a support for various compiler selection strategies, including used-defined custom ones (CompilerVersionSelectionStrategy
interface implementations).
Package supports switching between native binary Solc compilers and its WASM versions. The CLI option --compiler-kind
and kind
argument of compile*()
functions family may be used for that purpose. Compilers are downloaded on-demand to the directory .compiler_cache
at the package installation directory (by default). The compiler cache location may be customized by setting SOL_AST_COMPILER_CACHE
environment variable to a custom path. For example:
SOL_AST_COMPILER_CACHE=~/.compiler_cache sol-ast-compile sample.sol --compiler-kind native --tree
or
export SOL_AST_COMPILER_CACHE=~/.compiler_cache
sol-ast-compile sample.sol --compiler-kind native --tree
If there is a need to invalidate the downloaded compiler cache, then follow next steps:
- Locate
.compiler_cache
directory:
sol-ast-compile --locate-compiler-cache
- Manually remove
list.json
files to invalidate list of available compilers, remove certain compilers, or remove entire directory. Missing pieces will be downloaded again.
After the source is compiled and original compiler has provided the raw AST, the ASTReader
could be used to read the typed universal AST:
import { ASTReader } from "solc-typed-ast";
const reader = new ASTReader();
const sourceUnits = reader.read(result.data);
console.log("Used compiler version: " + result.compilerVersion);
console.log(sourceUnits[0].print());
The typed universal AST has following benefits:
- It is universal between legacy and compact AST. Solc changed AST format between releases, so dependent packages were either forced to migrate from one raw AST version to another or to build adaptive logic (such as implemented in this package).
- It has TypeScript types and proper AST node classes hierarchy.
- It has built-in tree traversal routines, like
walk()
,getChildrenBySelector()
,getParents()
,getClosestParentBySelector()
and so on.
One of the goals of each AST is to provide a way for programmatic modification. To do that, you could modify properties of the typed universal AST nodes. Then use the ASTWriter
to write modified AST back to source code:
import {
ASTWriter,
DefaultASTWriterMapping,
LatestCompilerVersion,
PrettyFormatter
} from "solc-typed-ast";
const formatter = new PrettyFormatter(4, 0);
const writer = new ASTWriter(
DefaultASTWriterMapping,
formatter,
result.compilerVersion ? result.compilerVersion : LatestCompilerVersion
);
for (const sourceUnit of sourceUnits) {
console.log("// " + sourceUnit.absolutePath);
console.log(writer.write(sourceUnit));
}
Package bundles a sol-ast-compile
CLI tool to provide help with development process. It is able to compile the Solidity source and output short AST structure with following:
sol-ast-compile sample.sol --tree
Use --help
to see all available features.
The project has following directory structure:
βββ .compiler_cache # Cache of downloaded compilers (by default, if not configured by SOL_AST_COMPILER_CACHE).
βββ coverage # Test coverage report, produced by "npm test" command.
βββ dist # Generated JavaScript sources for package distribution (produced by "npm run build" and published by "npm publish" commands).
βββ docs # Project documentation and API reference, produced by "npm run docs:render" or "npm run docs:refresh" commands.
βββ src # Original TypeScript sources.
βΒ Β βββ ast # AST-related definitions and logic:
βΒ Β βΒ Β βββ implementation # - Implemented universal AST nodes:
βΒ Β βΒ Β βΒ Β βββ declaration # - declarations or definitions;
βΒ Β βΒ Β βΒ Β βββ expression # - expressions;
βΒ Β βΒ Β βΒ Β βββ meta # - directives, units, specifiers and other information nodes;
βΒ Β βΒ Β βΒ Β βββ statement # - statements;
βΒ Β βΒ Β βΒ Β βββ type # - type-related nodes.
βΒ Β βΒ Β βββ legacy # - Solc legacy raw AST processors, that are producing arguments for constrcuting universal AST nodes.
βΒ Β βΒ Β βββ modern # - Solc modern (or compact) raw AST processors, that are producing arguments for constrcuting universal AST nodes.
βΒ Β βΒ Β βββ postprocessing # - AST postprocessors to apply additional logic (fixes and discovery) during tree finalization process.
βΒ Β βΒ Β βββ writing # - Components to convert universal AST nodes back to Solidity source code.
βΒ Β βββ bin # Executable files, that are shipped with the package and deployed via "npm install" or "npm link" commands.
βΒ Β βββ compile # Compile-related definitions and logic.
βΒ Β βββ misc # Miscellaneous functionality and utility modules.
βΒ Β βββ types # Solc AST typeString parser and AST.
βββ test # Tests:
Β Β βββ integration # - Integration test suites.
Β Β βββ samples # - Solidity and compiler output JSON samples for the tests.
Β Β βββ unit # - Unit test suites.
A key points for better understanding:
- The
ASTNode
is generic implementation of universal AST node and a base class for all concrete node implementations, except Yul nodes. - The
ASTReader
takes raw Solc AST (legacy or modern) and produces universal AST. - The
ASTContext
provides node-to-node dynamic reference resolution map. The example of such reference would be avReferencedDeclaration
property of theIdentifier
node. - The
LegacyNodeProcessor
class,ModernNodeProcessor
class and their descendant classes are raw-to-universal AST conversion bridge. - The compiling-related logic (located in
src/compile/*
) should not have imports from or any bindings to the universal AST (src/ast/*
). Compile-related utils should be standalone as much as possible.
Preinstall NodeJS of compatible version. If there is a need to run different NodeJS versions, consider using NVM or similar tool for your platform.
Clone repository, install and link:
git clone https://github.com/ConsenSys/solc-typed-ast.git
cd solc-typed-ast/
npm install
npm link
Prior to running the tests it would be better to setup local compiler cache:
sol-ast-compile --download-compilers native wasm
Supported platforms are listed here: https://github.com/ethereum/solc-bin
The project documentation is contained in the docs/
directory. It could be built via following command:
npm run docs:refresh
It is also published here: https://consensys.github.io/solc-typed-ast/
The list of known AST node types can be found here.