• Stars
    star
    123
  • Rank 290,145 (Top 6 %)
  • Language
    TypeScript
  • License
    Apache License 2.0
  • Created almost 4 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A TypeScript package providing a normalized typed Solidity AST along with the utilities necessary to generate the AST (from Solc) and traverse/manipulate it.

NodeJS CI Coverage Documentation npm npm downloads License

solc-typed-ast

A TypeScript package providing a normalized typed Solidity AST along with the utilities necessary to generate the AST (from Solc) and traverse/manipulate it.

Features

  • Various Solidity compiler versions support, starting from 0.4.13.
  • Various compiler selection strategies, including compiler auto-detection via pragma solidity directives in Solidity sources with possible fallback options.
  • Interaction with AST nodes, using type definitions.
  • Compensation of differences between legacy and compact Solidity AST.
  • Referenced AST nodes linking.
  • Easy tree traversal with siblings, parent/children relations, recursive walks and node selection via custom predicates.
  • XPath AST traversal (based on jSel package).
  • Generating the source code from the AST.

Installation

Package could be installed globally via following command:

npm install -g solc-typed-ast

Also it can be installed as the dependency:

npm install --save solc-typed-ast

Usage

Easy compiling

The package introduces easy and universal compiler invocation. Starting with Solidity 0.4.13, source compilation could be done universally:

import { CompileFailedError, CompileResult, compileSol } from "solc-typed-ast";

let result: CompileResult;

try {
    result = await compileSol("sample.sol", "auto", []);
} catch (e) {
    if (e instanceof CompileFailedError) {
        console.error("Compile errors encountered:");

        for (const failure of e.failures) {
            console.error(`Solc ${failure.compilerVersion}:`);

            for (const error of failure.errors) {
                console.error(error);
            }
        }
    } else {
        console.error(e.message);
    }
}

console.log(result);

The second argument with the "auto" value specifies a compiler selection strategy. If "auto" is specified and source code contains valid pragma solidity directive, then compiler version will be automatically picked from it. If compile process will not succeed, the execution will fall back to "compiler guessing": trying to compile source with a few different versions of the new and old Solidity compilers. The other option would be to specify a concrete supported compiler version string, like "0.7.0" for example. There is also a support for various compiler selection strategies, including used-defined custom ones (CompilerVersionSelectionStrategy interface implementations).

Used compilers

Package supports switching between native binary Solc compilers and its WASM versions. The CLI option --compiler-kind and kind argument of compile*() functions family may be used for that purpose. Compilers are downloaded on-demand to the directory .compiler_cache at the package installation directory (by default). The compiler cache location may be customized by setting SOL_AST_COMPILER_CACHE environment variable to a custom path. For example:

SOL_AST_COMPILER_CACHE=~/.compiler_cache sol-ast-compile sample.sol --compiler-kind native --tree

or

export SOL_AST_COMPILER_CACHE=~/.compiler_cache

sol-ast-compile sample.sol --compiler-kind native --tree

Invalidation of downloaded compiler cache

If there is a need to invalidate the downloaded compiler cache, then follow next steps:

  1. Locate .compiler_cache directory:
sol-ast-compile --locate-compiler-cache
  1. Manually remove list.json files to invalidate list of available compilers, remove certain compilers, or remove entire directory. Missing pieces will be downloaded again.

Typed universal AST

After the source is compiled and original compiler has provided the raw AST, the ASTReader could be used to read the typed universal AST:

import { ASTReader } from "solc-typed-ast";

const reader = new ASTReader();
const sourceUnits = reader.read(result.data);

console.log("Used compiler version: " + result.compilerVersion);
console.log(sourceUnits[0].print());

The typed universal AST has following benefits:

  • It is universal between legacy and compact AST. Solc changed AST format between releases, so dependent packages were either forced to migrate from one raw AST version to another or to build adaptive logic (such as implemented in this package).
  • It has TypeScript types and proper AST node classes hierarchy.
  • It has built-in tree traversal routines, like walk(), getChildrenBySelector(), getParents(), getClosestParentBySelector() and so on.

Converting an AST back to source

One of the goals of each AST is to provide a way for programmatic modification. To do that, you could modify properties of the typed universal AST nodes. Then use the ASTWriter to write modified AST back to source code:

import {
    ASTWriter,
    DefaultASTWriterMapping,
    LatestCompilerVersion,
    PrettyFormatter
} from "solc-typed-ast";

const formatter = new PrettyFormatter(4, 0);
const writer = new ASTWriter(
    DefaultASTWriterMapping,
    formatter,
    result.compilerVersion ? result.compilerVersion : LatestCompilerVersion
);

for (const sourceUnit of sourceUnits) {
    console.log("// " + sourceUnit.absolutePath);
    console.log(writer.write(sourceUnit));
}

CLI tool

Package bundles a sol-ast-compile CLI tool to provide help with development process. It is able to compile the Solidity source and output short AST structure with following:

sol-ast-compile sample.sol --tree

Use --help to see all available features.

Project overview

The project has following directory structure:

β”œβ”€β”€ .compiler_cache             # Cache of downloaded compilers (by default, if not configured by SOL_AST_COMPILER_CACHE).
β”œβ”€β”€ coverage                    # Test coverage report, produced by "npm test" command.
β”œβ”€β”€ dist                        # Generated JavaScript sources for package distribution (produced by "npm run build" and published by "npm publish" commands).
β”œβ”€β”€ docs                        # Project documentation and API reference, produced by "npm run docs:render" or "npm run docs:refresh" commands.
β”œβ”€β”€ src                         # Original TypeScript sources.
β”‚Β Β  β”œβ”€β”€ ast                     # AST-related definitions and logic:
β”‚Β Β  β”‚Β Β  β”œβ”€β”€ implementation      #   - Implemented universal AST nodes:
β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ declaration     #       - declarations or definitions;
β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ expression      #       - expressions;
β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ meta            #       - directives, units, specifiers and other information nodes;
β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ statement       #       - statements;
β”‚Β Β  β”‚Β Β  β”‚Β Β  └── type            #       - type-related nodes.
β”‚Β Β  β”‚Β Β  β”œβ”€β”€ legacy              #   - Solc legacy raw AST processors, that are producing arguments for constrcuting universal AST nodes.
β”‚Β Β  β”‚Β Β  β”œβ”€β”€ modern              #   - Solc modern (or compact) raw AST processors, that are producing arguments for constrcuting universal AST nodes.
β”‚Β Β  β”‚Β Β  β”œβ”€β”€ postprocessing      #   - AST postprocessors to apply additional logic (fixes and discovery) during tree finalization process.
β”‚Β Β  β”‚Β Β  └── writing             #   - Components to convert universal AST nodes back to Solidity source code.
β”‚Β Β  β”œβ”€β”€ bin                     # Executable files, that are shipped with the package and deployed via "npm install" or "npm link" commands.
β”‚Β Β  β”œβ”€β”€ compile                 # Compile-related definitions and logic.
β”‚Β Β  β”œβ”€β”€ misc                    # Miscellaneous functionality and utility modules.
β”‚Β Β  └── types                   # Solc AST typeString parser and AST.
└── test                        # Tests:
 Β Β  β”œβ”€β”€ integration             #   - Integration test suites.
 Β Β  β”œβ”€β”€ samples                 #   - Solidity and compiler output JSON samples for the tests.
 Β Β  └── unit                    #   - Unit test suites.

A key points for better understanding:

  • The ASTNode is generic implementation of universal AST node and a base class for all concrete node implementations, except Yul nodes.
  • The ASTReader takes raw Solc AST (legacy or modern) and produces universal AST.
  • The ASTContext provides node-to-node dynamic reference resolution map. The example of such reference would be a vReferencedDeclaration property of the Identifier node.
  • The LegacyNodeProcessor class, ModernNodeProcessor class and their descendant classes are raw-to-universal AST conversion bridge.
  • The compiling-related logic (located in src/compile/*) should not have imports from or any bindings to the universal AST (src/ast/*). Compile-related utils should be standalone as much as possible.

Development installation

Prerequisites

Preinstall NodeJS of compatible version. If there is a need to run different NodeJS versions, consider using NVM or similar tool for your platform.

Clone and build

Clone repository, install and link:

git clone https://github.com/ConsenSys/solc-typed-ast.git
cd solc-typed-ast/
npm install
npm link

Prior to running the tests it would be better to setup local compiler cache:

sol-ast-compile --download-compilers native wasm

Supported platforms are listed here: https://github.com/ethereum/solc-bin

Project documentation and API reference

The project documentation is contained in the docs/ directory. It could be built via following command:

npm run docs:refresh

It is also published here: https://consensys.github.io/solc-typed-ast/

The list of AST node types

The list of known AST node types can be found here.

More Repositories

1

smart-contract-best-practices

A guide to smart contract security best practices
HTML
7,473
star
2

ethereum-developer-tools-list

A guide to available tools and platforms for developing on Ethereum.
5,321
star
3

quorum

A permissioned implementation of Ethereum supporting data privacy
Go
4,581
star
4

mythril

Security analysis tool for EVM bytecode. Supports smart contracts built for Ethereum, Hedera, Quorum, Vechain, Rootstock, Tron and other EVM-compatible blockchains.
Python
3,817
star
5

Tokens

Ethereum Token Contracts
JavaScript
2,059
star
6

eth-lightwallet

Lightweight JS Wallet for Node and the browser
JavaScript
1,466
star
7

gnark

gnark is a fast zk-SNARK library that offers a high-level API to design circuits. The library is open source and developed under the Apache 2.0 license
Go
1,383
star
8

surya

A set of utilities for exploring Solidity contracts
JavaScript
1,076
star
9

ethql

A GraphQL interface to Ethereum πŸ”₯
TypeScript
623
star
10

abi-decoder

Nodejs and Javascript library for decoding data params and events from ethereum transactions
JavaScript
605
star
11

vscode-solidity-auditor

Solidity language support and visual security auditor for Visual Studio Code
JavaScript
568
star
12

teku

Java Implementation of the Ethereum 2.0 Beacon Chain
Java
557
star
13

cakeshop

An integrated development environment and SDK for Ethereum-like ledgers
JavaScript
507
star
14

gnark-crypto

gnark-crypto provides elliptic curve and pairing-based cryptography on BN, BLS12, BLS24 and BW6 curves. It also provides various algorithms (algebra, crypto) of particular interest to zero knowledge proof systems.
Go
487
star
15

Token-Factory

Basic Token Factory dapp.
JavaScript
479
star
16

constellation

Peer-to-peer encrypted message exchange
Haskell
379
star
17

UniversalToken

Implementation of Universal Token for Assets and Payments
JavaScript
346
star
18

doc.linea

Linea documentation
JavaScript
338
star
19

quorum-examples

Examples for Quorum
Shell
317
star
20

scribble

Scribble instrumentation tool
TypeScript
315
star
21

anonymous-zether

A private payment system for Ethereum-based blockchains, with no trusted setup.
Solidity
295
star
22

defi-score

DeFi Score: An open framework for evaluating DeFi protocols
Python
280
star
23

EthOn

EthOn - The Ethereum Ontology
HTML
245
star
24

solidity-metrics

Solidity Code Metrics
JavaScript
235
star
25

Mahuta

IPFS Storage service with search capability
Java
230
star
26

tessera

Tessera - Enterprise Implementation of Quorum's transaction manager
Java
169
star
27

PLCRVoting

Partial Lock Commit Reveal Voting System that utilizes ERC20 Tokens
JavaScript
169
star
28

ethjsonrpc

Python JSON-RPC client for the Ethereum blockchain
Python
156
star
29

linea-attestation-registry

Verax is a shared registry for storing attestations of public interest on EVM chains, designed to enhance data discoverability and consumption for dApps across the network.
TypeScript
129
star
30

zero-knowledge-proofs

Zero Knowledge Proofs and how they can be implemented in Quorum
C++
128
star
31

evm-dafny

An EVM interpreter in Dafny
Dafny
125
star
32

python-solidity-parser

An experimental Solidity parser for Python built on top of a robust ANTLR4 grammar πŸ“š
Python
125
star
33

truffle-security

MythX smart contract security verification plugin for Truffle Framework
JavaScript
124
star
34

web3signer

Web3Signer is an open-source signing service capable of signing on multiple platforms (Ethereum1 and 2, Filecoin) using private keys stored in an external vault, or encrypted on a disk.
Java
122
star
35

daedaluzz

Benchmark Generator for Smart-Contract Fuzzers
Solidity
120
star
36

btcrelay-fetchd

Just the fetchd script of btcrelay
Python
116
star
37

ethsigner

DEPRECATED. A transaction signing application to be used with a web3 provider.
Java
112
star
38

ethereum-dissectors

πŸ”Wireshark dissectors for Ethereum devp2p protocols
C
109
star
39

quorum-dev-quickstart

The Quorum Developer Quickstart utility can be used to rapidly generate local Quorum blockchain networks for development and demo purposes using Besu, GoQuorum, and Codefi Orchestrate.
Solidity
108
star
40

truffle-webpack-demo

A demo Webpack + React App using truffle-solidity-loader
JavaScript
95
star
41

orion

Orion is a PegaSys component for doing private transactions
Java
92
star
42

blockchainSecurityDB

JavaScript
90
star
43

quorum-kubernetes

Helm charts for Hyperledger Besu and GoQuorum
Mustache
85
star
44

quorum-docs

Documentation assets for Quorum
84
star
45

mythx-cli

A command line interface for the MythX smart contract security analysis API
Python
83
star
46

gpact

General Purpose Atomic Crosschain Transaction Protocol
Java
81
star
47

bytecode-verifier

Compile Solidity source code and verify its bytecode matches the blockchain
JavaScript
80
star
48

goff

goff (go finite field) is a unix-like tool that generates fast field arithmetic in Go.
Go
76
star
49

starknet-snap

The MetaMask Snap for Starknet
TypeScript
74
star
50

linea-contracts

Linea smart-contracts
Solidity
73
star
51

eth2.0-dafny

Eth2.0 spec in Dafny
Dafny
72
star
52

zsl-q

ZSL on Quorum
C++
71
star
53

security-workshop-for-devs

Secure smart contract development workshop hosted by ConsenSys Diligence and MythX.
70
star
54

Legions

Ethereum/EVM Node Security Toolkit
Python
69
star
55

support-metamask-io

Public-facing repository of content live on support.metamask.io. Open for contributions and suggestions.
MDX
66
star
56

quorum-docker-Nnodes

Run a bunch of Quorum nodes, each in a separate Docker container.
Shell
65
star
57

Project-Alchemy

Ethereum-Zcash Integration effort
63
star
58

handel

Multi-Signature Aggregation in a Large Byzantine Committees
Go
53
star
59

qubernetes

Quorum on Kubernetes.
Go
52
star
60

Uniswap-audit-report-2018-12

51
star
61

quorum-tools

Tools for running Quorum clusters and integration tests
Haskell
51
star
62

doc.teku

ConsenSys Ethereum 2.0 client
CSS
47
star
63

vscode-solidity-metrics

Generate Solidity Source Code Metrics, Complexity and Risk profile reports for your project.
JavaScript
46
star
64

private-networks-deployment-scripts

This repository contains out-of-the-box deployment scripts for private PoA networks
Shell
45
star
65

awesome-quorum

A curated list of awesome softwares, libraries, tools, articles, educational resources, discussion channels and more to build on ConsenSys Quorum.
45
star
66

wittgenstein

Simulator for some PoS or consensus algorithms. Includes dfinity, casper IMD and others
Java
45
star
67

linea-tutorials

An EVM-equivalent zk-rollup for scaling Ethereum dapps
Shell
43
star
68

vscode-ethover

Ethereum Account Address Hover Info and Actions
JavaScript
42
star
69

permissioning-smart-contracts

Smart contracts for the Besu permissioning system
TypeScript
41
star
70

besu-sample-networks

Hyperledger Besu Ethereum client quick-start makes you able to simply test all Besu features.
40
star
71

linea-token-list

Linea Token List
TypeScript
39
star
72

0x-review

Security review of 0x smart contracts
HTML
39
star
73

quorum-key-manager

A universal Key & Account Management solution for blockchain applications.
Go
39
star
74

mythx-playground

Exercises to go along with smart contract security workshops by MythX and ConsenSys Diligence
Solidity
39
star
75

kubernetes-action

GitHub Action to run kubectl
Dockerfile
38
star
76

evm-analyzer-benchmark-suite

A benchmark suite for evaluating the precision of EVM code analysis tools.
HTML
38
star
77

quorum-cloud

Deploy Quorum network in a cloud provider of choice
HCL
36
star
78

quorum.js

Quorum.js is an extension to web3.js providing support for JP Morgan's Quorum API
JavaScript
36
star
79

web3js-eea

EEA JavaScript libraries.
JavaScript
35
star
80

truffle-solidity-loader

A Webpack loader that will parse and provision Solidity files to Javascript using Truffle for compilation
JavaScript
35
star
81

secureum-diligence-bootcamp

Solidity
35
star
82

rimble-app-demo

React Ethereum dApp demonstrating onboarding and transaction UX
JavaScript
35
star
83

linea-monorepo

The principal Linea repository. This mainly includes the smart contracts covering Linea's core functions, the prover in charge of generating ZK proofs, the coordinator responsible for multiple orchestrations, and the postman to execute bridge messages.
Go
35
star
84

linea-tracer

Part of the Linea stack responsible for extracting data from the execution of an EVM client in order to construct large matrices called execution traces.
Java
34
star
85

infura-sdk

Infura NFT SDK
TypeScript
34
star
86

web3studio-soy

Static Websites on the Distributed Web
JavaScript
33
star
87

pythx

A Python library for the MythX smart contract security analysis platform
Python
33
star
88

quorum-aws

Tools for deploying Quorum clusters to AWS
HCL
33
star
89

react-metamask

JavaScript
32
star
90

diligence-fuzzing

Python
32
star
91

boilerplate-react

React app boilerplate by ConsenSys France
JavaScript
29
star
92

hellhound

HellHound is a decentralized blind computation platform.
Go
29
star
93

aragraph

Visualize your Aragon DAO Templates
JavaScript
29
star
94

quorum-wizard

Quorum Wizard is a command line tool that allow users to set up a development Quorum network on their local machine in less than 2 minutes.
JavaScript
28
star
95

doc.goquorum

Documentation site for GoQuorum, the ConsenSys Enterprise Ethereum client
CSS
27
star
96

quorum-explorer

A light-weight front-end explorer for Besu and GoQuorum to visualise private networks and deploy smart contracts
TypeScript
27
star
97

mythxjs

TypeScript
26
star
98

hackathon-2021-dapp-workshop

JavaScript
25
star
99

0x_audit_report_2018-07-23

0x Protocol v2 Audit
HTML
24
star
100

web3js-quorum

JavaScript
24
star