• This repository has been archived on 26/Jan/2022
  • Stars
    star
    413
  • Rank 100,805 (Top 3 %)
  • Language
  • Created almost 8 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Unambiguous JavaScript Grammar

Status DRAFT
Authors @jdalton @bmeck
Date June 14, 2016

Proposal Accepted!

On July 6, 2016 this proposal was accepted as section 5.1 of the Node ES6 Module Interoperability draft proposal ‼️

TL;DR

  • CJS and ES modules just work without new extensions, extra ceremony, or excessive scaffolding
  • Performance is generally on par with existing CJS module loading
  • Performance is significantly improved for ES modules over transpilation workflows
  • Change JS grammars for Script and Module to be unambiguous
  • Determine grammar of .js files by parsing as one grammar and falling back to others

Problem

The Script and Module goal of ECMA262 have a grammatical ambiguity where some code can run in both goals, having the exact same source text, but produce different results. Unlike "use strict", the signal to have a specific behavior is not in the code, thus the code has a multitude of possible effects which are not controlled by the programmer.

Note: The following example highlights a few of the effects of ambiguous grammar and is by no means exhaustive.

Example

function foo(value) {
  value = value || '';
  var args = [].slice.call(arguments);
  args.unshift(this);
  return args;
}
foo(null);

Differences:

script module
variable scope of foo global local
arguments object modified unmodified
this binding of foo global undefined

Since there is no way in source text to enforce the goal with the current grammar; this leads to the behavior of certain constructs being undefined by the programmer, and defined by the host environment. In turn, existing code could be run in the wrong goal and partially function, or function without errors but produce incorrect results.

ECMA262 Solution

Require that Module source text has at least one import or export declaration. A module with only an import declaration and no export declaration is valid. Modules, that do not export anything, should specify an export {} to make intentions clear and avoid accidental parse errors while removing import declarations. The export {} is not new syntax and does not export an empty object. It is simply the standard way to specify exporting nothing.

Note: While the ES2015 specification does not forbid this extension, Node wants to avoid acting as a rogue agent. Node has a TC39 representative, @bmeck, to champion this proposal. A specification change or at least an official endorsement of this Node proposal would be welcomed. If a resolution is not possible, this proposal will fallback to the previous .mjs file extension proposal.

Script Example

function foo(value) {
  value = value || '';
  var args = [].slice.call(arguments);
  args.unshift(this);
  return args;
}
foo(null);
script module (cannot parse)
variable scope of foo global n/a
arguments object modified n/a
this binding of foo global n/a

Module Example

function foo(value) {
  value = value || '';
  var args = [].slice.call(arguments);
  args.unshift(this);
  return args;
}
foo(null);
export {};
script (cannot parse) module
variable scope of foo n/a local
arguments object n/a unmodified
this binding of foo n/a undefined

Problem

Node currently requires a means for programmers to signal what goal their code is written to run in.

Leading solutions have either hefty ecosystem tolls, ceremony, or scaffolding. They lack a way to define the intent of the source text from the ECMA262 standard.

Solution

A package opts-in to the Module goal by specifying "module" as the parse goal field (name not final) in its package.json. Package dependencies are not affected by the opt-in and may be a mix of CJS and ES module packages. If a parse goal is not specified, then attempt to parse source text as the preferred goal (Script for now since most modules are CJS). If there is a parse error that may allow another goal to parse, then parse as the other goal, and so on. After this, the goal is known unambiguously and the environment can safely perform initialization without the possibility of the source text being run in the wrong goal.

Algorithm

Parse (source, goal, throws)

The abstract operation to parse source text as a given goal.

  1. Bootstrap source for goal.
  2. Parse source as goal.
  3. If success, return true.
  4. If throws, throw exception.
  5. Return false.

Operation

  1. If a package parse goal is specified, then

  2. Let goal be the resolved parse goal.

  3. Call Parse(source, goal, true) and return.

  4. Else fallback to multiple parse.

  5. If Parse(Source, Script, false) is true, then 1. Return.

  6. Else 1. Call Parse(Source, Module, true).

Note: A host can choose either goal to parse first and may change their order over time or as new parse goals are introduced. Feel free to swap the order of Script and Module.

Implementation

To improve performance, host environments may want to specify a goal to parse first. This can be done in several ways:
cache on disk, a command line flag, a manifest file, HTTP header, file extension, etc.

Tooling Concerns

Some tools, outside of Node, may not have access to a JS parser (Bash programs, some asset pipelines, etc.). These tools generally operate on files as opaque blobs / plain text files and can use the techniques, listed under Implementation, to get parse goal information.

External Examples and Impact

  • Esprima relies on a --module flag to signal the Module goal. However, this has proven to be unintuitive for many users. Unambiguous Script and Module goals would enable things to “just work” without flags.

  • Facebook Flow performs a series of inferences to detect CJS and ES modules. Unambiguous Script and Module goals would improve its ability to determine module types.

  • JSCS can accept input through stdin, so identification of parse goals by source text is ideal.

  • Linters, like xo, could use unambiguous Script and Module goals to enable module specific linting rules without extra configuration.

  • Microsoft packaged web applications can benefit from unambiguous Script and Module goals. The bytecode cache for a packaged web application is generated upon installation. When the application is running, files are loaded by script tags so their intended parse goals are understood. However, bytecode cache generation is done without running the application so the intended parse goals are unknown. Because of this, the bytecode cache is generated for the Script goal and ignored for ES modules.

  • TypeScript has taken the stance from early on that a script becomes a module when it has at least one import or export declaration. Over the years they have experienced very few user issues with this approach.

Special Thanks

This proposal would not have been possible without the tireless effort, conviction, and collaboration of
@bmeck, @dherman, and @wycats.

While iterating on this proposal we have reached out to several people from areas affected by it.
Although opinions on this proposal are varied, I am grateful for feedback from:

Thank you! ❤️
@jdalton

More Repositories

1

proposals

Tracking ECMAScript Proposals
17,177
star
2

ecma262

Status, process, and documents for ECMA-262
HTML
14,437
star
3

proposal-pipeline-operator

A proposal for adding a useful pipe operator to JavaScript.
HTML
7,380
star
4

proposal-pattern-matching

Pattern matching syntax for ECMAScript
HTML
5,341
star
5

proposal-optional-chaining

HTML
4,952
star
6

proposal-type-annotations

ECMAScript proposal for type syntax that is erased - Stage 1
JavaScript
4,090
star
7

proposal-temporal

Provides standard objects and functions for working with dates and times.
HTML
3,135
star
8

proposal-observable

Observables for ECMAScript
JavaScript
3,032
star
9

proposal-signals

A proposal to add signals to JavaScript.
TypeScript
2,668
star
10

proposal-decorators

Decorators for ES6 classes
2,640
star
11

proposal-record-tuple

ECMAScript proposal for the Record and Tuple value types. | Stage 2: it will change!
HTML
2,423
star
12

test262

Official ECMAScript Conformance Test Suite
JavaScript
2,073
star
13

proposal-dynamic-import

import() proposal for JavaScript
HTML
1,859
star
14

proposal-bind-operator

This-Binding Syntax for ECMAScript
1,736
star
15

proposal-class-fields

Orthogonally-informed combination of public and private fields proposals
HTML
1,720
star
16

proposal-async-await

Async/await for ECMAScript
HTML
1,577
star
17

proposal-object-rest-spread

Rest/Spread Properties for ECMAScript
HTML
1,496
star
18

proposal-shadowrealm

ECMAScript Proposal, specs, and reference implementation for Realms
HTML
1,365
star
19

proposal-nullish-coalescing

Nullish coalescing proposal x ?? y
HTML
1,233
star
20

proposal-iterator-helpers

Methods for working with iterators in ECMAScript
HTML
1,220
star
21

proposal-top-level-await

top-level `await` proposal for ECMAScript (stage 4)
HTML
1,082
star
22

proposal-partial-application

Proposal to add partial application to ECMAScript
HTML
1,002
star
23

proposal-do-expressions

Proposal for `do` expressions
HTML
990
star
24

agendas

TC39 meeting agendas
JavaScript
952
star
25

proposal-binary-ast

Binary AST proposal for ECMAScript
945
star
26

proposal-built-in-modules

HTML
886
star
27

proposal-async-iteration

Asynchronous iteration for JavaScript
HTML
854
star
28

proposal-explicit-resource-management

ECMAScript Explicit Resource Management
JavaScript
671
star
29

proposal-operator-overloading

JavaScript
610
star
30

proposal-string-dedent

TC39 Proposal to remove common leading indentation from multiline template strings
HTML
588
star
31

proposal-bigint

Arbitrary precision integers in JavaScript
HTML
560
star
32

proposal-set-methods

Proposal for new Set methods in JS
HTML
557
star
33

proposal-import-attributes

Proposal for syntax to import ES modules with assertions
HTML
538
star
34

ecmascript_simd

SIMD numeric type for EcmaScript
JavaScript
536
star
35

proposal-slice-notation

HTML
515
star
36

proposal-change-array-by-copy

Provides additional methods on Array.prototype and TypedArray.prototype to enable changes on the array by returning a new copy of it with the change.
HTML
509
star
37

ecma402

Status, process, and documents for ECMA 402
HTML
506
star
38

notes

TC39 meeting notes
JavaScript
496
star
39

proposal-class-public-fields

Stage 2 proposal for public class fields in ECMAScript
HTML
489
star
40

proposal-iterator.range

A proposal for ECMAScript to add a built-in Iterator.range()
HTML
464
star
41

proposal-uuid

UUID proposal for ECMAScript (Stage 1)
JavaScript
462
star
42

proposal-throw-expressions

Proposal for ECMAScript 'throw' expressions
JavaScript
425
star
43

proposal-module-expressions

HTML
424
star
44

proposal-decimal

Built-in decimal datatype in JavaScript
HTML
408
star
45

proposal-array-grouping

A proposal to make grouping of array items easier
HTML
407
star
46

proposal-async-context

Async Context for JavaScript
HTML
406
star
47

proposal-weakrefs

WeakRefs
HTML
404
star
48

proposal-error-cause

TC39 proposal for accumulating errors
HTML
378
star
49

proposal-ecmascript-sharedmem

Shared memory and atomics for ECMAscript
HTML
376
star
50

proposal-cancelable-promises

Former home of the now-withdrawn cancelable promises proposal for JavaScript
Shell
376
star
51

proposal-relative-indexing-method

A TC39 proposal to add an .at() method to all the basic indexable classes (Array, String, TypedArray)
HTML
351
star
52

proposal-first-class-protocols

a proposal to bring protocol-based interfaces to ECMAScript users
350
star
53

proposal-global

ECMAScript Proposal, specs, and reference implementation for `global`
HTML
346
star
54

proposal-private-methods

Private methods and getter/setters for ES6 classes
HTML
344
star
55

proposal-numeric-separator

A proposal to add numeric literal separators in JavaScript.
HTML
327
star
56

proposal-private-fields

A Private Fields Proposal for ECMAScript
HTML
320
star
57

proposal-object-from-entries

TC39 proposal for Object.fromEntries
HTML
317
star
58

proposal-module-declarations

JavaScript Module Declarations
HTML
314
star
59

proposal-promise-allSettled

ECMAScript Proposal, specs, and reference implementation for Promise.allSettled
HTML
314
star
60

tc39.github.io

Get involved in specifying JavaScript
HTML
313
star
61

proposal-regex-escaping

Proposal for investigating RegExp escaping for the ECMAScript standard
JavaScript
309
star
62

proposal-await.ops

Introduce await.all / await.race / await.allSettled / await.any to simplify the usage of Promises
HTML
308
star
63

proposal-logical-assignment

A proposal to combine Logical Operators and Assignment Expressions
HTML
302
star
64

proposal-export-default-from

Proposal to add `export v from "mod";` to ECMAScript.
HTML
297
star
65

proposal-promise-finally

ECMAScript Proposal, specs, and reference implementation for Promise.prototype.finally
HTML
278
star
66

proposal-asset-references

Proposal to ECMAScript to add first-class location references relative to a module
268
star
67

proposal-cancellation

Proposal for a Cancellation API for ECMAScript
HTML
262
star
68

proposal-json-modules

Proposal to import JSON files as modules
HTML
259
star
69

proposal-promise-with-resolvers

HTML
255
star
70

proposal-string-replaceall

ECMAScript proposal: String.prototype.replaceAll
HTML
254
star
71

proposal-export-ns-from

Proposal to add `export * as ns from "mod";` to ECMAScript.
HTML
241
star
72

proposal-ses

Draft proposal for SES (Secure EcmaScript)
HTML
217
star
73

proposal-structs

JavaScript Structs: Fixed Layout Objects
216
star
74

proposal-intl-relative-time

`Intl.RelativeTimeFormat` specification [draft]
HTML
215
star
75

proposal-flatMap

proposal for flatten and flatMap on arrays
HTML
215
star
76

proposal-json-parse-with-source

Proposal for extending JSON.parse to expose input source text.
HTML
204
star
77

ecmarkup

An HTML superset/Markdown subset source format for ECMAScript and related specifications
TypeScript
201
star
78

proposal-promise-any

ECMAScript proposal: Promise.any
HTML
198
star
79

proposal-decorators-previous

Decorators for ECMAScript
HTML
184
star
80

proposal-smart-pipelines

Old archived draft proposal for smart pipelines. Go to the new Hack-pipes proposal at js-choi/proposal-hack-pipes.
HTML
181
star
81

proposal-defer-import-eval

A proposal for introducing a way to defer evaluate of a module
HTML
174
star
82

proposal-array-filtering

A proposal to make filtering arrays easier
HTML
171
star
83

proposal-optional-chaining-assignment

`a?.b = c` proposal
168
star
84

proposal-array-from-async

Draft specification for a proposed Array.fromAsync method in JavaScript.
HTML
167
star
85

proposal-extractors

Extractors for ECMAScript
JavaScript
166
star
86

proposal-upsert

ECMAScript Proposal, specs, and reference implementation for Map.prototype.upsert
HTML
165
star
87

proposal-ptc-syntax

Discussion and specification for an explicit syntactic opt-in for Tail Calls.
HTML
165
star
88

how-we-work

Documentation of how TC39 operates and how to participate
161
star
89

proposal-collection-methods

HTML
160
star
90

proposal-Array.prototype.includes

Spec, tests, reference implementation, and docs for ESnext-track Array.prototype.includes
HTML
157
star
91

proposal-error-stacks

ECMAScript Proposal, specs, and reference implementation for Error.prototype.stack / System.getStack
HTML
156
star
92

proposal-promise-try

ECMAScript Proposal, specs, and reference implementation for Promise.try
HTML
154
star
93

proposal-hashbang

#! for JS
HTML
148
star
94

proposal-resizablearraybuffer

Proposal for resizable array buffers
HTML
145
star
95

proposal-import-meta

import.meta proposal for JavaScript
HTML
145
star
96

proposal-intl-segmenter

Unicode text segmentation for ECMAScript
HTML
145
star
97

proposal-extensions

Extensions proposal for ECMAScript
HTML
143
star
98

proposal-seeded-random

Proposal for an options argument to be added to JS's Math.random() function, and some options to start it with.
HTML
143
star
99

proposal-intl-duration-format

141
star
100

proposal-regexp-unicode-property-escapes

Proposal to add Unicode property escapes `\p{…}` and `\P{…}` to regular expressions in ECMAScript.
HTML
134
star