1. Introduction

This is the reference document for Sweet.js. For a gentle explaination of Sweet’s concepts see the tutorial.

2. Command Line API

  • --out-file <file>: write result to file

  • --out-dir <dir>: write result to directory

  • --no-babel: do not use babel backend

3. Binding Forms

3.1. syntax

syntax <name> = <init>

Bind <name> to the result of evaluating <init> in the compiletime environment. Scoping follows let (i.e. block scoped with a temporal dead zone).

If the result of evaluating <init> is a function, then the result is a syntax transformer.

3.2. syntaxrec

syntaxrec <name> = <init>

Identical to the syntax form except that <name> is also bound inside of <init>. This enables recursive macro definitions.

4. Syntax Transformer

transformer : (TransformerContext) -> List(Syntax)

A syntax transformer is a function bound to a compile-time name. A syntax transformer is invoked with a transformer context that provides access to the syntax at the call-site and returns a list of syntax objects.

4.1. Transformer Context

A transformer context is an iterable object that provides access to syntax at the call-site of a syntax transformer.

TransformerContext = {
  name: () -> Syntax
  next: () -> {
    done: boolean,
    value: Syntax
  }
  expand: (string) -> {
    done: boolean,
    value: Syntax
  }
  mark: () -> Marker
  reset: (Marker?) -> undefined
}

Each call to next returns the syntax object following the transformer call.

A call to expand initiates expansion at the current state of the iterator and matches the specified grammar production. Matching is "greedy" so expand('expr') with the syntax 1 + 2 matches the entire binary expression rather than just 1. The following productions are accepted by expand:

  • Statement with alias stmt

  • AssignmentExpression with alias expr

  • Expression

  • BlockStatement

  • WhileStatement

  • IfStatement

  • ForStatement

  • SwitchStatement

  • BreakStatement

  • ContinueStatement

  • DebuggerStatement

  • WithStatement

  • TryStatement

  • ThrowStatement

  • ClassDeclaration

  • FunctionDeclaration

  • LabeledStatement

  • VariableDeclarationStatement

  • ReturnStatement

  • ExpressionStatement

  • YieldExpression

  • ClassExpression

  • ArrowExpression

  • NewExpression

  • ThisExpression

  • FunctionExpression

  • IdentifierExpression

  • LiteralNumericExpression

  • LiteralInfinityExpression

  • LiteralStringExpression

  • TemplateExpression

  • LiteralBooleanExpression

  • LiteralNullExpression

  • LiteralRegExpExpression

  • ObjectExpression

  • ArrayExpression

  • UnaryExpression

  • UpdateExpression

  • BinaryExpression

  • StaticMemberExpression

  • ComputedMemberExpression

  • AssignmentExpression

  • CompoundAssignmentExpression

  • ConditionalExpression

The name() method returns the syntax object of the macro name at the macro invocation site. This is useful[1] because it allows a macro transformer to get access to the lexical context at the invocation site.

A call to mark returns a pointer to the current state of the iterator.

Calling reset with no arguments returns the context to its initial state, while passing a Marker instance returns the context to the state pointed to by the marker.

syntax m = function (ctx) {
  ctx.expand('expr');
  ctx.reset();

  const a = ctx.next().value;
  ctx.next();

  const marker = ctx.mark();
  ctx.expand('expr');
  ctx.reset(marker);

  const b = ctx.next().value;
  return #`${a} + ${b} + 24`; // 30 + 42 + 24
}
m 30 + 42 + 66

5. Syntax Objects

Syntax objects represent the syntax from the source program. Syntax objects have a number of methods to inspect their contents and construct new syntax object.

5.1. val

Syntax.prototype.val() -> string?

Returns a nullable value representing the textual value of the syntax object. For example, the .val() of the identifier foo is the string "foo". Some syntax objects (in particular delimiters) do not have a reasonable representation of their syntax and so .val() returns null in these cases.

syntax m = ctx => {
  let id = ctx.next().value;
  let delim = ctx.next().value;

  id.val() === 'foo';   // true
  delim.val() === null; // true
  // ...
}
m foo (1)

5.2. lineNumber

Syntax.prototype.lineNumber() -> number

Returns the original line number for this syntax object.

5.3. inner

Syntax.prototype.inner() -> TransformerContext

If the syntax object is a delimiter, returns a transformer context iterator into the delimiter. Otherwise, throw an exception.

syntax m = ctx => {
  let delim = ctx.next().value;
  var arr = [];
  for (let item of delim.inner()) {
    arr.push(item);
  }
  return #`[${arr}]`;
}
m {
  1, 2, 3
}

5.4. from

Syntax.prototype.from(type, value) -> Syntax
  type : string
  value : any

Construct a new syntax object from the provided type and value using the instance syntax object’s lexical context. Valid types are:

  • 'null'

  • 'number'

  • 'string'

  • 'punctuator'

  • 'keyword'

  • 'identifier'

  • 'regularExpression'

  • 'braces'

  • 'brackets'

  • 'parens'

Note

Be careful which syntax object you use to create a new syntax object via from* since the new object will share the original’s lexical context. In most cases you will want to create a "dummy" syntax object inside a macro definition and then use that as a base to create new objects. For example,

syntax to_str = ctx => {
  let dummy = #`dummy`.get(0);
  let arg = ctx.next().value;
  return #`${dummy.fromString(arg.val())}`;
}
to_str foo

You may be tempted to reuse the syntax object provided by ctx.name() but resist that feeling! The ctx.name() syntax object comes from the macro call-site and so any syntax objects created from it will carry the lexical context of the call-site. Sometimes this is what you want, but most of the time this breaks hygiene!

5.5. fromNull

Syntax.prototype.fromNull() -> Syntax

Creates a null literal with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  return #`${dummy.fromNull()}`;
}
m
(expansion)
null

5.6. fromNumber

Syntax.prototype.fromNumber(value) -> Syntax
  value : number

Creates a numeric literal matching value with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  return #`${dummy.fromNumber(1)}`;
}
m
(expansion)
1

5.7. fromString

Syntax.prototype.fromString(value) -> Syntax
  value : string

Creates a string literal matching value with the instance syntax object’s lexical context.

syntax to_str = ctx => {
  let dummy = #`dummy`.get(0);
  let arg = ctx.next().value;
  return #`${dummy.fromString(arg.val())}`;
}
to_str foo
(expansion)
'foo'

5.8. fromPunctuator

Syntax.prototype.fromPunctuator(value) -> Syntax
  value : string

Creates a punctuator (e.g. +, ==, etc.) matching value with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  return #`1 ${dummy.fromPunctuator('+')} 1`;
}
m
(expansion)
1 + 1

5.9. fromKeyword

Syntax.prototype.fromKeyword(value) -> Syntax
  value : string

Creates a keyword matching value with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  return #`${dummy.fromKeyword('let')} x = 1`;
}
m
(expansion)
let x = 1

5.10. fromIdentifier

Syntax.prototype.fromIdentifier(value) -> Syntax
  value : string

Creates a identifier matching value with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  let arg = ctx.next().value;
  return #`${dummy.fromIdentifier(arg.val())}`;
}
m foo
(expansion)
foo

5.11. fromRegularExpression

Syntax.prototype.fromRegularExpression(value) -> Syntax
  value : string

Creates a regular expression literal matching value with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  return #`${dummy.fromRegularExpression('[a-zA-Z]*')}`;
}
m
(expansion)
/[a-zA-Z]/

5.12. fromBraces

Syntax.prototype.fromBraces(inner) -> Syntax
  inner : List(Syntax)

Creates a curly brace delimiter with inner syntax objects inner with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  let block = #`let x = 1;`;
  return #`${dummy.fromBraces(block)}`;
}
m
(expansion)
{
  let x = 1;
}

5.13. fromBrackets

Syntax.prototype.fromBrackets(inner) -> Syntax
  inner : List(Syntax)

Creates a square bracket delimiter with inner syntax objects inner with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  let elements = #`1, 2, 3`;
  return #`${dummy.fromBrackets(elements)}`;
}
m
(expansion)
[1, 2, 3]

5.14. fromParens

Syntax.prototype.fromParens(inner) -> Syntax
  inner : List(Syntax)

Creates a parenthesis delimiter with inner syntax objects inner with the instance syntax object’s lexical context.

syntax m = ctx => {
  let dummy = #`dummy`.get(0);
  let expr = #`5 * 5`;
  return #`1 + ${dummy.fromParens(expr)}`;
}
m
(expansion)
1 + (5 * 5)

5.15. isIdentifier

Syntax.prototype.isIdentifier() -> boolean

Returns true if the syntax object is an identifier.

5.16. isBooleanLiteral

Syntax.prototype.isBooleanLiteral() -> boolean

Returns true if the syntax object is a boolean literal.

5.17. isNullLiteral

Syntax.prototype.isNullLiteral() -> boolean

Returns true if the syntax object is a null literal.

5.18. isNumericLiteral

Syntax.prototype.isNumericLiteral() -> boolean

Returns true if the syntax object is a numeric literal.

5.19. isStringLiteral

Syntax.prototype.isStringLiteral() -> boolean

Returns true if the syntax object is a string literal.

5.20. isKeyword

Syntax.prototype.isKeyword() -> boolean

Returns true if the syntax object is a keyword.

5.21. isPunctuator

Syntax.prototype.isPunctuator() -> boolean

Returns true if the syntax object is a puncuator.

5.22. isRegularExpression

Syntax.prototype.isRegularExpression() -> boolean

Returns true if the syntax object is a regular expression literal.

5.23. isTemplate

Syntax.prototype.isTemplate() -> boolean

Returns true if the syntax object is a template literal.

5.24. isDelimiter

Syntax.prototype.isDelimiter() -> boolean

Returns true if the syntax object is a delimiter.

5.25. isParens

Syntax.prototype.isParens() -> boolean

Returns true if the syntax object is a parenthesis delimiter (e.g. ( …​ )).

5.26. isBrackets

Syntax.prototype.isBrackets() -> boolean

Returns true if the syntax object is a bracket delimiter (e.g. [ …​ ]).

5.27. isBraces

Syntax.prototype.isBraces() -> boolean

Returns true if the syntax object is a braces delimiter (e.g. { …​ }).

5.28. isSyntaxTemplate

Syntax.prototype.isSyntaxTemplate() -> boolean

Returns true if the syntax object is a syntax template.

6. Syntax Templates

Syntax templates construct a list of syntax objects from a literal representation using backtick (#`foo bar baz`). They are similar to ES2015 templates but with the special sweet.js specific # template tag.

Syntax templates support interpolations just like normal templates via ${…​}:

syntax m = function (ctx) {
  return #`${ctx.next().value} + 24`;
}
m 42

The expressions inside an interpolation must evaluate to a syntax object, an array, a list, or an transformer context.


1. or will become useful as more features are implemented in Sweet