Flowscript language reference

Flowscript is a small language for trading strategies and indicators. A program reads time series (prices, volumes, event tapes), computes values over them, and fires actions when its conditions turn true. It is deterministic: the same inputs always produce the same values and the same actions, which is why a backtest over history and a live run over new bars make the same decisions.

This reference describes the language as flowscript-core and @flowscript/core implement it, without ftw. Every program in a flowscript block below is compiled by the library's tests (src/docs_reference.rs) against the host in The examples' host and replayed over a synthetic history, so the examples cannot drift from the compiler. The semantics are specified in FLOWSCRIPT_V2_SPEC.md at the repository root; this is the user's view of them.

• A first program
• The host
• Values and types
• Expressions
• Functions, lambdas and fold
• Metrics
• Triggers
• Actions and handlers
• Enums and match
• Phase machines
• Tapes
• The prelude
• Built-in functions
• Diagnostics
• What runs where

// A chart study: price, volume, open, high, low and time are the chart's bars.
const len = 20;
metric("fast", ema(price, 9));
metric("slow", sma(price, len));
 

A first program#

uses { bars: "^1", paper: "^1" }

const close = bars.close("BTC-USD", "1h");
const fast = ema(close, 5);
const slow = ema(close, 20);

metric("fast", fast);
metric("slow", slow);

when (crossover(fast, slow)) { paper.buy({ symbol: "BTC-USD", quote: 100 }); }
when (crossunder(fast, slow)) { paper.sell({ symbol: "BTC-USD" }); }

• uses { … } names the modules the program calls, with a version range. A module is either a source namespace (bars) or a capability (paper).
• const binds a name to a value. Every binding is a series: close is the close of every hourly bar, and ema(close, 5) is a new series computed from it, one value per bar.
• metric(name, value) publishes a value: a replay returns it for every bar, and a live run returns it at each step.
• when (condition) { … } fires its actions when the condition turns true. crossover(fast, slow) is true at the bar fast crosses above slow.

Comments are // … to the end of the line, or /* … */.

The host#

The language has no built-in data and no built-in actions. The application compiling a program declares what it may read and do. That declaration is the host.

• Sources are the reads. Each has a namespace.name, a type (number, bool, text, a tape of events with typed fields, or a record of typed fields read one bar at a time, such as a position's size and pnl), whether it takes a literal key as its first argument (a symbol, an address), and which bar intervals it is published at. bars.close("BTC-USD", "1h") is a keyed number source read at 1h. A key can instead be a typed identity: a source declared with identity: 'pool' ('token', 'market', 'venue', 'wallet') takes pool("base:0x…") (wallet("main"), keyed by the wallet's name) and refuses a bare string or another kind of identity (market.identity), so a pool's bars cannot be read with a token's address. Optional args follow the key and the interval, each typed (duration, number, quantity, text): a TWAP's window, twap(pool("…"), "5m"). Reading a source the host does not declare is the stream.unknown error, with the nearest name it does declare; an interval it does not publish is refused too.
• Capabilities are the actions, declared as YAML manifests: the module's name and version, and each action with its typed arguments. A call that does not match a manifest (an unknown action, an argument it does not take, or one of the wrong kind, such as a number where it takes Text) is refused at compile time. A manifest does not say which arguments are required, so a missing one is the executor's to refuse. A manifest's streams are the module's reads: the first argument's type is the key's identity (Market, Token, Pool, Wallet, or Venue for either a pool or a market), the rest are optional arguments (Duration, Bps, Number, Text), and out is Series<number>, Series<bool>, Series<text> or a record, "Series<{ size: number }>".
• Markets are the names market("…") may take, when the application has any.
• Units are the numeric reads' scales, Host::with_stream_units(json), in the format of ftw's platform/stream-units.json. A row's identity (pool, token, market, venue, wallet) is what its read is keyed by, and makes the row a read the host declares: a number with the row's bar intervals, a tape when the row lists its events ({ "px": "number" }), otherwise a number taking the row's args ([{ "name": "window", "type": "duration" }]). A record row is not a call, and a read the host already declares keeps its declaration. A row naming any other identity is refused when the host is built. A bar's forms_from names the live mark its forming bar is decided on (when (…, intrabar: true)), and unformed: true marks a bar price the forming bar does not deliver, which an intrabar condition may not read. A tape's row may name a candle read, "candles": { "read": "dex.candles", "price": "price", "volume": "usd" }: bars of any period built from the tape (Host::with_candles declares one directly).

In TypeScript the host is a plain object (see npm/README.md):

const host = {
  sources: [{ name: 'bars.close', type: 'number', keyed: true, intervals: ['1h'] }],
  capabilities: [paperManifestYaml],
};

In Rust it is flowscript_core::Host, built with Host::empty(), .with_source(Source::number("bars.close").keyed().intervals(["1h"])) and .with_capability(yaml).

The examples' host#

Every example in this reference compiles against this host:

Two capabilities: paper with buy({ symbol: Text, quote: Number }) and sell({ symbol: Text }), and alerts with send(message: Text).

Values and types#

na is how the language says a value is not known yet: an ema(close, 20) is na for its first 19 bars rather than a made-up number. It propagates through arithmetic and comparisons, and a condition that is na is not true. A condition is true, false or na: close > open is na where either side is, ! of na is na, and && and || answer na only when the known side cannot decide (false && na is false, true || na is true, true && na is na). c ? a : b and if over an na condition are na, and a when does not fire on one. A chart and a running strategy read conditions the same way. isNa(x) tests for it and nz(x, 0) replaces it.

Scalars lift over series. close * 2, close > open and abs(change(close)) apply bar by bar.

Expressions#

• Arithmetic + - * / % on numbers, comparison < <= > >= == !=, logic && || ! on conditions. A condition is not a number: (a > b) * 2 is an error, and a > b ? 1 : 0 is the number.
• Comparisons do not chain. 1 < x < 3 is an error; write 1 < x && x < 3.
• A comparison with na is na at every bar, so x == na is an error; test with isNa(x).
• Conditional: cond ? a : b, or if (cond) a else b as an expression. Both arms must have the same type.
• History: close[1] is the previous bar's close, close[k] the value k bars ago (na before there are k bars).
• Blocks: { let x = …; return …; } where a function or lambda body is expected. let names cannot shadow.

uses { bars: "^1" }
const close = bars.close("BTC-USD", "1h");
const open = bars.open("BTC-USD", "1h");
const body = abs(close - open);
const bias = if (close > open) "up" else (close < open ? "down" : "flat");
metric("body", body);
metric("bias", bias);
metric("gap", (open - close[1]) / close[1]);

A const is bound once: a second const of the same name is an error, not a replacement.

There are no loops, no recursion and no mutation. That is deliberate: every program terminates at every bar with bounded memory, which is what makes it replayable.

Lexical structure and grammar#

Text. A program is UTF-8. Outside strings and comments it is ASCII: a non-ASCII character there is an error naming it, except Unicode spaces, which separate tokens as ASCII whitespace does. A leading byte-order mark is ignored. Comments are // … to the end of the line and /* … */, which does not nest.

Tokens.

Precedence, loosest first. Binary operators on one level associate left, except comparisons, which do not associate at all.

So -x[1] is -(x[1]), a || b && c is a || (b && c), and x - 1 - 2 is (x - 1) - 2.

Limits. An expression nests at most 128 levels as written: each parenthesis, operand, call and record is one, and so is each operator in a chain (a + b + c is two). Once its consts are substituted and its library calls inlined, one binding may lower to 160 levels and 1 MB. The largest program in the repository uses about a sixth of the depth and a fortieth of the size. A program past either limit is refused with syntax.depth or expr.size instead of exhausting the memory of whatever compiles it.

What a program keeps is bounded where it is written: one expression keeps at most 1,000,000 samples of state (a window's length, a history depth [n], a warm-up), and a window over a tape spans at most "7d", since it keeps every event in its span. Both are refused with expr.size. A text value holds at most 4,096 bytes; joining past that is na, so a fold that grows its text cannot grow without end. A NaN or infinity a source feeds is na.

Functions, lambdas and fold#

fn defines a function. Parameters and the result are typed; a Series<…> parameter takes a series, count a whole number. A call whose argument, or a body whose result, is a different kind (a number, a condition, text) from what the annotation names is refused. A parameter with no annotation takes anything.

uses { bars: "^1" }
fn spread(a: Series<number>, b: Series<number>): Series<number> {
  return (a - b) / b;
}
const close = bars.close("BTC-USD", "1h");
metric("stretch", spread(close, sma(close, 50)));

fold(series, seed, (acc, x) => next) is the one stateful loop: it steps once per bar, from seed, with the last result and the new sample. The seed may be a record, and the accumulator is typed by it. Every indicator in the prelude is written with it.

uses { bars: "^1" }
const close = bars.close("BTC-USD", "1h");
// Bars since the close last made a new high.
const since = fold(close, { high: 0, n: 0 },
  (st, x) => x > st.high ? { high: x, n: 0 } : { high: st.high, n: st.n + 1 });
metric("bars_since_high", since.n);

A lambda is (a, b) => expr or (a, b) => { let …; return …; }. It captures the names around it by value.

@warmup(n) after a const's expression, or before a fn, states a lookback the compiler cannot derive. It can only ask for more history, never less.

Metrics#

metric(name, value) publishes one value per bar, and each metric has its own name. A numeric or boolean metric comes back as numbers (true is 1). A metric that is na (still warming up) reads NaN, and a text metric comes back as words ("" while na).

Triggers#

A trigger is a condition and the actions to fire when it holds.

• when (cond) { … } fires on the rising edge: the bar the condition turns from false to true, once per crossing.
• while (cond) { … } fires at every bar the condition holds and that brought something new the condition reads.

uses { bars: "^1", alerts: "^1" }
const close = bars.close("BTC-USD", "1h");
const volume = bars.volume("BTC-USD", "1h");
when (close > rolling_high(close, 24)[1]) { alerts.send("new 24h high"); }
while (volume > 3 * sma(volume, 48)) { alerts.send("volume still heavy"); }

Combinators, all usable inside any condition:

uses { bars: "^1", alerts: "^1" }
const close = bars.close("BTC-USD", "1h");
when (sustained(close > sma(close, 20), 3) && cooldown(hours(6))) {
  alerts.send("three closes above the 20-bar average");
}
when (every("1w", at: "Mon 14:00") && close < sma(close, 200)) {
  alerts.send("weekly buy window");
}

every counts chain time from the Unix epoch, from the time a run is given with each bar (or the bar number when there is none). It fires on the first bar at or after each boundary, once per boundary.

Actions and handlers#

An action is a call to a capability: paper.buy({ symbol: "BTC-USD", quote: 100 }). Each action expression is an action site, named by where it is in the program (when1#1:paper.buy). A decision carries its site, and an outcome reported for it names the site back.

Handlers attach to an action and run when it ends: .onFill, .onRevert, .onAbort and .onStuck. A handler reads the outcome as fill.price, fill.amountOut and outcome.reason, and may fire actions of its own, guarded by if if it likes.

uses { bars: "^1", paper: "^1", alerts: "^1" }
const close = bars.close("BTC-USD", "1h");
when (crossover(close, ema(close, 20))) {
  paper.buy({ symbol: "BTC-USD", quote: 100 })
    .onFill {
      if (fill.price > close * 1.01) { paper.sell({ symbol: "BTC-USD" }); }
      alerts.send("bought at " + fill.price);
    }
    .onRevert { alerts.send("buy failed: " + outcome.reason); };
}

An if in a when or while body guards the actions inside it. It is read at the bar the when fires, so when (a) { if (b) { … } } acts the bar a turns true, if b holds then, and not later when b turns true while a still holds (that is when (a && b)). The other actions in the body fire either way.

uses { bars: "^1", paper: "^1", alerts: "^1" }
const close = bars.close("BTC-USD", "1h");
const volume = bars.volume("BTC-USD", "1h");
when (crossover(close, ema(close, 20))) {
  alerts.send("crossed up");
  if (volume > sma(volume, 24)) { paper.buy({ symbol: "BTC-USD", quote: 100 }); }
}

Who decides the outcome is the application's choice. With a paper broker, a replay or a run fills each order itself. An application with its own venue reports each outcome to the run (run.report in TypeScript, Run::report in Rust), and the handlers fire at the last bar stepped.

Enums and match#

enum declares a closed set of words. An unknown variant is a compile error with the nearest one, and match must cover every variant or end in _.

uses { bars: "^1" }
enum Regime { hot, quiet, normal }
const close = bars.close("BTC-USD", "1h");
const vol = stddev(change(ln(close)), 24);
const regime = vol > 0.02 ? Regime.hot : (vol < 0.005 ? Regime.quiet : Regime.normal);
metric("regime", regime);
metric("size", match (regime) { hot => 0, quiet => 1, _ => 0.5 });

Phase machines#

phases declares a state machine. Each phase holds the triggers that apply in it, and a when inside one names the phase it moves to. A phase's let bindings start over each time the phase is entered, so cummax(close) inside Long is the high since this entry, the entry bar's close included. A phase's entry { … } block fires when the machine enters it, right after the transition's own actions.

uses { bars: "^1", paper: "^1", alerts: "^1" }
const close = bars.close("BTC-USD", "1h");
phases Trade {
  phase Flat {
    when (crossover(close, ema(close, 20))) => Long {
      paper.buy({ symbol: "BTC-USD", quote: 100 });
    }
  }
  phase Long {
    entry { alerts.send("long"); }
    let stop = cummax(close) * 0.95;
    when (close < stop) => Flat { paper.sell({ symbol: "BTC-USD" }); }
  }
}
metric("phase", phase("Trade"));

A target that does not exist, a when in a phase with no target, and a phase nothing can reach are compile errors. Every trigger decides against the phases the bar began in, so a machine moves at most once a bar. An if inside a transition's body fires with the move, when its guard holds then; the machine moves either way. An if inside an entry block is a compile error (phases.guard): put the guard on the transition that enters the phase, or in a when inside it.

Tapes#

A tape is a source of events (trades, prints, messages) that arrive whenever they happen, not once a bar. map picks a field off each event, resample or window buckets the events, and a reducer (sum, mean, count, first, last, distinct) turns each bucket back into one number.

uses { trades: "^1" }
const prints = trades.prints("BTC-USD");
const buys = map(prints, (p) => p.side == "buy" ? p.size : 0);
metric("buy_volume_1h", sum(resample(buys, "1h")));
metric("prints_15m", count(window(prints, "15m")));

Using a tape where a bar series is expected is the cadence.events error.

The prelude#

The prelude is Flowscript source (prelude/standard.flow), and its doc comments are the definitions. In brief:

Built-in functions#

The compiler publishes its own list: flowscript_core::vocabulary() returns the built-ins, the prelude source and the diagnostic registry, which is what an editor or a code-writing model should read rather than a copy.

Diagnostics#

Every compile error has a stable code (FS0412), a slug (record.field), a message, and a position. A code never changes meaning and is never reused, so tooling matches on it rather than on the message. The registry is flowscript_core::diagnostic_codes().

uses { bars: "^1" }
const close = bars.clsoe("BTC-USD", "1h");

That is stream.unknown: "did you mean bars.close?".

What runs where#

flowscript-core and @flowscript/core compile any program in this reference, and replay or run all of it except the parts each says it does not model:

• Replay (replay, flowscript.replay) evaluates a whole history at once. With a paper broker it fills the orders and runs their handlers.
• Live run (live::Run, flowscript.run) steps one bar at a time and makes the replay's decisions on the same bars. Outcomes come from a paper broker or from the application (report).
• Not modelled here yet: maintain blocks and invariants are listed in the result's unmodelled with the reason, never silently dropped.

Some of the language is for a full execution host such as ftw's and needs data this library does not supply on its own: position context (inPosition(), avgEntry(), unrealizedPnl() …) reads the strategy's own execution stream, param(…) and config(…) read a configuration stream, and pool(…), token(…) and market(…) name an application's venues.