@loess/program

A circle's program accepts or rejects each transaction before a committer commits it, and a device runs the same program on its own writes before it shows them. This package runs programs as WebAssembly within three limits, holds the AssemblyScript library programs are written with, and ships the author program, which any app can use.

import { layerWasmPrograms, Programs } from '@loess/program';
 
const validate = yield * (yield * Programs).load(programHash, openProgramObject);
const validation = validate({ transaction, role: 'write', now: Date.now(), facts: database.facts });
// { status: 'accepted' } or { status: 'rejected', reason: 'only the member at the end of the id changes it' }

Within its limits, a program gives the same answer for the same transaction, facts, and time in every runtime. Each run gets a new instance and a new memory, so nothing carries over from one transaction to the next, and the program has no clock but now, no network, and no randomness.

Interface

A sealed program object holds the CBOR of a ProgramObject: the WebAssembly and its three limits. ProgramObject.make({ wasm }) fills in the defaults.

const object = encodeCbor(
  Schema.encodeSync(ProgramObject)(ProgramObject.make({ wasm: authorProgram.wasm })),
);
// { wasm, readLimit: 128, timeLimit: 50, memoryLimit: 16777216 }

layerWasmPrograms compiles a program the first time it sees its hash and keeps it. A Cloudflare Worker refuses to compile WebAssembly from bytes, so a committer in a Worker passes the modules it imported at build time, keyed by the hex SHA-256 of the program's WebAssembly (the hash beside each program). The runner opens and checks the program object as usual, and then uses the module from the map instead of compiling. A module in the map must import the runner's tick, so it has to be a .runner.wasm file. In a Worker without a match, load fails with ProgramError, so openCommitter fails and the circle's transactions wait in the inbox. The host answers 422 and doesn't try again until the next write to the circle.

import authorRunner from '@loess/program/programs/author.runner.wasm';
import { authorProgram } from '@loess/program/programs';
 
const layer = layerWasmPrograms({ precompiled: new Map([[authorProgram.hash, authorRunner]]) });

Testing a program

testProgram runs a program's WebAssembly in the runner a committer uses, on facts held in an array, oldest first. An app's tests list the transactions its program must accept and reject:

import { testProgram } from '@loess/program/test';
 
const validate = yield * testProgram({ wasm: authorProgram.wasm });
expect(validate({ transaction: bobsEditOfAnnsPost, role: 'write', now: 0 })).toEqual({
  status: 'rejected',
  reason: 'only the member at the end of the id changes it',
});

The ABI

A program is a WebAssembly module of version 1. It exports validateTransaction: () -> i32, which returns 0 to accept and 1 to reject. Before it returns 1, it calls reject with the reason. Any other result, or 1 without a reason, gives program failed.

It may import only env.memory and these functions of module loess. Text is UTF-8, given as a pointer and a length in bytes. The host holds the bytes of its last answer, and read copies them into the program's memory, so the program allocates the buffer itself.

The input is the CBOR map { transaction, role, now }: the transaction as protocol's Transaction schema encodes it, the writer's role (read, write, or own), and the time in milliseconds. A fact is the CBOR map { id, field, value, writer, committedAt }. A member id is text or bytes in CBOR, so a program compares writers by their CBOR bytes.

The limits

The limit counts facts, not pages, because the runner sees only the facts Facts returns. A fact costs at most one page, and often less, because a page holds many rows, so 128 facts read at most 128 pages. The database can still fail a read on its own page limit.

A JavaScript runtime can't interrupt a synchronous WebAssembly call, so the runner adds a call to an imported function, loess-runner.tick, at the start of every function and of every loop iteration before it compiles the program. The tick reads the clock every 1,024 ticks and stops the run once it passes the deadline. A Cloudflare Worker doesn't move its clock during a synchronous run, so the tick also counts: a run stops after 100,000 ticks per millisecond of the limit, 5 million for 50 ms. A tick of a tight loop took about 9 ns in Node 24 on a cloud build machine, so the count stops such a loop after about 45 ms there. The host functions read the clock too.

A slow phone stops a program by its clock after 50 ms, and a fast committer may stop it by the count first, so when a program runs near its limit, the two can give different answers. The design allows this difference. The count is the same in every runtime, so two fast machines give the same answer.

A crash, a stack overflow, a passed limit, or bytes a host function can't read all give { status: 'rejected', reason: 'program failed' }. An error that Facts throws, such as a database that lacks a page, is thrown again unchanged, so the caller can fetch the page and run the program again.

The runner refuses a module with ProgramError when it defines its own memory, imports anything else, uses more than one memory or an instruction it doesn't decode (SIMD, threads, exceptions, and tail calls stay out), or lacks validateTransaction. Custom sections are dropped.

Each run calls new WebAssembly.Instance, because ValidateTransaction is synchronous. Older Chrome refused that on a page's main thread for a module over 4 KiB. Headless Chromium 153 ran the author program, about 6 KiB, on the main thread, so the sdk runs programs there.

Building a program

Programs are written in AssemblyScript, in assembly/programs/. The library in assembly/ reads the input and calls the host:

import { accept, first, readInput, rejectWith, sameMember } from '../program';
 
export function validateTransaction(): i32 {
  const transaction = readInput().transaction;
  for (let index = 0; index < transaction.changes.length; index++) {
    const author = first(transaction.changes[index].id);
    if (author != null && !sameMember(author.writer, transaction.writer)) {
      return rejectWith('only the author edits it');
    }
  }
  return accept();
}

The author program

The author program lets only an id's author change its facts, and reads no facts, so a device checks a write with no pages. The author is the member whose id comes after the last ~ of the fact's id, as in post:1~{member id}. An identity's member id is its circle id, 64 lowercase hex digits. A key member's id is its public key, the CBOR of its algorithm and key, as lowercase hex, so it is at least 68 digits long.

The rule covers creating, changing, and retracting, so Bob can't create post:2~{Ann's id} either. The program reads the text after an id's last ~ as a member id if it holds 64 or more lowercase hex digits in pairs, so a tag may hold another member's id, as a like's tag holds its post's id. An id without a ~ has no author. Don't end an id with ~ and a hash in lowercase hex: no writer's id matches it, and the program rejects every change to it. Every other id takes every change the writer's role allows. The committer checks the role before it runs the program.

The program has no rule of any one app, such as chat's limit of 4,000 characters for a message. If your app needs one, check it on the device before you write; no committer enforces it.

pnpm programs compiles each file in assembly/programs/ and writes programs/{name}.wasm, programs/{name}.runner.wasm, and src/programs/{name}Program.ts. A test compiles them again and fails when the committed bytes differ, so run it after any change to assembly/. The compiler options are in src/build/compileAssemblyScript.ts: the stub runtime, which never frees memory and is enough for one transaction; memory imported as env.memory; and a trap in place of env.abort.