Goals

Loess is a data substrate for personal apps: your photos, notes, journal, recipes, tasks, or mail, with parts of it shared with the people and agents you choose. It syncs across your devices, works offline, and is end-to-end encrypted, so storage can't read it. Many apps use the same data, and you pick where it is stored, so it never gets stuck in one app or platform.

We build it for small places on the web where people know each other: a family, a group of friends, a team. The people using the apps aren't technical, so encryption adds no step they have to understand. If you write an app, you need no backend and write no crypto code.

Storage can't read a circle, so it can't enforce rules about the circle's data. Each circle's owners choose a committer instead, which reads the circle and enforces its rules, so neither the members nor the owners check anything in the background. In exchange, the owners trust one more reader.

If you find a simpler way to keep everything below, use it. Architecture says what the parts are, and Flows says what a person sees.

An example

Ann posts a photo from the lake, and Ben likes it. That is five facts in Ann's circle:

Apps agree on field names, not on a schema. A map app can add place/rating to place:lake-tahoe without asking the photo app, and "show me my trip" is one query per circle, merged on the device:

SELECT id FROM stored_facts
WHERE field = 'post/place' AND value = 'place:lake-tahoe' AND current

Ann wrote the post in one transaction and Ben wrote the like in another. The program of Ann's circle lets only a post's author change it, and her phone is the circle's committer: it runs the program on Ben's transaction, accepts it, and commits it. Until her phone does that, Ben's like is pending.

Terms

Priorities

When two of these conflict, the higher one comes first. The first two come first because a leak or a lost write can't be undone.

1. Only a circle's members and the committers its owners chose can read it, and that excludes storage. Nobody but a committer the owners chose can change a write or rewrite the circle's history.
2. A write is never lost silently, even through a closed tab, a reload, a dropped response, or a crash.
3. Actions feel instant, because they happen on the device first. An app opens and shows or lists every action with no wait on storage, offline too, and other devices see a change within about a second. Each flow costs a fixed, small number of round trips, however much data the circle holds.
4. The store stays generic. If one app needs something, we build it for every app or leave it in that app.
5. One person can understand all of Loess: you can read the goals, principles, and architecture in one sitting, each part is simple and easy to delete, and we delete a feature that doesn't justify its complexity.
6. Anything we run runs behind an interface, so it can be swapped for another implementation.
7. The reference storage costs about $5 a month for a small group.

Constraints

The goals above leave many designs open. These narrow them before a design starts, because we chose each on purpose:

• A device downloads only the data a view needs, and never a whole circle. The requests and bytes of opening a view grow with the view and not with the circle.
• A device does no replay and no background work. It opens data a committer already checked and built, so a phone that was off for a week shows the newest data without processing the writes it missed, and nothing runs while the app is closed. An open app still sends its own unsent writes, and commits if its device is a committer.
• A device keeps its data in SQLite, in a browser too, and an app queries it with SQL.
• Storage has more than one backend behind one interface, and Cloudflare is the cheap, easy default.

Who you trust

You should be able to put your data on a platform you don't trust. Encryption keeps it from looking at your photos, and signatures keep it from changing them unnoticed.

Three limits apply today:

• Pick a committer you trust to read the circle and enforce its rules. Every device shows what a committer committed, so a dishonest one could commit a write the program rejects, change a write, or write as a member. Storage keeps every signed transaction as the evidence of what each member wrote. Where every member commits, as in a household's tasks, the program only stops mistakes.
• Storage can withhold new writes from a device, which can't tell that from no new writes, but storage can't move the device back to an older copy.
• Storage sees patterns: circle ids, sizes, times, which committer commits, and which connection writes to which circles. Anyone who knows a circle's id can also download its sealed objects from the reference storage, which shows them the same sizes and times and nothing they can open. We hide the patterns that cost little to hide, and the architecture lists the ones storage still sees. The reference storage doesn't mine, sell, or report any of it to a third party.

An owner can remove a member or a committer who misbehaves, but they keep whatever they already read.

Promises

Each promise has a test named in its words, and Flows lists the tests for each journey.

Nothing is forged

• Every write and every commit is signed, and storage, a committer, and a device each check the signatures on what they accept or show.
• A device accepts only what it can verify with keys it already has, and never goes back to an older state.
• Storage can't serve one piece of sealed data where another belongs, and the size of sealed data says little about what it holds.
• Storage runs no app code. It refuses any write a role isn't allowed to make, and a committer refuses any transaction the program rejects, so a device never throws out a write that both accepted. A rule such as "only the author edits a post" goes in the program. A device never ignores a write that storage and a committer accepted, and no design may rely on one that does.
• A program decides only about the fields it knows and accepts a change to any other, so you can add a second app's facts to a circle without changing the first app's program.
• A committer commits a transaction only if the program accepts it, and commits its changes as the writer wrote them. What a committer accepted and what it rejected are on record, each rejection with its reason, so what a committer did can be checked.
• A signed or sealed format is frozen and versioned, because only a member's device can rewrite it. Anything else can change with a deploy. Until v0.1 no format is frozen, and a format change wipes the preview data.

No write is lost

• A transaction shows as pending on the writer's devices at once, and on the circle's other devices from the relay, until a commit settles it.
• A device shows a pending transaction only after the program accepts it there. If the device doesn't hold the data the program needs, the app lists its own write as unchecked until that data arrives.
• Any flow can be cut off between two steps and run again.
• Data never references a missing blob, such as a photo that was never uploaded.
• We keep history, and people can still retract facts and delete old data. No default setting deletes anything, and a fact written today can still be read and verified in ten years.

Everyone sees the same data

• Each circle has one order of writes. Storage accepts writes in the order it receives them, and commits keep that order, though a committer rejects some. No clock decides the order, so every device expects the same result for a pending write, and two devices that got the same commits show the same data. We chose one order over consensus, which would make the software much more complicated.
• With a plain change, the last write in that order is kept. A change that says which value it replaces can conflict, and the app gets an error that says what conflicted and with which value. If you need more, such as text two people edit together, build it in your app.
• A fact's writer signed its transaction, and its time is when the committer committed it, by the committer's clock. You can trust both as far as you trust the committer.
• A view across people is a query. A feed of every friend's posts runs one query per circle on the device and merges them, and no data is uploaded again.

Your identity and your data are yours

• No company gives you your identity, and no one can take it from you. It comes from your recovery phrase alone, and only you change it. An identity has no username or profile, so other people learn about you only from what you share.
• Nobody ever sees a key, and losing a device doesn't lose data. A passkey signs you in on a new device. Sign-up also shows a recovery phrase once, and the phrase brings the data back after every device and passkey is lost.
• A new person joins from an invite link with no other account. Their device makes an identity and joins in the same visit.
• A removed member or device reads nothing written after the circle's next key change, which happens as soon as one of the circle's committers runs.
• You can always get a full copy of your data.

Any storage, same apps

You might use a big provider, share storage with family, or run your own, and use the same apps either way. You pick storage with one setting, so your data doesn't have to sit with the app's developer.

Friends on different storage share circles, because a device talks to whichever storage each circle lives on. Trust comes from signatures and keys, and every storage accepts at least the same sizes, so each app works the same everywhere, with the same identity. Each circle lives in one storage, and you should be able to leave that storage with your data and your identity.

Always sealed

Every circle is sealed, and there's no plain mode. Supporting both would make storage know the members and check them on every read, which adds a second authority and a second read path, and traditional data stores already cover the unsealed case.

Anything that needs to read the data, like a search indexer or a committing server, is a trusted reader: one more member or committer, which an owner adds on purpose. To make data public, you publish a copy of it, so nobody reads your archive. To let one more person or service read, you share a read link, or add them as a member who can only read.

What the SDK provides

Example apps

These are examples of the apps Loess aims to make easy to build, with the views each one needs to be fast. They aren't the only apps it supports.

Comparisons has bench results for Loess, Jazz 1, and Jazz 2.

A circle's owners choose who commits it, and an app suggests a default:

A committer is separate from storage, and from the app's author, who runs neither unless you choose the author for it. One host often does both jobs, but they are separate choices, and an owner changes a circle's committers in one step.

This repo carries two example apps, moments (a photo feed among friends) and chat (channels), and Flows walks through their journeys.

These fit poorly:

• Guarantees that must hold while no committer you trust is online, because storage can't read the data to enforce them.
• Strangers at scale: public discovery, virality, and follower graphs with millions of members.
• Big data: a circle holds one person's data or a small group's, and the default limits suit a few hundred members, because removing a member costs more as a circle grows.