Memory as L4, NVMe as L5, object storage as L6, archive as L7. Move the sliders to see what a miss costs the single-threaded executor, how much of the hot set fits, what each object-store layout costs to write, keep, and restore, and what the database costs per month against all-memory.
–
Throughput, abort-or-block
–
Revert or fork, time to serve
–
Cheaper than all-memory
Executor under misses
A miss either stalls the executor for the fetch latency or aborts the transaction and repeats its work so far. The rule: abort when work so far is less than the fetch latency and other work is queued.
UsefulRepeated after abortStalled on fetchIdle
Where accesses are served
Hot rows are packed into memory first, then NVMe, then object storage. Page tiering pays for every cold row that shares a page with a hot one; row tiering (anti-caching) does not.
MemoryNVMeObject storage
Object storage layouts
Seven ways to keep snapshots in object storage, priced per snapshot interval; stored and upload are billed bytes (pages travel as compressed frames). The proposal picks base chunks plus delta packs, consolidated every snapshots.
Snapshot interval
The trigger constant sets a trade: short intervals upload nearly a page per updated row, long intervals replay more log per gap. Both curves are for the chosen layout at the current write rate; the marker is the current interval.
Upload rate to object storageWorst-case replay per gap
Retention schedule
What is kept at each age under the revert SLA schedule, at the current snapshot interval and horizon. Thinned snapshots are taken at consolidation boundaries so they pin base chunks only.
Revert SLA
Time to serve a revert to a target of a given age, by database size, against the SLA targets. Each cell is index rebuild or manifest read, plus fetching and replaying the archived commitlog across the gap to the nearest older kept snapshot, plus L7 restore past the archive age. Write density is held constant across sizes, capped at the executor's throughput.
Monthly cost
Memory for the working set, NVMe for the warm set, one object copy of the live data with its requests, against holding the whole database in memory. Retained history is shown separately: the all-memory option has none.
MemoryNVMeObject storage and requests
Model assumptions
Hit-path critical section, prices, and tier latencies match the public calculator on spacetimedb.com: $5, $0.08, and $0.023 per GB-month; 100 ns, 100 µs, 50 ms. Requests: $5 per million PUTs, $0.40 per million GETs.
A fraction of accesses goes to a hot set of rows; the rest are uniform. Page tiering's hot footprint is the hot rows divided by the hot page fill (clustered) or every page containing a hot row (scattered).
Misses per transaction are binomial over the pages it touches. Abort waste places misses uniformly over the transaction; the i-th of j misses sits at the i-th order statistic. Mean latency uses the Pollaczek-Khinchine formula at the offered load, capped just below capacity.
Throughput is the minimum of the executor bound, the NVMe fetch rate divided by NVMe misses per transaction, object-storage concurrency over its latency divided by object misses, and parked slots over the parked time per transaction.
Dirty pages per interval: N_h(1 − e^(−k_h/N_h)) + N_c(1 − e^(−k_c/N_c)) for the hot and cold updates in the interval. Chunks touched: M(1 − e^(−d/M)).
Restore time is the larger of requests over concurrency times first-byte latency and bytes over aggregate bandwidth (64 connections at 100 MB/s).
The archived commitlog (one segment per interval, row bytes plus 40 bytes of record overhead per update, compressed like pages) is kept for the whole horizon and priced in the history line; older segments move to L7. A revert fetches the segments of its gap (one request each, bytes over aggregate bandwidth) before replaying.
Revenue uses the published pricing (2.592e15 eV per dollar): storage $1.00/GB-month on live logical bytes, writes $38.58/GB, scans $7.72/GB, seeks $0.154/M, instructions $0.193/G; client bandwidth ($77.16/GB sent) is not modeled, and the pricing page has no line item for retained history.
The revert SLA table evaluates the chosen layout only; replay time is gap × write rate ÷ replay rate; the write rate is scaled with database size so write density is constant, capped at the hit-path throughput of one executor. L7 restore applies when the kept snapshot is older than the archive age.
Retention follows the revert SLA schedule (every snapshot for a day, 50% for a week, 25% for a month, 12.5% to the horizon; older than the archive age in L7). Stored bytes per layout count the distinct versions each retained manifest pins: for a unit rewritten with probability q per interval, 1 + Σ over gaps g between retained manifests of (1 − (1 − q)^g).
Delta packs stay alive while any page in them is still current or while a retained manifest references them; restoring fetches a pack whole when at least half of it is live, otherwise its live pages by range.
Compression applies a fixed zstd ratio to page frames in packs, chunks, and files; version packs use a separate delta ratio.
Time to serve after revert or fork: in-memory indexes require reading every row (equal to the full copy time); paged indexes require only the manifest (chunks times 40 bytes). Fully local is the time to copy every chunk and pack, which only disk and memory tables need.
COW heap: dirty pages per interval are the appended pages plus compaction rewrites of (1 − g)/g live bytes per byte reclaimed; pages carry on average g/2 dead space. The in-place closed form N(1 − e^(−k/N)) was checked against storage_sim's heap_index store to within 1% at k/N from 0.01 to 4; cow_heap_index matched k·r/p to within one page.