Benchmarks

Timing measurements for sct commands run against a real SNOMED CT release:

• UK Monolith - SnomedCT_MonolithRF2_PRODUCTION_20260701T120000Z (837,930 active concepts)

Last verified: 2026-07-28, against sct 0.20.1. If you're reading this much later than that date, treat the numbers as a rough shape rather than gospel - re-run How to benchmark yourself below for your own hardware and release.

Benchmark machines

Methodology

Each command was timed as a single run (not averaged) with peak RSS captured via /usr/bin/time -v (GNU time getrusage) on the Raspberry Pi and /proc/PID/status VmHWM polling on the Lenovo (which lacks GNU time). Wall time is wall-clock seconds. Both methods report the maximum resident set size of the process in GiB.

On the Lenovo, the source ZIP was on NVMe with a warm page cache. On the Raspberry Pi, the source ZIP was on the microSD card. Both machines processed the same uk_sct2mo_42.3.0_20260701000001Z.zip archive with default settings (active concepts only, simple refsets). The sct ndjson stage extracts the ZIP to a temporary directory before processing.

This is a single documented run per machine, not an average over many iterations - treat the numbers as a real, reproducible order of magnitude rather than a precise statistical claim. Wall-clock time is sensitive to whatever else is running at the time.

FHIR terminology server timings should be treated differently from command timings. Run the FHIR conformance harness first, then benchmark only servers that pass the relevant profile:

benchmarks/conformance.sh --server http://localhost:8080/fhir
benchmarks/bench.sh --db snomed.db --server http://localhost:8080/fhir --runs 20 --warmup 5

See FHIR Conformance And Benchmarks for the full methodology.

Results - UK Monolith Edition (837,930 concepts)

Pipeline timings and peak memory

Fresh verification on the Lenovo produced a 1,972,310,016-byte SQLite database before TCT and a 2,799,837,184-byte database afterwards. The human-readable values above use the same 1024-based formatter as sct info; ls -lh rounds the same files to approximately 1.9 GiB and 2.7 GiB.

Android phone (OnePlus 13, Termux)

sct runs under Termux on Android, so the whole pipeline was run on a phone. This is kept separate from the table above rather than added as two more columns, because it is not a like-for-like comparison: peak RSS was not captured, and the binary was built from source under Termux (linking against Bionic) rather than being the released linux-aarch64 musl build the other machines effectively represent. Single run, wall-clock only, thermal state not controlled.

A 2026 flagship phone builds a national SNOMED CT edition end to end in about 4.5 minutes, and matches a 22-core laptop on sct fst build.

The interesting part is that it does not behave like uniformly slower hardware. On sct fst build, sct tct, and sct sqlite it lands much closer to the laptop than to the Pi. On sct ndjson and sct parquet it drops back to Pi-5 pace, despite far faster silicon and UFS 4.0 storage where the Pi has a microSD card.

Those two stages are the allocation-heavy ones - sct ndjson builds millions of short-lived strings and records, and sct parquet accumulates Arrow arrays - which suggests the ceiling is allocator throughput rather than CPU or I/O. Bionic's hardened scudo allocator is a reasonable suspect against glibc on the other two machines. Supporting evidence: the sct parquet progress bar reported ~108 MiB/s reading the NDJSON, far below what UFS 4.0 delivers, so the stage is not storage-bound.

This is a hypothesis, not a measured result. Testing it properly means running the same stage on the same handset under a glibc userland (proot-distro Debian) and comparing, plus capturing peak RSS. If you have a phone and ten minutes, that data would be welcome.

Raspberry Pi 5: from OOM crash to completion

Before the v0.20.1 streaming optimisation, sct ndjson crashed on the Raspberry Pi 5 (8 GB RAM) when building the UK Monolith, almost certainly due to out-of-memory conditions. The old implementation materialised every RF2 file as a whole-file Vec<Row> before aggregation, then built the complete Vec<ConceptRecord> before writing a single byte - so on a national edition the loaded dataset, multi-gigabyte transient row vectors, and the full output record set could all be resident at once.

The v0.20.1 streaming implementation streams rows directly into the dataset maps and writes records to the output file as they are built. The Raspberry Pi 5 now completes the full pipeline (ndjson, sqlite, parquet, tct, fst) successfully within its 8 GB RAM, with 3.69 GiB peak RSS during the most memory-intensive stage.

sct ndjson memory improvement in v0.20.1

The v0.20.1 streaming implementation was compared with its immediate pre-change parent (b2ca527) on 2026-07-27. Both were optimised release builds processing the same uk_sct2mo_42.3.0_20260701000001Z.zip archive with the default active-concept and simple-refset settings, a disk-backed temporary directory, and regular file output. Peak RSS is Linux getrusage maximum resident set size as reported by zsh %M; wall time is one paired run rather than a statistical benchmark.

Both runs emitted 837,930 concepts in 1,346,971,886-byte artefacts with byte-identical concept records and the same content fingerprint (sha256:abc9de055e67073b56cc21c01b95762c60cb138f839cbf2bdff5894b4a84500e). The lower peak leaves substantially more headroom on 8 GB machines and avoids the severe swap pressure that can make RF2 conversion appear to stall on slower storage.

Only the UK Monolith is benchmarked currently. The previous version of this page also carried UK Clinical Edition numbers; they've been dropped rather than left stale, since re-running them needs a fresh TRUD-authenticated download this environment didn't have to hand. Re-add if useful - Clinical is ~24x smaller and everything scales down accordingly.

MCP server startup time

The sct mcp server should start fast enough to avoid a perceptible delay when an MCP client opens it. Benchmark the current stateless server/discover lifecycle with a complete 2026-07-28 metadata object:

time printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"server/discover","params":{"_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientCapabilities":{}}}}' \
  | sct mcp --db snomed.db --codelist-root ./codelists >/dev/null

The recorded figures below are the pre-rmcp v0.18.2 baseline using the legacy initialization path. They remain useful evidence that database size does not force terminology loading at startup, but the SDK-backed 2026-07-28 path must be remeasured before publishing a current comparison.

The server opens the SQLite file without reading it into memory and reads provenance from a small keyed table. Current discovery responses advertise supported versions, capabilities, cache hints, and server identity; release provenance is carried in result _meta:

{"jsonrpc":"2.0","id":1,"result":{"resultType":"complete","supportedVersions":["2024-11-05","2025-03-26","2025-06-18","2025-11-25","2026-07-28"],"capabilities":{"tools":{}},"ttlMs":60000,"cacheScope":"public","_meta":{"org.sct/provenance":{"release_id":"...","release_date":"..."},"io.modelcontextprotocol/serverInfo":{"name":"sct-mcp","version":"..."}}}}

Note on an earlier regression: a prior release briefly took ~370 ms to start against a full Monolith database, because its startup schema-version check ran SELECT MAX(schema_version) FROM concepts - a full-table scan of an unindexed column. Reading a single row instead (the value is uniform across concepts) restored the few-millisecond startup shown above, on databases of any size. See issue #32.

How to benchmark yourself

sct ndjson

--rf2 accepts either an RF2 directory or a .zip file directly:

# Using a zip file
time sct ndjson --rf2 ~/downloads/SnomedCT_MonolithRF2_PRODUCTION_20260701T120000Z.zip

# Using a pre-extracted directory (warm the page cache first for a fair comparison)
find ~/downloads/SnomedCT_MonolithRF2_PRODUCTION_20260701T120000Z -type f -exec cat {} + > /dev/null 2>&1
time sct ndjson --rf2 ~/downloads/SnomedCT_MonolithRF2_PRODUCTION_20260701T120000Z/

sct sqlite

time sct sqlite --ndjson snomedct-monolithrf2-production-20260701t120000z.ndjson --output snomed.db
ls -lh snomed.db

Verify FTS works:

sqlite3 snomed.db "SELECT id, preferred_term FROM concepts_fts WHERE concepts_fts MATCH 'heart attack' LIMIT 5"

sct parquet

time sct parquet --ndjson snomedct-monolithrf2-production-20260701t120000z.ndjson --output snomed.parquet
ls -lh snomed.parquet

Verify DuckDB can read it:

duckdb -c "SELECT hierarchy, COUNT(*) n FROM 'snomed.parquet' GROUP BY hierarchy ORDER BY n DESC LIMIT 5"

sct markdown

time sct markdown --ndjson snomedct-monolithrf2-production-20260701t120000z.ndjson --output snomed-concepts/
du -sh snomed-concepts/
find snomed-concepts/ -name "*.md" | wc -l

sct tct

Builds the transitive closure table (concept_ancestors) over an existing SQLite database - needed for subsumption-heavy workloads or the SCT-QL compiler, not built by default:

time sct tct --db snomed.db
ls -lh snomed.db
sqlite3 snomed.db "SELECT COUNT(*) FROM concept_ancestors"

sct fst build

time sct fst build --ndjson snomedct-monolithrf2-production-20260701t120000z.ndjson --output snomed.fst
ls -lh snomed.fst

Verify search works:

sct fst search "myocardial infarction" --index snomed.fst