SynapCores v1.18.0 — Parquet analytics that outrun DuckDB

Analytical queries that outrun DuckDB, lookups that stay fast as your data grows, and transactions you can rely on.

This release is about speed you can feel and behaviour you can trust. Queries over Parquet in object storage are now faster than DuckDB on most shapes, point lookups no longer slow down as your database fills up, and ROLLBACK, BACKUP and international text all do exactly what you expect.

Upgrading is a one-line change. No schema migration, no config change, no client changes.

Faster analytics over your data lake

SynapCores reads Parquet directly from S3-compatible storage. In v1.18.0-ce that path is substantially faster — and in most cases faster than DuckDB reading the same files.

Measured against DuckDB on identical Parquet objects in the same bucket, with every answer compared row by row to confirm the results match:

Faster than DuckDB on 9 of 12 shapes, at a median of 0.63× its time.

What this means day to day: dashboards that aggregate lake data return in milliseconds instead of seconds, and exploratory joins are fast enough to work with interactively rather than waiting on.

Smarter query planning

Several improvements compound here, and they apply automatically — no query rewriting, no hints, no new syntax:

• Filters now reach the Parquet reader, so SynapCores skips row groups and pages that cannot match instead of decoding them. Bloom filters and page-index selection are now used on lake scans.
• Filters travel across joins. If you filter one side of a join on the join key, the other side is filtered too — so a lookup that used to scan a whole table now reads almost nothing.
• IN (…) lists are now pushed down like any other filter. On a data-returning join this took one query from 187 ms to 11 ms.
• LIMIT stops work early. A preview query no longer builds tens of thousands of rows to return ten.
• Aggregates read only the columns they need, instead of decoding every column in the table.

Leaner results over the wire

Query responses are assembled far more efficiently. For a 189,000-row result the server-side cost dropped from 116 ms to 61 ms, and a 173,000-row join from 92 ms to 56 ms — with byte-identical output, so nothing in your client changes.

Lookups stay fast as your data grows

Previously, a lookup that found nothing could slow down as unrelated tables grew, because the engine searched more broadly than it needed to. That is fixed: lookups are now addressed directly.

On our own production deployment — five reference tables, ~560,000 rows — the difference is immediate:

A miss now costs about the same as a hit, and neither grows as you add data elsewhere. If your application looks up keys that often don't exist — address validation, enrichment, cache-miss paths — this is the headline change.

Table scans saw the same treatment and are up to 80× faster on small tables in a large database.

Transactions you can rely on

ROLLBACK now reverses the writes made inside a transaction. Previously it reported success without undoing anything.

BEGIN;
INSERT INTO orders (id, total) VALUES (1001, 250.00);
ROLLBACK;        -- the row is now correctly gone

If you have been using explicit transactions, this is worth upgrading for on its own.

Backups that produce a real artifact

BACKUP now uses a consistent RocksDB checkpoint — hard-linked, no downtime, no pause in serving — and verifies it produced a usable result before reporting success.

International text handled safely

Text containing accented or non-Latin characters — Montréal, Québec, 日本語 — is now handled correctly everywhere, including inside string literals in your SQL. Previously certain character positions could interrupt query processing.

If you store names, addresses or any user-entered text in languages other than English, upgrade.

Other fixes in this release

• DEFAULT values are now applied when you add a column with ALTER TABLE.
• Concurrent updates to the same row no longer race.
• Primary-key lookups no longer return data from a different table in rare cases.
• DROP DATABASE no longer leaves tables that reappear on re-create.
• Secondary indexes stay correct through UPDATE and DELETE.
• Exact DECIMAL arithmetic is preserved end to end.
• One REST endpoint was moved to its documented path.

Upgrading

docker pull synapcores/community:v1.18.0-ce

Or download for Linux (x86_64 / ARM64, including Ubuntu 24.04 builds) and macOS (Apple Silicon) from the release page.

• No schema migration
• No configuration change
• No client or wire-format change — responses are byte-identical to v1.17.0-ce

Validation

Every release is gated on a state-asserting feature validator (245 assertions, all green), a full recipe certification sweep, a 36-case performance guard covering both row storage and lake queries in both directions, and a runtime check on non-AVX-512 hardware. This release added 140 tests, each asserting the new code path returns exactly what the previous one did.

A note on ACID: ROLLBACK is reliable in this release for explicit, in-session use. Full ACID guarantees — isolation between concurrent connections, and atomicity across a process restart — land in v1.19.0-ce, together with FOREIGN KEY enforcement and savepoints.