SynapCores v1.16.0 — your MySQL history as open Parquet on S3, and money that adds up

What you can do now

Move a MySQL table into your own S3 bucket, on a schedule

Point SynapCores at a MySQL database and an object-storage bucket. It copies the history out as Parquet — the open format every analytics tool reads — and keeps doing it on a schedule.

POST /v1/lake/destinations
{
  "name": "warehouse", "kind": "s3",
  "bucket": "my-lake", "prefix": "raw/",
  "region": "us-east-1",
  "credentials": { "kind": "static",
                   "access_key_id": "...", "secret_access_key": "..." }
}

Then describe the export. You choose what leaves the source — drop a column outright, or replace one with a salted one-way hash, and the original never reaches the bucket:

POST /v1/lake/jobs
{
  "name": "nightly-pings",
  "source_connection_id": "...", "destination_id": "...",
  "tables": [{
    "source_table": "pings",
    "mode": { "mode": "incremental",
              "partition_column": "created_at", "granularity": "day" },
    "key_column": "id",
    "transforms": [ { "transform": "drop", "column": "email" },
                    { "transform": "hash", "column": "phone",
                      "salt_env": "LAKE_SALT" } ]
  }],
  "lookback_days": 7, "enabled": true
}

The reason this exists is memory. The extractor this replaces read a 394 GB table with pandas, exhausted 16 GB of RAM and was killed by the kernel. Here every stage is bounded — an unbuffered cursor, one Parquet row group resident at a time, multipart parts that ship and are freed. Exporting 240,000 rows peaked at 99 MB, and that ceiling does not move as the table grows.

Exports also run in a child process. A runaway export cannot take your database down with it, and an out-of-memory kill is reported as one rather than as success.

A partition only counts once its rows and key range are reconciled against the source, and the manifest is written last. Object stores have no rename, so write order is the atomicity: a partition without a verified manifest is invisible to every reader, ours or anyone else's. Re-running replaces a partition instead of appending to it.

Query what you exported — and Parquet other tools wrote

CREATE EXTERNAL TABLE pings STORED AS PARQUET
  LOCATION 's3://my-lake/raw/pings/'
  PARTITIONED BY (dt DATE)
  WITH (CONNECTION = 'warehouse');

SELECT dt, COUNT(*) FROM pings WHERE dt = DATE '2026-09-15' GROUP BY dt;

It behaves like any other table: partitions are pruned from their dt= directory names before an object is opened, row groups are pruned from footer statistics, and only the columns you asked for are fetched. You can join it straight to a live table.

This is not limited to files SynapCores wrote. Point it at Parquet produced by DuckDB, Spark or pandas and query it in place — no import, no copy. Nested data comes through intact, so LIST, STRUCT and MAP columns keep the shape they were written in.

Writes are refused, deliberately: an external table is a view onto files you own.

Serve model predictions over HTTP

Train a model, then call it from your application and get an answer back in milliseconds — no SQL round trip, no separate serving stack:

POST /v2/predict
{ "model": "churn", "rows": [ { "tenure": 14, "monthly": 79.9 } ] }

Eight model families are supported: linear, tree, random forest, gradient boosting, k-nearest neighbours, naive Bayes, SVM and neural. Every one was checked by asking the REST endpoint and the database the same question with the same model and requiring the same answer back, against stored known-good values.

Money that adds up

DECIMAL is now an exact type — it stores the digits, not a binary approximation:

CREATE TABLE invoices (id INT PRIMARY KEY, customer TEXT, amount DECIMAL(12,2));
INSERT INTO invoices VALUES (1,'Acme',19.99),(2,'Acme',0.10),(3,'Globex',0.20);

SELECT SUM(amount) FROM invoices;   -- 20.29, not 20.290000000000003
SELECT amount FROM invoices WHERE id = 2;   -- 0.10

The scale you declare is the scale you get: a DECIMAL(12,2) column stores and reports 0.10, and a value with more fractional digits than the column declares is rounded half away from zero, as MySQL does. 1.1 and 1.10 are one value — they compare equal and form one group, so a money amount can't split across two rows of a GROUP BY.

A million-row SUM matches what MySQL returns for the same data, and when the column is exported it becomes a real Parquet DECIMAL that other tools read as one.

Over REST, MCP, the CLI and exports a decimal is rendered as exact text rather than a JSON number — most JSON readers parse a number into a float and would undo the exactness on the way out.

Connect a BI tool

Metabase, or anything speaking the MySQL wire protocol, can now connect, list your tables and chart them — including data sitting in Parquet on S3. SHOW GRANTS FOR CURRENT_USER() answers truthfully, so a read-only key is offered read-only.

MySQL spellings that BI tools emit are accepted, including CAST(x AS SIGNED) and CAST(x AS UNSIGNED INTEGER).

Faster

Measured on one machine against DuckDB reading the same Parquet in the same object store:

Writing: about 3× DuckDB's throughput on append-only tables at a matched compression setting, in roughly 70% of the space.
Reading: six of eight query shapes match or beat DuckDB, the set as a whole at about 0.8× its time.
Two shapes are still slower — GROUP BY at roughly 2× DuckDB's time and COUNT(DISTINCT) at about 1.6×. Both are known work. If your queries lean on those, DuckDB wins today.
Ordinary SQL on regular tables got faster too: every query shape we track is the same or quicker than v1.15.0-ce, and one join improved 17%.

The AI query explainer now ships off by default. A single mistyped query used to start a language model explaining the error in the background, and enough of them would saturate the machine. Enable it with AIDB_QUERY_ADVISOR=on.

Fixes

Two concurrent writes could quietly become one. UPDATE and DELETE read, decide, then write, with nothing making that atomic against another statement doing the same. An application built on the engine had to wrap every write in its own lock to work around it. Reproduced 10 times out of 10; writes are now serialized per table.
database.table didn't always mean that table. Qualified names in UPDATE and DELETE could act on the wrong database.
An upsert could corrupt a row's index. After updating an existing row, an exact-key lookup stopped finding it while the row stayed visible in scans. On a composite key duplicate detection stopped working, so repeated upserts accumulated real duplicate rows and the primary key stopped being enforced.
A grouped SUM over a money column returned a value for one group and blank for the rest — silently, with HTTP 200. Counting the same groups was correct, which is what made it hard to see.
A query containing an accented character could drop your connection. Truncating the text for a log line cut mid-character and took down the worker handling the statement.
A table visible in the UI wasn't always one you could query. The Data Lake page listed a table and offered a "Query this" button producing a statement that failed with "table does not exist".
The lake page reported the wrong size — 21 partitions and 1.68 million rows for a lake holding 3 and 240,000.
A filtered catalog query returned unfiltered rows. Asking for one table's columns returned every column of every table, each row individually correct.
The web UI couldn't be built from a clean checkout. Two packages were imported but never declared.

Upgrading

Drop-in. No migration, no configuration change, no new required settings.

DECIMAL columns written before this release were stored as approximate numbers and are widened on read, so existing rows read back as you'd expect. New DECIMAL columns are exact from the start.

Known limitations

GROUP BY and COUNT(DISTINCT) over Parquet are slower than DuckDB (above).
CAST(x AS DECIMAL(10,2)) is accepted but does not apply the scale you ask for — a decimal passes through at its own scale and text or integers become approximate. Declare the column at the scale you want instead; INSERT and UPDATE both coerce to it.
If the text-generation model fails to load, GENERATE returns a sentence saying the model is loading rather than reporting an error. This also affects v1.15.0-ce and is not new here.

Install

curl -fsSL https://get.synapcores.com/install.sh | sh

Docker:

docker run -d --name synapcores -p 8080:8080 \
  -e AIDB_ACCEPT_LICENSE=1 synapcores/community:v1.16.0-ce

Linux x86_64 and aarch64 (glibc 2.31+ and Ubuntu 24.04 builds), macOS aarch64, and multi-arch Docker images on GHCR and Docker Hub. See https://docs.synapcores.com for setup.