SynapCores v1.16.0 — your MySQL history as open Parquet on S3, and money that adds up
What you can do now
Move a MySQL table into your own S3 bucket, on a schedule
Point SynapCores at a MySQL database and an object-storage bucket. It copies the history out as Parquet — the open format every analytics tool reads — and keeps doing it on a schedule.
Register where the data goes:
POST /v1/lake/destinations
{
"name": "warehouse", "kind": "s3",
"bucket": "my-lake", "prefix": "raw/",
"region": "us-east-1",
"credentials": { "kind": "static",
"access_key_id": "...", "secret_access_key": "..." }
}
Then describe the export. You choose what leaves the source — drop a column outright, or replace one with a salted one-way hash, and the original never reaches the bucket:
POST /v1/lake/jobs
{
"name": "nightly-pings",
"source_connection_id": "...", "destination_id": "...",
"tables": [{
"source_table": "pings",
"mode": { "mode": "incremental",
"partition_column": "created_at", "granularity": "day" },
"key_column": "id",
"transforms": [ { "transform": "drop", "column": "email" },
{ "transform": "hash", "column": "phone",
"salt_env": "LAKE_SALT" } ]
}],
"lookback_days": 7, "enabled": true
}
The reason this exists is memory. The extractor this replaces read a 394 GB table with pandas, exhausted 16 GB of RAM and was killed by the kernel. Here every stage is bounded — an unbuffered cursor, one Parquet row group resident at a time, multipart parts that ship and are freed. Exporting 240,000 rows peaked at 99 MB, and that ceiling does not move as the table grows.
Exports also run in a child process. A runaway export cannot take your database down with it, and an out-of-memory kill is reported as one rather than as success.
A partition only counts once its rows and key range are reconciled against the source, and the manifest is written last. Object stores have no rename, so write order is the atomicity: a partition without a verified manifest is invisible to every reader, ours or anyone else's. Re-running replaces a partition instead of appending to it.
Query what you exported — and Parquet other tools wrote
CREATE EXTERNAL TABLE pings STORED AS PARQUET
LOCATION 's3://my-lake/raw/pings/'
PARTITIONED BY (dt DATE)
WITH (CONNECTION = 'warehouse');
SELECT dt, COUNT(*) FROM pings WHERE dt = DATE '2026-09-15' GROUP BY dt;
It behaves like any other table: partitions are pruned from their dt= directory names before an object is opened, row groups are pruned from footer statistics, and only the columns you asked for are fetched. You can join it straight to a live table.
This is not limited to files SynapCores wrote. Point it at Parquet produced by DuckDB, Spark or pandas and query it in place — no import, no copy. Nested data comes through intact, so LIST, STRUCT and MAP columns keep the shape they were written in.
Writes are refused, deliberately: an external table is a view onto files you own.
Serve model predictions over HTTP
Train a model, then call it from your application and get an answer back in milliseconds — no SQL round trip, no separate serving stack:
POST /v2/predict
{ "model": "churn", "rows": [ { "tenure": 14, "monthly": 79.9 } ] }
Eight model families are supported: linear, tree, random forest, gradient boosting, k-nearest neighbours, naive Bayes, SVM and neural. Every one was checked by asking the REST endpoint and the database the same question with the same model and requiring the same answer back, against stored known-good values.
Money that adds up
DECIMAL is now an exact type — it stores the digits, not a binary approximation:
CREATE TABLE invoices (id INT PRIMARY KEY, customer TEXT, amount DECIMAL(12,2));
INSERT INTO invoices VALUES (1,'Acme',19.99),(2,'Acme',0.10),(3,'Globex',0.20);
SELECT SUM(amount) FROM invoices; -- 20.29, not 20.290000000000003
SELECT amount FROM invoices WHERE id = 2; -- 0.10
The scale you declare is the scale you get: a DECIMAL(12,2) column stores and reports 0.10, and a value with more fractional digits than the column declares is rounded half away from zero, as MySQL does. 1.1 and 1.10 are one value — they compare equal and form one group, so a money amount can't split across two rows of a GROUP BY.
A million-row SUM matches what MySQL returns for the same data, and when the column is exported it becomes a real Parquet DECIMAL that other tools read as one.
Over REST, MCP, the CLI and exports a decimal is rendered as exact text rather than a JSON number — most JSON readers parse a number into a float and would undo the exactness on the way out.
Connect a BI tool
Metabase, or anything speaking the MySQL wire protocol, can now connect, list your tables and chart them — including data sitting in Parquet on S3. SHOW GRANTS FOR CURRENT_USER() answers truthfully, so a read-only key is offered read-only.
MySQL spellings that BI tools emit are accepted, including CAST(x AS SIGNED) and CAST(x AS UNSIGNED INTEGER).