SynapCores v1.15.0 — columnar tables, speech-to-text, and searchable video

What you can do now

Run analytical queries on columnar tables

You could always ask for a columnar table. Until now you didn't get one — the setting was recorded and then ignored, so you got an ordinary row table with analytical intentions, and updates and deletes were refused outright.

CREATE TABLE events (
  id INT, ts TIMESTAMP, region TEXT, amount DOUBLE
) ENGINE = SYNAPCORES_COLUMNAR;

Now that table really is columnar: a query reads only the columns it asks for, scans run in parallel across cores, and inserts, updates and deletes all work.

Use it for the tables you scan and aggregate — event streams, logs, metrics, anything wide that you query a few columns at a time. Keep row tables for the records you look up one at a time by key.

Two things to know:

• Write in batches. Changing even a few rows rewrites the whole chunk holding them, so loading a thousand rows at once costs far less than a thousand single inserts.
• Primary keys are not enforced on columnar tables. They're accepted as an organisational hint, and duplicates go in. Every analytical warehouse works this way — Snowflake, BigQuery, Redshift, ClickHouse — because checking uniqueness on every insert would destroy the bulk-loading speed you chose columnar for. A row table here does enforce it, so the same statement behaves differently depending on the table type, and we now say so when you create one:

WARNING: columnar table 'events' declares PRIMARY KEY. On columnar storage
PRIMARY KEY / UNIQUE is a clustering and metadata hint only.

Load a large dataset without pushing it through an API

Loading data meant uploading CSV or JSON over HTTP in small batches, with a size cap. That is fine for a sample and hopeless for a hundred gigabytes.

If your data is already in Parquet — the format most data tools export — you can now point the engine at the file and have it taken in directly:

POST /v1/data/import/parquet

Nothing is converted, re-encoded, or sent over the network. Loading is as fast as copying the file.

You must nominate a directory before this works. Set import_dir under [columnar] in your config, and only files inside it can be loaded. If you don't set it, imports are refused — we'd rather a new feature do nothing until you've chosen where it may read than have an upgrade quietly grant access to your filesystem.

Turn recordings into text you can search

SELECT TRANSCRIBE('meeting.mp3') AS transcript;

Transcription in SQL simply didn't exist before — the function was unrecognised in every version we have ever shipped. Now you can transcribe a recording and treat the result like any other text: search it, embed it, summarise it, join it to your other data.

Your users can upload recordings directly from a browser, rather than needing a file already sitting on the server:

POST /v1/transcription/transcribe    upload a file, optionally name the language
GET  /v1/transcription/status        check whether transcription is available

The status endpoint exists so your interface can hide the feature when it isn't available, instead of showing a button that always fails.

Community edition does not include a speech-to-text engine. Transcription will tell you so clearly rather than failing in a confusing way.

Find the moment in a video where something happened

Uploading a video used to give you file details, a thumbnail, and a transcript of what was said. Nothing described what was on screen.

Now the engine watches the video too. You get:

• A readable summary of the footage, with timestamps written into it, stored alongside the transcript — so an ordinary text search across your videos finds it.
• A description of each sampled frame, with its exact time, so your interface can jump straight to the moment.

You choose how often to sample, per upload or as a default — every 30 seconds to start with. Because describing frames takes real time, this happens in the background after upload rather than making the person wait. There's a cap on how many frames any one video will use, and you're told when a long video hits it, so a partial description is never presented as a complete one.

Two ready-made recipes ship for making video and audio libraries searchable.

Get the right answers out of vector search

Three things were quietly wrong, and all three are fixed.

Your similarity search wasn't using the fast index you asked for. Every collection was doing an exhaustive comparison, no matter what you configured — and the settings you sent were echoed back to you as though they'd been applied. Now the real index is built, and it finds substantially more of the right results:

Anything built on similarity — recommendations, retrieval for AI answers, knowledge-graph traversal — gets more accurate for free.

Filters and thresholds were ignored. If you asked for matches above a score, or restricted to a category, you got results regardless. They now apply.

Collections vanished on restart. They were never saved to disk. They persist now.

Also: deleted items no longer come back in results, and you can update an existing entry instead of having to remove and re-add it.

Use more of your graphics card

If you run on NVIDIA hardware, the engine now decides how many requests to run at once by looking at how much memory your card actually has free, rather than using a fixed guess of two. A large card was being wasted; a small one could still be pushed over its limit and take the whole service down with it.

Measured on a 12 GB card with a 7-billion-parameter model, the engine now predicts its own memory use to within 4% and stops short of the edge instead of crossing it.

Apple Silicon support is in place in the code for the first time. It is not switched on in the macOS download yet — see the notes at the end.

Save your work in the query editor

Saving a query used to put it in your browser and nothing read it back, so nothing appeared in a list and nothing could be reopened. Worse, signing in wiped what you'd saved. If you ever said "I'm sure I used to be able to save these" — you were right, and they were being deleted.

Saved queries now live with your data on the server. They survive signing out, clearing your browser, and moving to another machine.

Select every column from one table in a join

SELECT o.*, c.name
FROM orders o JOIN customers c ON c.id = o.customer_id;

This used to be rejected. It works now, including the fully-qualified form.

Faster queries across the board

• Filters are applied earlier and evaluated in bulk rather than row by row. A filtered scan of five million rows went from around 1.6 seconds to under a tenth of a second.
• Grouping and totals are computed in parallel and in a single pass.
• A query no longer reads columns it never mentioned.
• Transcription now uses the whole machine instead of a fixed two jobs at a time: 38% more throughput, and a single recording comes back in 3.0 seconds instead of 4.8.

What we fixed

Transcripts of your media files were invented

This is the one to read.

When the engine indexed audio and video from a watched folder, it did not transcribe them. It stored placeholder text — a sentence naming the file type and its size — and then embedded that, indexed it, and made it searchable.

Every audio and video file ever taken in from a watched folder was searchable against text that was never in the recording. There was no error and no warning; the results simply looked plausible.

Real transcription now runs on that path. If you have indexed media this way, re-index it — what's in there now is not from your recordings.

Restarting the engine stopped your primary keys working

A primary key is supposed to make a duplicate impossible. After the engine restarted, it silently stopped checking.

INSERT INTO accounts (id, owner) VALUES (1, 'original');
INSERT INTO accounts (id, owner) VALUES (1, 'duplicate');
-- rejected, correctly

-- restart the engine, then the same statement again:
INSERT INTO accounts (id, owner) VALUES (1, 'clobbered');
-- accepted

SELECT * FROM accounts WHERE id = 1;
-- (1, 'clobbered')

Not a second row, not an error — the original row is overwritten and gone. Confirmed on the version that was current until today. If you run in production and have ever restarted, this is the reason to upgrade.

Adding an index to a table that already had data did nothing

Indexes were only filled in as rows arrived. So if you loaded your data first and added the index afterwards — the normal way to load a large table — the index stayed empty and your queries quietly went back to reading the whole table. Correct answers, no speed-up, no error to tell you.

The same thing happened on a table with no primary key, even if you created the index first.

Measured on 20,000 rows:

If your table had a primary key and you created the index before loading, your indexes were already working properly. That's the common case and nothing changes for you. The other rows are the bug.

Existing indexes that were left empty are filled in when you upgrade.

Audit tables treated every number as text

Immutable tables — the tamper-evident ones used for audit trails and compliance records — stored every value as text, whatever type you declared.

SELECT amount FROM ledger ORDER BY amount DESC;   -- 100, 25, 9
SELECT SUM(amount) FROM ledger;                   -- empty
SELECT * FROM ledger WHERE amount > 50;           -- error

Sorting put 100 before 25 before 9, because it was comparing them as words. Totals came back empty. Comparisons failed. For records kept precisely because they have to be trustworthy, that's the wrong place to lose the difference between a number and a word. Present since v1.12.0.

Everything else

• An index in any database other than the default was ignored, and one with the same name from the default database was used instead — so a query could return rows that didn't match what you asked for.
• A primary key spanning two columns was accepted and then thrown away.
• An index over several columns returned nothing when you searched on just the first one.
• Looking up a row by key on a columnar table found nothing.
• Adding options in brackets to CREATE TABLE silently turned every column into text and dropped the primary key. That form is now refused with an explanation.
• SELECT * sometimes returned an empty extra column, and in joins exposed an internal identifier.
• Empty values in vector columns lost the distinction between "empty" and "zero".
• Certain filters involving sub-queries or AI functions failed on some queries.
• Builds for NVIDIA hardware ran on the processor while reporting that the graphics card was in use.
• A typo in the interface broke the build of every download.

Testing

Known limitations

• The macOS download does not use Apple Silicon GPU acceleration. The support exists in the code but is not switched on in the published build, and has not yet been tested on a Mac.
• There is no NVIDIA-specific download for this release, although it contains graphics-card fixes.
• Parquet import refuses everything until you nominate a directory (import_dir under [columnar]). This is deliberate.
• Columnar tables don't take compression or chunk-size settings — the storage type is the whole option.
• Audit and time-series tables still ignore constraints you declare on columns. An audit table will accept a duplicate key and replace the earlier row. The tamper-evidence check does catch it afterwards, but the write shouldn't have been allowed.
• Audit tables created before or during this release keep storing numbers as text. The fix applies to newly created tables; there is no conversion for existing ones yet.
• Timestamp columns are reported as text when read over the API.
• Six recipes in the catalogue still fail, the same six as the previous release.

Get it

curl -fsSL https://get.synapcores.com | bash

docker pull synapcores/community:v1.15.0-ce

Linux x86_64 and aarch64 (glibc and Ubuntu 24.04), macOS aarch64, multi-arch Docker on GHCR and Docker Hub.

Full notes and checksums: github.com/SynapCores/synapcores-releases