SynapCores Vector Database - User Documentation

Document Date: September 1st, 2025 Version: 1.0 (Public) Status: Production Ready

Executive Summary

SynapCores provides cloud-native vector database capabilities with advanced indexing, similarity search, and AI-powered embedding generation. This document describes how to integrate SynapCores into your applications for semantic search, recommendations, and AI-powered features.

Table of Contents

1. Overview
2. Getting Started
3. Embedding Generation
4. Distance Metrics
5. Indexing Strategies
6. CRUD Operations
7. SQL Integration
8. REST API
9. Best Practices
10. Common Use Cases

Overview

SynapCores combines traditional SQL database capabilities with native vector operations, enabling you to:

• Store and search high-dimensional embeddings
• Perform semantic similarity searches
• Execute hybrid queries combining relational and vector data
• Generate embeddings directly within the database
• Scale to millions of vectors with sub-100ms query latency

Key Features

Vector Operations:

• Multiple distance metrics (Cosine, Euclidean, Dot Product, Manhattan)
• Advanced indexing with HNSW for fast approximate search
• Exact and approximate nearest neighbor search
• Batch operations for high throughput

SQL Integration:

• Native vector data types
• AI functions callable in SQL queries
• Join vector and relational data in a single query
• Standard SQL syntax with vector extensions

Enterprise Features:

• ACID transactions
• Automatic data persistence
• Multi-tenant isolation
• High availability

Getting Started

1. Create an Account

Sign up at https://synapcores.com to get your API credentials.

2. Obtain API Token

After creating your account, generate an API token from your dashboard:

# Your API token will look like this
export SYNAPCORES_TOKEN="sc_live_abc123xyz..."

3. Connect to Your Database

Socket Connection (SQL Interface):

# Connection via SynapCores native protocol
synapcores://username:password@your-instance.synapcores.com:5433/your_database

REST API:

# Base URL
https://api.synapcores.com/api/v1

Connection Methods:

• Native Socket Protocol: For SQL queries and high-performance operations
• REST API: For language-agnostic HTTP-based access

4. Create Your First Vector Space

Using SQL:

SELECT create_vector_space('products', 384, 'cosine');

Using REST API:

curl -X POST https://api.synapcores.com/api/v1/vectors/collections \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "products",
    "dimensions": 384,
    "distance_metric": "cosine",
    "index_type": "hnsw"
  }'

Embedding Generation

Supported Models

SynapCores provides built-in embedding generation with multiple model options:

Usage

Generate Embeddings in SQL:

-- Use default model (MiniLM)
SELECT EMBED('wireless headphones');

-- Specify model explicitly
SELECT EMBED('wireless headphones', 'minilm');
SELECT EMBED('wireless headphones', 'bert-base');
SELECT EMBED('wireless headphones', 'bert-large');

During Data Insert:

INSERT INTO products (name, description, embedding)
VALUES (
    'Bluetooth Headphones',
    'Premium wireless audio device',
    EMBED('Bluetooth Headphones Premium wireless audio device')
);

Batch Processing:

-- Generate embeddings for existing data
UPDATE products
SET embedding = EMBED(name || ' ' || description)
WHERE embedding IS NULL;

Distance Metrics

Choosing the Right Metric

1. Cosine Similarity (Recommended for Text)

Best for: Text embeddings, semantic search, document similarity

Range: [-1, 1] where 1 = most similar

Use when: Comparing documents, products, or text-based content

SELECT COSINE_SIMILARITY(vector1, vector2) as similarity
FROM comparisons;

Example: Find similar product descriptions

2. Euclidean Distance (L2)

Best for: Spatial data, image embeddings

Range: [0, ∞] where 0 = identical

Use when: Comparing spatial coordinates or image features

SELECT EUCLIDEAN_DISTANCE(vector1, vector2) as distance
FROM comparisons;

Example: Find similar images

3. Dot Product

Best for: Recommendation systems

Range: (-∞, ∞)

Use when: Computing relevance scores with normalized vectors

SELECT INNER_PRODUCT(vector1, vector2) as score
FROM comparisons;

Example: User-item recommendations

4. Manhattan Distance (L1)

Best for: Sparse high-dimensional data

Range: [0, ∞]

Use when: Working with sparse feature vectors

SELECT MANHATTAN_DISTANCE(vector1, vector2) as distance
FROM comparisons;

Indexing Strategies

Flat Index (Exact Search)

Characteristics:

• Guarantees 100% recall (exact results)
• Searches every vector (brute-force)
• Best for small datasets

When to Use:

• < 10,000 vectors
• When exact results are required
• Validation and benchmarking

Performance: 1-10ms for small datasets

-- Create with flat index (default)
SELECT create_vector_space('small_collection', 384, 'cosine');

HNSW Index (Fast Approximate Search)

Characteristics:

• Graph-based approximate nearest neighbor search
• 10-100x faster than flat index
• Tunable accuracy vs. speed

When to Use:

• 10K+ vectors
• Production applications
• When sub-100ms latency is required

Performance: 5-50ms even with millions of vectors

-- Create with HNSW index
SELECT create_vector_space('large_collection', 384, 'cosine', 'hnsw');

Index Selection Guide

CRUD Operations

1. Create Vector Space

Initialize a new collection for vectors:

SQL:

SELECT create_vector_space(
    'products',     -- Space name
    384,            -- Dimensions
    'cosine'        -- Distance metric
);

REST API:

curl -X POST https://api.synapcores.com/api/v1/vectors/collections \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "products",
    "dimensions": 384,
    "distance_metric": "cosine",
    "index_type": "hnsw"
  }'

Response:

{
  "status": "success",
  "data": {
    "name": "products",
    "dimensions": 384,
    "distance_metric": "cosine",
    "index_type": "hnsw",
    "created_at": "2025-09-01T10:00:00Z"
  }
}

2. Insert Vectors

Single Insert with Auto-Generated ID

SQL:

INSERT INTO vector_spaces.products (values, metadata)
VALUES (
    EMBED('Wireless Bluetooth Headphones'),
    '{"product_id": "12345", "category": "electronics"}'::JSON
);

REST API:

curl -X POST https://api.synapcores.com/api/v1/vectors/collections/products/vectors \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "vectors": [{
      "values": [0.1, 0.2, 0.3, ...],
      "metadata": {
        "product_id": "12345",
        "category": "electronics"
      }
    }]
  }'

Insert with Custom ID

SQL:

INSERT INTO vector_spaces.products (id, values, metadata)
VALUES (
    'prod_12345',
    EMBED('Wireless Bluetooth Headphones'),
    '{"category": "electronics"}'::JSON
);

Batch Insert (Recommended for Bulk Data)

SQL:

-- Insert multiple vectors efficiently
INSERT INTO vector_spaces.products (values, metadata)
SELECT
    EMBED(description),
    JSON_BUILD_OBJECT('product_id', product_id, 'category', category)
FROM products
WHERE embedding IS NULL
LIMIT 1000;

REST API:

curl -X POST https://api.synapcores.com/api/v1/vectors/collections/products/vectors \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "vectors": [
      {"values": [0.1, ...], "metadata": {"product_id": "1"}},
      {"values": [0.2, ...], "metadata": {"product_id": "2"}},
      ...
    ]
  }'

Performance Tip: Batch inserts are 10-100x faster than individual inserts.

3. Search Vectors

Semantic Search

SQL:

SELECT
    v.id,
    v.metadata->>'product_id' as product_id,
    v.metadata->>'category' as category,
    COSINE_SIMILARITY(v.values, EMBED('wireless headphones')) as similarity
FROM vector_spaces.products v
WHERE COSINE_SIMILARITY(v.values, EMBED('wireless headphones')) > 0.7
ORDER BY similarity DESC
LIMIT 10;

REST API:

curl -X POST https://api.synapcores.com/api/v1/vectors/collections/products/search \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query_text": "wireless headphones",
    "k": 10,
    "threshold": 0.7,
    "include_metadata": true
  }'

Response:

{
  "status": "success",
  "data": [
    {
      "id": "prod_12345",
      "score": 0.95,
      "metadata": {
        "product_id": "12345",
        "category": "electronics"
      }
    },
    {
      "id": "prod_67890",
      "score": 0.87,
      "metadata": {
        "product_id": "67890",
        "category": "electronics"
      }
    }
  ],
  "total_results": 2,
  "query_time_ms": 12
}

Hybrid Search (Vectors + Filters)

Combine semantic search with traditional SQL filters:

SELECT
    p.product_id,
    p.name,
    p.price,
    COSINE_SIMILARITY(p.embedding, EMBED('noise cancelling headphones')) as similarity
FROM products p
WHERE
    p.category = 'electronics'
    AND p.price BETWEEN 50 AND 200
    AND p.in_stock = true
    AND COSINE_SIMILARITY(p.embedding, EMBED('noise cancelling headphones')) > 0.7
ORDER BY similarity DESC, p.price ASC
LIMIT 20;

Search with Metadata Filters (REST API)

curl -X POST https://api.synapcores.com/api/v1/vectors/collections/products/search \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query_text": "wireless headphones",
    "k": 20,
    "threshold": 0.7,
    "filter": {
      "category": "electronics",
      "in_stock": true
    }
  }'

4. Update Vectors

SQL:

UPDATE vector_spaces.products
SET
    values = EMBED('Updated product description'),
    metadata = JSON_BUILD_OBJECT(
        'product_id', '12345',
        'category', 'audio',
        'updated_at', CURRENT_TIMESTAMP
    )
WHERE id = 'prod_12345';

REST API:

curl -X PUT https://api.synapcores.com/api/v1/vectors/collections/products/vectors/prod_12345 \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "values": [0.1, 0.2, ...],
    "metadata": {
      "category": "audio",
      "updated_at": "2025-09-01T10:00:00Z"
    }
  }'

5. Delete Vectors

SQL:

-- Delete by ID
DELETE FROM vector_spaces.products
WHERE id = 'prod_12345';

-- Delete by criteria
DELETE FROM vector_spaces.products
WHERE metadata->>'category' = 'discontinued';

REST API:

# Delete single vector
curl -X DELETE https://api.synapcores.com/api/v1/vectors/collections/products/vectors/prod_12345 \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN"

# Batch delete
curl -X POST https://api.synapcores.com/api/v1/vectors/collections/products/delete_batch \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"ids": ["prod_12345", "prod_67890"]}'

6. Get Vector by ID

SQL:

SELECT id, values, metadata
FROM vector_spaces.products
WHERE id = 'prod_12345';

REST API:

curl -X GET https://api.synapcores.com/api/v1/vectors/collections/products/vectors/prod_12345 \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN"

SQL Integration

Vector Data Type

Create tables with native vector columns:

CREATE TABLE products (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL,
    description TEXT,
    price DECIMAL(10,2),
    category TEXT,
    embedding VECTOR(384)  -- 384-dimensional vector
);

AI Functions

EMBED() - Generate Embeddings

-- Default model (MiniLM, 384 dimensions)
SELECT EMBED('wireless headphones');

-- Specify model
SELECT EMBED('wireless headphones', 'minilm');    -- 384d
SELECT EMBED('wireless headphones', 'bert-base'); -- 768d
SELECT EMBED('wireless headphones', 'bert-large'); -- 1024d

-- Use in INSERT
INSERT INTO products (name, description, embedding)
VALUES (
    'Bluetooth Headphones',
    'Premium wireless audio device',
    EMBED('Bluetooth Headphones Premium wireless audio device')
);

Vector Similarity Functions

COSINE_SIMILARITY():

SELECT
    product_id,
    name,
    COSINE_SIMILARITY(embedding, EMBED('wireless bluetooth headphones')) as similarity_score
FROM products
WHERE COSINE_SIMILARITY(embedding, EMBED('wireless bluetooth headphones')) > 0.7
ORDER BY similarity_score DESC
LIMIT 10;

EUCLIDEAN_DISTANCE():

SELECT
    id,
    EUCLIDEAN_DISTANCE(embedding, :query_vector) as distance
FROM image_embeddings
ORDER BY distance ASC
LIMIT 5;

INNER_PRODUCT():

SELECT
    user_id,
    item_id,
    INNER_PRODUCT(user_embedding, item_embedding) as relevance_score
FROM recommendations
WHERE relevance_score > 0.5
ORDER BY relevance_score DESC;

Advanced SQL Patterns

Semantic Search with Joins

-- Find customers who purchased similar products
SELECT
    c.customer_id,
    c.customer_name,
    p.product_name,
    o.order_date,
    COSINE_SIMILARITY(p.embedding, EMBED('premium headphones')) as relevance
FROM customers c
JOIN orders o ON c.customer_id = o.customer_id
JOIN order_items oi ON o.order_id = oi.order_id
JOIN products p ON oi.product_id = p.product_id
WHERE
    o.order_date > CURRENT_DATE - INTERVAL '90 days'
    AND COSINE_SIMILARITY(p.embedding, EMBED('premium headphones')) > 0.75
ORDER BY relevance DESC, o.order_date DESC
LIMIT 50;

Aggregations with Vectors

-- Average similarity by category
SELECT
    category,
    COUNT(*) as product_count,
    AVG(COSINE_SIMILARITY(embedding, EMBED('premium quality'))) as avg_relevance
FROM products
GROUP BY category
HAVING avg_relevance > 0.6
ORDER BY avg_relevance DESC;

Subqueries with Vectors

-- Find products similar to top sellers
WITH top_products AS (
    SELECT product_id, embedding
    FROM products p
    JOIN order_items oi ON p.product_id = oi.product_id
    GROUP BY p.product_id, p.embedding
    ORDER BY SUM(oi.quantity) DESC
    LIMIT 10
)
SELECT DISTINCT
    p.product_id,
    p.name,
    MAX(COSINE_SIMILARITY(p.embedding, tp.embedding)) as max_similarity
FROM products p
CROSS JOIN top_products tp
WHERE p.product_id NOT IN (SELECT product_id FROM top_products)
GROUP BY p.product_id, p.name
HAVING MAX(COSINE_SIMILARITY(p.embedding, tp.embedding)) > 0.8
ORDER BY max_similarity DESC
LIMIT 20;

REST API

Base URL

https://api.synapcores.com/api/v1

Authentication

All requests require Bearer token authentication:

Authorization: Bearer <your_api_token>

Endpoints

Complete Workflow Example

# 1. Create collection
curl -X POST https://api.synapcores.com/api/v1/vectors/collections \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "documents",
    "dimensions": 384,
    "distance_metric": "cosine",
    "index_type": "hnsw"
  }'

# 2. Insert documents
curl -X POST https://api.synapcores.com/api/v1/vectors/collections/documents/vectors \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "vectors": [{
      "id": "doc_001",
      "values": [0.1, 0.2, ...],
      "metadata": {"title": "Getting Started", "type": "guide"}
    }]
  }'

# 3. Search documents
curl -X POST https://api.synapcores.com/api/v1/vectors/collections/documents/search \
  -H "Authorization: Bearer $SYNAPCORES_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query_text": "how to get started",
    "k": 10,
    "threshold": 0.7
  }'

Rate Limits

Best Practices

1. Choosing Embedding Dimensions

Recommendation: Start with 384 dimensions (MiniLM) for most use cases.

2. Batch Operations

Always use batch operations for:

• Bulk data imports
• Periodic reindexing
• Data migrations

Performance Gains:

• 10-100x faster than individual operations
• Reduced API call overhead
• Better throughput

Optimal Batch Size: 100-1000 vectors per request

3. Query Optimization

Use Filters Effectively:

-- Good: Filter before vector search
SELECT *
FROM products
WHERE
    category = 'electronics'  -- Traditional filter first
    AND price < 200
    AND COSINE_SIMILARITY(embedding, EMBED('headphones')) > 0.7

Cache Frequent Queries:

• Cache popular search embeddings in your application
• Reuse embeddings for identical queries
• Reduces embedding generation overhead

4. Error Handling

Implement Retry Logic:

import time
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

session = requests.Session()
retry = Retry(
    total=3,
    backoff_factor=0.3,
    status_forcelist=[429, 500, 502, 503, 504]
)
adapter = HTTPAdapter(max_retries=retry)
session.mount('https://', adapter)

Handle Rate Limits:

• Check response headers for rate limit info
• Implement exponential backoff
• Queue requests during high traffic

5. Monitoring and Observability

Track Key Metrics in Your Application:

• Query latency (p50, p95, p99)
• Search accuracy/relevance
• Embedding generation time
• API error rates

Log Important Events:

• Failed embedding generations
• Slow queries (> 100ms)
• Rate limit hits

Common Use Cases

Use Case 1: E-Commerce Semantic Search

-- Create product table with embeddings
CREATE TABLE products (
    product_id SERIAL PRIMARY KEY,
    name TEXT,
    description TEXT,
    category TEXT,
    price DECIMAL(10,2),
    embedding VECTOR(384)
);

-- Index products
INSERT INTO products (name, description, category, price, embedding)
SELECT
    name,
    description,
    category,
    price,
    EMBED(name || ' ' || description)
FROM product_catalog;

-- Search by semantic meaning
SELECT
    product_id,
    name,
    price,
    COSINE_SIMILARITY(embedding, EMBED('wireless headphones')) as relevance
FROM products
WHERE relevance > 0.7
ORDER BY relevance DESC
LIMIT 20;

Use Case 2: Content Recommendations

-- Find similar articles based on user reading history
WITH user_interests AS (
    SELECT AVG(a.embedding) as avg_embedding
    FROM user_reading_history urh
    JOIN articles a ON urh.article_id = a.article_id
    WHERE urh.user_id = :user_id
)
SELECT
    a.article_id,
    a.title,
    COSINE_SIMILARITY(a.embedding, ui.avg_embedding) as relevance
FROM articles a
CROSS JOIN user_interests ui
WHERE a.article_id NOT IN (
    SELECT article_id FROM user_reading_history WHERE user_id = :user_id
)
AND relevance > 0.6
ORDER BY relevance DESC
LIMIT 10;

Use Case 3: Duplicate Detection

-- Find near-duplicate documents
SELECT
    d1.document_id as doc1_id,
    d2.document_id as doc2_id,
    COSINE_SIMILARITY(d1.embedding, d2.embedding) as similarity
FROM documents d1
JOIN documents d2 ON d1.document_id < d2.document_id
WHERE COSINE_SIMILARITY(d1.embedding, d2.embedding) > 0.95
ORDER BY similarity DESC;

Use Case 4: Customer Support Routing

-- Find similar resolved tickets
SELECT
    t.ticket_id,
    t.subject,
    t.resolution,
    COSINE_SIMILARITY(t.embedding, EMBED(:new_ticket_text)) as similarity
FROM support_tickets t
WHERE
    t.status = 'resolved'
    AND similarity > 0.8
ORDER BY similarity DESC
LIMIT 5;

Troubleshooting

Slow Query Performance

Symptoms: Queries taking > 100ms

Solutions:

1. Verify HNSW index is being used (check query plan)
2. Reduce number of results requested (lower k value)
3. Use metadata filters to narrow search space
4. Consider using lower-dimensional embeddings (384 instead of 768)

High API Error Rates

Symptoms: Frequent 429 (rate limit) or 5xx errors

Solutions:

1. Implement exponential backoff retry logic
2. Use batch operations instead of individual requests
3. Contact support to increase rate limits
4. Cache frequently used embeddings

Unexpected Search Results

Symptoms: Irrelevant results in semantic search

Solutions:

1. Increase similarity threshold (try 0.8 instead of 0.7)
2. Verify input text is being embedded correctly
3. Check that the correct embedding model is being used
4. Review metadata filters for correctness

Embedding Generation Failures

Symptoms: EMBED() function errors or timeouts

Solutions:

1. Verify text length is under 512 tokens
2. Check for special characters or encoding issues
3. Retry with exponential backoff
4. Contact support if errors persist

Client Libraries

Python

import requests

class SynapCoresClient:
    def __init__(self, api_token):
        self.base_url = "https://api.synapcores.com/api/v1"
        self.headers = {
            "Authorization": f"Bearer {api_token}",
            "Content-Type": "application/json"
        }

    def search(self, collection, query_text, k=10, threshold=0.7):
        response = requests.post(
            f"{self.base_url}/vectors/collections/{collection}/search",
            headers=self.headers,
            json={
                "query_text": query_text,
                "k": k,
                "threshold": threshold
            }
        )
        return response.json()

# Usage
client = SynapCoresClient("your_api_token")
results = client.search("products", "wireless headphones", k=10)

JavaScript/TypeScript

class SynapCoresClient {
    constructor(apiToken) {
        this.baseUrl = 'https://api.synapcores.com/api/v1';
        this.headers = {
            'Authorization': `Bearer ${apiToken}`,
            'Content-Type': 'application/json'
        };
    }

    async search(collection, queryText, k = 10, threshold = 0.7) {
        const response = await fetch(
            `${this.baseUrl}/vectors/collections/${collection}/search`,
            {
                method: 'POST',
                headers: this.headers,
                body: JSON.stringify({
                    query_text: queryText,
                    k,
                    threshold
                })
            }
        );
        return await response.json();
    }
}

// Usage
const client = new SynapCoresClient('your_api_token');
const results = await client.search('products', 'wireless headphones', 10);

Performance Expectations

Query Latency

Latency measured from SynapCores servers. Add network latency for total client response time.

Throughput

Scalability Limits

Support and Resources

Documentation

• Developer Docs: https://docs.synapcores.com
• API Reference: https://docs.synapcores.com/api
• SQL Guide: https://docs.synapcores.com/sql

Community

• Community Forum: https://community.synapcores.com
• Discord: https://discord.gg/synapcores
• Stack Overflow: Tag your questions with synapcores

Support

• Email: support@synapcores.com
• Chat: Available in dashboard
• Status Page: https://status.synapcores.com

Tutorials

• Getting Started: Build your first semantic search in 15 minutes
• Production Deployment: Best practices for scaling to production
• Advanced Patterns: Hybrid search, RAG, and multi-modal applications

Pricing

Visit https://synapcores.com/pricing for current pricing details.

Free Tier:

• 100K vectors
• 1M API requests/month
• Community support

Pro Tier:

• 10M vectors
• Unlimited API requests
• Email support
• 99.9% SLA

Enterprise Tier:

• Unlimited vectors
• Dedicated support
• Custom SLAs
• On-premise deployment options

Limitations

Current Limitations

1. Dimension Changes: Vector space dimension cannot be changed after creation
2. Metric Changes: Distance metric cannot be changed after space creation
3. Max Dimension: 4096 dimensions maximum
4. Batch Size: 1,000 vectors per request maximum

Planned Features

• Quantization for reduced storage costs
• GPU-accelerated search
• Multi-vector per document support
• Advanced filtered search optimizations
• Real-time index updates

Conclusion

SynapCores provides a production-ready, cloud-native vector database with:

✅ Semantic Search - Find similar content by meaning, not keywords ✅ SQL Integration - Combine vector and relational queries ✅ Easy to Use - Simple API and SQL functions ✅ High Performance - Sub-100ms queries even at scale ✅ Fully Managed - No infrastructure to maintain

Start building AI-powered applications today at https://synapcores.com

Document Version: 1.0 (Public) Last Updated: September 1st, 2025 For Technical Support: support@synapcores.com

Copyright © 2025 SynapCores. All rights reserved. Performance characteristics may vary based on workload patterns and network conditions.