Skip to content

Embed dirsql in your application ​

Run the same engine in-process instead of as a server: construct a DirSQL over a directory, hand it your tables, and query it like any library — no HTTP, no separate process. The SDKs for Python, TypeScript, and Rust are thin bindings over one shared core, so behavior matches the CLI exactly.

1. Install the SDK ​

bash
uv add dirsql
bash
pnpm add dirsql
bash
cargo add dirsql

2. Index a directory and query it ​

Suppose comments live as JSON files filed by thread:

comments/t1/c1.json    # {"body": "first!", "author": "alice"}
comments/t1/c2.json    # {"body": "nice post", "author": "bob"}
comments/t2/c1.json    # {"body": "following up", "author": "alice"}

Unlike a config-file table, a programmatic table takes an on_file callback — your code reads each matched file and returns its rows. The row's columns are exactly what the callback returns; dirsql merges nothing on top (see Columns), so the callback derives the thread from the file's path itself:

python
import asyncio
import json
import os

from dirsql import DirSQL, Table


def on_file(path: str) -> list[dict]:
    thread = os.path.basename(os.path.dirname(path))
    with open(path, encoding="utf-8") as f:
        return [{**json.load(f), "thread": thread}]


async def main() -> None:
    db = DirSQL(
        "./comments-root",
        tables=[
            Table(
                name="comments",
                ddl="CREATE TABLE comments (thread TEXT, author TEXT, body TEXT)",
                glob="comments/*/*.json",
                on_file=on_file,
            )
        ],
    )
    rows = await db.query("SELECT thread, author, body FROM comments ORDER BY thread, author")
    for row in rows:
        print(f"[{row['thread']}] {row['author']}: {row['body']}")


asyncio.run(main())
typescript
import { readFileSync } from "node:fs";
import { basename, dirname } from "node:path";
import { DirSQL } from "dirsql";

const db = new DirSQL({
  root: "./comments-root",
  tables: [
    {
      name: "comments",
      ddl: "CREATE TABLE comments (thread TEXT, author TEXT, body TEXT)",
      glob: "comments/*/*.json",
      onFile: (path) => [
        { ...JSON.parse(readFileSync(path, "utf8")), thread: basename(dirname(path)) },
      ],
    },
  ],
});

const rows = await db.query(
  "SELECT thread, author, body FROM comments ORDER BY thread, author",
);
for (const row of rows) {
  console.log(`[${row.thread}] ${row.author}: ${row.body}`);
}
rust
use std::collections::HashMap;

use dirsql::{DirSQL, Table, Value};

fn on_file(path: &str) -> Vec<HashMap<String, Value>> {
    let raw = std::fs::read_to_string(path).unwrap_or_default();
    let Ok(serde_json::Value::Object(obj)) = serde_json::from_str(&raw) else {
        return vec![];
    };
    let mut row = HashMap::new();
    for (key, value) in obj {
        if let serde_json::Value::String(s) = value {
            row.insert(key, Value::Text(s));
        }
    }
    let thread = std::path::Path::new(path)
        .parent()
        .and_then(|p| p.file_name())
        .and_then(|n| n.to_str())
        .unwrap_or_default();
    row.insert("thread".into(), Value::Text(thread.into()));
    vec![row]
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = DirSQL::builder()
        .root("./comments-root")
        .table(Table::new(
            "comments",
            "CREATE TABLE comments (thread TEXT, author TEXT, body TEXT)",
            "comments/*/*.json",
            on_file,
        ))
        .build()?;

    let rows = db.query("SELECT thread, author, body FROM comments ORDER BY thread, author")?;
    for row in rows {
        if let (Value::Text(thread), Value::Text(author), Value::Text(body)) =
            (&row["thread"], &row["author"], &row["body"])
        {
            println!("[{thread}] {author}: {body}");
        }
    }
    Ok(())
}

Rows come back keyed by column name (a dict / object / HashMap<String, Value> per row); all three programs print:

[t1] alice: first!
[t1] bob: nice post
[t2] alice: following up

A few things happened implicitly. In Python and TypeScript the constructor returned immediately and the scan ran in the background — query awaited readiness for you. In Rust, .build() scanned synchronously (there is a build_async() for the non-blocking equivalent). Queries are read-only in every SDK. The complete contract — constructor parameters, ready, value type mappings, error shapes — is the SDK reference.

3. React to changes in-process ​

watch() yields the same row events the server streams over /events, as your language's native async iteration:

python
async for event in db.watch():
    print(event.action, event.table, event.row)
typescript
for await (const event of db.watch()) {
  console.log(event.action, event.table, event.row);
}
rust
use futures::StreamExt;

let mut stream = db.watch()?;
while let Some(event) = stream.next().await {
    println!("{event:?}");
}

Event fields and actions are under RowEvent.

Reusing your CLI setup ​

Everything the config file expresses is available programmatically — ignore patterns, persist, extensions — and the constructor's config parameter loads a .dirsql.toml directly, so an application can share the exact table definitions the CLI serves (constructor reference).

Released under the MIT License.