Architecture

One Rust process (DataFusion in-process) plus a small SQLite file for the ledger and optional auth. One server serves many agents; deploy it next to your data, behind your usual auth.

• Federated SQL — one statement across every registered source, no application-side joins. docs/federated-queries.md
• Pipelines — a YAML file with parameterized SQL becomes a REST endpoint (schema inferred) and a skardi run verb. This is what the loop promotes into. docs/pipelines.md
• Semantic overlay — plain-English descriptions of tables and columns served on GET /data_source, so the agent reads what data means before querying instead of guessing from a schema dump. docs/semantics.md
• Offline jobs — async batch writes with atomic commit, crash recovery and a run ledger you can list and inspect. docs/jobs.md
• Retrieval built in — KNN, FTS and hybrid RRF in SQL; candle() embeddings, chunk(), onnx_predict() and llm_extract() as UDFs, so content and vector land on the same row atomically. docs/embeddings/

_{View interactive diagram →}

Most sources read a store that already holds rows. RSS/Atom is the one that reaches out over the open web at query time — one DataFusion partition per feed so a dead feed degrades alone, and, because feed URLs are agent-authored input, a fetch deliberately left un-sandboxed in OSS for operators to gate at the infrastructure layer. See docs/rss.md.