Skip to main content
Version: 0.2.0

Federated Queries

One of Skardi's most powerful features is the ability to JOIN data across different source types in a single SQL query. DataFusion handles the federation transparently.

Example: CSV + PostgreSQL​

Join a CSV file with a PostgreSQL table and write results back to PostgreSQL:

metadata:
  name: "federated_join_and_insert"
  version: "1.0"

query: |
  INSERT INTO user_order_stats (user_id, user_name, total_orders, total_spent)
  SELECT
    u.id as user_id,
    u.name as user_name,
    COUNT(o.order_id) as total_orders,
    SUM(o.amount) as total_spent
  FROM users u                    -- PostgreSQL table
  INNER JOIN csv_orders o         -- CSV file
    ON u.id = o.user_id
  WHERE u.name = {name}
  GROUP BY u.id, u.name

More Examples​

Federated query examples are included in each data source's docs:

  • PostgreSQL — pipelines/federated_join_and_insert.yaml
  • MySQL — pipelines/federated_join_and_insert.yaml
  • SQLite — pipelines/federated_join_and_insert.yaml
  • Redis — pipelines/federated_join_and_insert.yaml