sibei-flow / sbflow
Star on GitHub

Your dbt run broke at 3am. The fix was a pull request by morning.

sibei-flow watches your dbt-in-Airflow pipelines. When one breaks from schema drift or a SQL error, it diagnoses the failure, verifies a fix in a sandbox, and opens a reviewable PR — in about 90 seconds, while you sleep.

Star on GitHub $ docker compose up
Apache-2.0 Self-hosted · no cluster BYO / local LLM Only write action: a PR

Nobody's watching when it breaks.

A few times a month an upstream column gets renamed, removed, or retyped, or a bad SQL change slips in, and the nightly dbt run fails. Nobody's awake to see it. You find out hours later, when someone notices a dashboard has gone stale.

Then someone burns 30 to 90 minutes on it, half-awake and off the clock: read the traceback, find the failing model, work out what changed upstream, patch it, re-run. Airflow and dbt tell you that something broke. Neither tells you what the fix is.

It's demoralizing, it comes back every month, and it gets worse as the models multiply. A 3-to-15-person team has no platform crew to build the tooling that would put a stop to it.

the manual loop
  1. page fires — dashboard is stale
  2. open the traceback, half-awake
  3. git blame the failing model
  4. diff the upstream schema by hand
  5. patch the SQL, re-run, wait
  6. next month, again

An incident that resolves itself before you wake up.

The same failure, walked from the 3am break to the pull request waiting in your queue — about 90 seconds, start to finish. There's no standing access to your systems; sibei-flow only ever hears about the run that broke.

03:00:00

Break

dbt_build_orders fails: column "customer_id" does not exist. Airflow's on_failure_callback POSTs a structured failure payload to the brain. That webhook is the entire integration — one line in a DAG.

03:00:12

Detect

The Rust brain classifies the failure as schema drift, in-scope, and opens a durable repair job. Out-of-scope failures — OOM, timeouts — are recorded and never dispatched, so your review queue stays clean.

03:00:31

Diagnose

A bounded agent reads the failing model read-only, pulls the current upstream schema, and pinpoints the drift: customer_id is gone, cust_id is present — a 1:1 rename.

03:01:05

Verify

It drafts a minimal edit and builds it in an ephemeral Docker sandbox: compiled ✓ · ran on a 10k-row sample ✓ · output schema unchanged ✓. A fix that can't even compile never becomes a PR.

03:01:18

Propose

A pull request appears on a branch — never main. It carries the diff, a plain-English explanation, the agent's full reasoning transcript, the verification evidence, and a confidence/risk label.

08:41

You

You wake up, read a five-line diff, and merge. The next run is green. Merging is the approval; closing rejects it; git revert rolls it back. No second tool, no new approval surface.

Two failure classes, honestly scoped.

sibei-flow stays in its lane. It fixes the failures that are small and mechanical to diagnose but expensive to catch at 3am, and hands everything else back to you.

Flagship schema drift

Upstream shape changed

A column is renamed, removed, retyped, or turns nullable and a downstream model breaks. sibei-flow diffs the referenced columns against the live upstream schema and updates the model to match.

code / SQL error

A bad change slipped in

A broken reference or malformed SQL fails the build. The agent reads the error and the source, drafts a corrected diff, and only proposes it once it compiles clean in the sandbox.

cron & scripts

No callback? Wrap it.

For a step with no failure hook, prefix it: sbflow run -- <cmd>. A non-zero exit is captured and sent as the same failure payload — same path, same PR.

The only thing it can write is a pull request.

Adopting a tool that touches your pipelines shouldn't take a security review to veto. The guarantees here aren't policy, they're structural: the credentials to do any harm don't exist in the system.

No standing access

Detection is a webhook. sibei-flow never sits inside your infrastructure or holds a session to it — it only learns about the run that broke.

Source is read-only

It reads your git repo at the failing ref through a scoped token — and nothing else. No writes to your source, ever.

The only write is a PR

On a branch — never main, never a production table. No prod-write credential exists anywhere in the system to be misused.

Verified before you see it

Every candidate is compiled and test-run in an ephemeral sandbox first. If it can't compile, it never reaches your queue.

Human-gated, always

Propose-and-approve, never autonomous. Merge to approve, close to reject, git revert to roll back. No auto-apply in v1.

Your model, your keys

Bring your own LLM — Claude, an OpenAI-compatible endpoint, or a fully local model. Your source and schema never leave an account you control.

The hard invariant, in one line: read-only source in → a branch PR out. Nothing in between can touch main or prod.

Self-host it on a laptop. Read every line.

One docker compose up stands up the whole stack. No cluster, no DSL, no migration. It rides on the dbt-in-Airflow setup you already run, so it's additive and easy to back out.

brain  — Rust · durable Postgres state, webhook, classify, dispatch
worker — Python · the bounded agent loop, behind a BYO LlmProvider
ui     — read-only · run history, transcript, diff, evidence
Built in the openfull v1 scope, shipped
  • Detectwebhook → classified repair job
    shipped
  • Diagnoseagent reads source, drafts a minimal fix
    shipped
  • Verifyephemeral sandbox: compile · sample · schema
    shipped
  • Proposeopen the PR with evidence + confidence
    shipped
  • Hardendedupe, crash recovery, one-line onboarding
    shipped