Fabric Data Lineage

Problem

Fabric workspaces built out of several lakehouses and a serving warehouse quickly accumulate dozens of tables and views referencing each other across items. There's no built-in way to see that dependency web at a glance — before changing a table or view, an engineer has to manually trace which downstream views break and which upstream sources feed it, often by opening SQL definitions one at a time.

Approach

The repo is self-contained end to end, not just the app. A PySpark notebook included in the project scans every Lakehouse and Warehouse in a workspace and writes three catalog tables — objects, dependencies, and view_definitions — directly from sys.sql_modules and sys.sql_expression_dependencies, with case-collision disambiguation built in. Those three tables are surfaced through a Direct Lake semantic model built right in Fabric (no Power BI Desktop step, no import/refresh schedule — it reads the same OneLake Delta files the notebook just wrote). Three DAX queries pull the raw catalog, and a client-side pipeline joins them into a typed dependency graph (placeholder nodes stand in for references that don't resolve to a scanned object).

From that graph the app renders three complementary views: an interactive canvas for exploring the whole graph, a decomposition tree rooted at each warehouse-layer view for top-down drill-down, and a flat dependency table for quick scanning. Selecting any node runs a BFS in both directions to show its full upstream and downstream impact radius, alongside its SQL definition. Search, per-item and per-object-type filters, a cross-item-only toggle, and a system-schema filter keep large graphs manageable.

The app itself is a React/TypeScript/Vite Fabric App built on Microsoft's Rayfin framework, authenticated via Fabric's embedded auth context and deployed as a native Fabric App item — not a separately hosted web app.

Result

Deployed to a live Fabric workspace, giving engineers a self-service way to check blast radius before making schema changes instead of grepping through SQL by hand. The data source is fully decoupled from the app code: repointing it at a different workspace's semantic model is a one-line config change (a fabric.yaml connection profile), with no source changes required.