Streaming Tables vs. Materialized Views
If you’re building pipelines in Databricks, you’ll eventually hit a fork in the road: should this dataset be a Streaming Table or a Materialized View? They look similar on the surface—both are managed by Unity Catalog, both are backed by Lakeflow/Delta Live Tables (DLT), and both handle incremental processing for you under the hood. But they solve fundamentally different problems.
Get the choice right and your pipeline is fast, cheap, and correct. Get it wrong and you end up with stale metrics, runaway compute bills, or joins that quietly return the wrong numbers. Here’s how to tell them apart.
The Core Difference
It comes down to semantics:
- Streaming Tables (ST) process data using streaming semantics—each row is seen exactly once.
- Materialized Views (MV) process data using batch semantics—recomputing or incrementally updating to guarantee mathematical correctness.
That single distinction—”see each row once” versus “always stay correct”—drives every other trade-off below.
Side-by-Side Comparison
| Feature | Streaming Tables (ST) | Materialized Views (MV) |
|---|---|---|
| Primary use case | Ingestion, Bronze layer, append-only sources | Transformations, Silver/Gold layers, BI reporting |
| Underlying semantics | Streaming (spark.readStream) | Batch (spark.read) |
| Source data expectation | Append-only (logs, message queues, Auto Loader) | Handles updates, deletes, and CDC changes |
| How joins behave | Fast-but-wrong: joins don’t recompute if dimensions change | Always-correct: recalculates when base tables change |
| Row processing | Each row processed exactly once | Incremental updates or full recomputation |
| Syntax | CREATE OR REFRESH STREAMING TABLE… | CREATE OR REFRESH MATERIALIZED VIEW… |
Streaming Tables, in Depth
Streaming tables are built explicitly for streaming data ingestion into your lakehouse.
Data freshness. They deliver low-latency, incremental updates as new data arrives—ideal for keeping a Bronze layer current.
State management. A streaming table sees incoming data only once. This has a subtle consequence: if you change the table’s query (say, you tweak a transformation), only new rows get the updated logic. Existing rows stay as they were—unless you trigger a destructive full refresh to reprocess everything.
Limitations. Streaming tables can’t handle complex stateful operations like full, non-time-bounded aggregations, out-of-order changes, or recalculating late-arriving dimension changes in a join. This is the “fast-but-wrong” trade-off: if a dimension updates after the fact, the streaming join won’t go back and fix it.
Ideal sources. Cloud object storage via Auto Loader, or message queues like Apache Kafka—anywhere data is exclusively appended.
Materialized Views, in Depth
Materialized views are precomputed query results, physically stored as Delta tables, designed to cut latency for end-user BI workloads.
Data correctness. This is the headline feature. Databricks tracks changes in the upstream data and applies them. If an upstream dimension record updates, the MV recalculates so your business metrics stay completely correct. No stale joins, no silently wrong numbers.
Incrementalization. Don’t mistake “always correct” for “always expensive.” Databricks evaluates the underlying logic and runs incremental updates instead of full scans whenever possible—keeping your results accurate while trimming cloud compute costs.
Ideal sources. Cleaned Silver tables, complex star-schema dimension joins, and aggregated Gold summary tables that feed dashboards.
When to Choose Which
Choose a Streaming Table if:
- You’re pulling raw records from an external source or landing directory where data is exclusively appended.
- You want the efficiency of exactly-once row ingestion.
- You’re building the Bronze layer of a medallion architecture.
Choose a Materialized View if:
- You’re exposing the table to business analysts or dashboards that need fast query responses.
- You have multi-table joins or complex window aggregations.
- Your upstream data undergoes frequent updates or deletes (CDC) and correctness is non-negotiable.
- You’re building Silver/Gold transformation and reporting layers.
The Bottom Line
Think of it as a division of labor along your medallion architecture:
Streaming Tables ingest raw, append-only data cheaply and incrementally at the Bronze layer. Materialized Views take over downstream at Silver and Gold, where correctness, complex joins, and BI-ready performance matter most.
The two aren’t competitors—they’re partners. Most well-designed Databricks pipelines use streaming tables to land data fast and materialized views to serve it correctly. Match each to the job it was built for, and you get the best of both: low-latency ingestion and trustworthy metrics.