{"id":5203,"date":"2026-05-31T21:31:11","date_gmt":"2026-05-31T21:31:11","guid":{"rendered":"https:\/\/ranaghazzi.com\/?p=5203"},"modified":"2026-05-31T22:14:30","modified_gmt":"2026-05-31T22:14:30","slug":"windowing-and-watermarking-in-databricks-streaming","status":"publish","type":"post","link":"https:\/\/ranaghazzi.com\/?p=5203","title":{"rendered":"Windowing and Watermarking in Databricks Streaming"},"content":{"rendered":"<p><style>\n    .light-font-container, .light-font-container p, .light-font-container h2, .light-font-container li {<br \/>\n        font-weight: #FFFFFF !important;<br \/>\n    }<br \/>\n<\/style>\n<\/p>\n<div class=\"light-font-container\" style=\"background-color: #FFFFFF; padding: 40px; border-radius: 15px;\">\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Windowing &amp; Watermarking in Databricks Streaming<\/h2>\n\n\n<div class=\"wp-block-post-date\"><time datetime=\"2026-05-31T21:31:11+00:00\">May 31, 2026<\/time><\/div>\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">What Is Windowing?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">In a continuous stream, data never stops arriving. Windowing is the mechanism that groups streaming events into finite, time-bounded buckets so you can compute meaningful aggregations \u2014 counts, averages, sums \u2014 over a defined period rather than over all data since the beginning of time.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Spark Structured Streaming supports three window types.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Tumbling Windows<\/strong> are fixed-size, non-overlapping buckets. Every event belongs to exactly one window \u2014 the simplest and most memory-efficient option.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>df.groupBy(window(\"event_time\", \"10 minutes\")).count()\n# &#91;0:00\u20130:10] &#91;0:10\u20130:20] &#91;0:20\u20130:30] ...\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Sliding Windows<\/strong> are fixed-size but advance at a shorter interval, creating overlap. An event can belong to multiple windows simultaneously, giving you a rolling view of the data.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>df.groupBy(window(\"event_time\", \"10 minutes\", \"5 minutes\")).count()\n# &#91;0:00\u20130:10] &#91;0:05\u20130:15] &#91;0:10\u20130:20] ...\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Session Windows are dynamic, defined by inactivity gaps rather than clock time. A window opens when an event arrives and closes after a period of silence. Ideal for user-session analytics.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>df.groupBy(session_window(\"event_time\", \"5 minutes\"), \"userId\").count()\n# Window closes after 5 min of inactivity per user\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Watermarking: Taming Late Data<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The hard problem with event-time processing: events don&#8217;t arrive in order. A sensor reading timestamped at 10:00 might reach Kafka at 10:12 due to network lag, buffering, or a brief device outage. Without watermarking, Spark holds state open for every window forever \u2014 just in case something arrives late. That means unbounded memory growth and eventual crashes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A watermark is a promise: <em>&#8220;I will wait up to X minutes for late data. After that, I discard it and close the window.&#8221;<\/em><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>df \\\n  .withWatermark(\"event_time\", \"10 minutes\") \\\n  .groupBy(window(\"event_time\", \"5 minutes\")) \\\n  .count()\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>How the watermark moves.<\/strong> It tracks the maximum event timestamp Spark has seen, then subtracts your threshold. It only ever moves forward \u2014 never backward.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Max event_time seen:  10:30:00\nWatermark threshold:  \u2212 10 minutes\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\nCurrent watermark:    10:20:00\n\n\u2192 Events with event_time &lt; 10:20 are dropped\n\u2192 Windows ending before 10:20 are finalised, state is freed\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Output modes<\/strong> control when results are emitted downstream:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>append \u2014 emits only finalised (past watermark) windows. Each result written once. Requires watermark.<\/li>\n\n\n\n<li>update \u2014 emits any window that changed this micro-batch, including partial results.<\/li>\n\n\n\n<li>complete \u2014 emits all windows every batch. State grows unboundedly without a watermark.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">append + watermark is the production-safe default. It bounds state size, guarantees each result is written exactly once, and prevents OOM on long-running streams.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Sizing Your Watermark in Production<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Your watermark threshold is a dial with a direct tradeoff:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Smaller watermark<\/th><th>Larger watermark<\/th><\/tr><\/thead><tbody><tr><td>Lower output latency<\/td><td>Higher output latency<\/td><\/tr><tr><td>Lower output latency<\/td><td>Less dropped late data<\/td><\/tr><tr><td>Less memory used<\/td><td>More memory used<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The practical approach: measure your actual lag distribution first, then set the watermark to 2\u20133\u00d7 your P95 lag.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Why 2\u20133\u00d7?<\/strong> Lag is not constant \u2014 it has a distribution. If your median lag is 2 minutes but your P99 is 6 minutes, a watermark of 2 minutes drops roughly 1% of events. A watermark of 6 minutes captures almost everything but adds 6 minutes of output latency. The 2\u20133\u00d7 multiplier gives you headroom for lag spikes without blindly targeting an extreme outlier.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 1 \u2014 Measure lag per event.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To compute lag you need to subtract two timestamps. You can&#8217;t do this directly in Spark \u2014 the result type is ambiguous. Instead, convert both to integers using unix_timestamp(), which returns seconds since the Unix epoch (1 Jan 1970). Subtracting two integers gives you a clean number of seconds.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>event_time = 10:20:00  \u2192  *1705315200\nnow        = 10:30:45  \u2192           *1705315845\nlag        = *1705315845 \u2212- *1705315200 = 645 seconds (\u2248 10.75 minutes)\n<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>df = df.withColumn(\n    \"lag_seconds\",\n    unix_timestamp(current_timestamp()) - unix_timestamp(col(\"event_time\"))\n)\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 2 \u2014 Find your distribution.<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>df.agg(\n    avg(\"lag_seconds\").alias(\"avg_lag\"),\n    percentile_approx(\"lag_seconds\", 0.95).alias(\"p95_lag\"),\n    percentile_approx(\"lag_seconds\", 0.99).alias(\"p99_lag\"),\n    max(\"lag_seconds\").alias(\"max_lag\")\n).show()\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 3 \u2014 Set and monitor.<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># If P95 lag = 3 minutes \u2192 set watermark to 6\u20139 minutes\ndf.withWatermark(\"event_time\", \"7 minutes\")\n<\/code><\/pre>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">Start wide, then tighten. Run with a conservatively large watermark for 24\u201348 hours covering peak traffic, collect real lag data, then recalculate. It&#8217;s far harder to recover from silently dropped data than to accept a bit of extra output latency.<\/p>\n<\/blockquote>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Monitoring Dropped Events<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Once your stream is running, you need to verify your watermark isn&#8217;t silently discarding too much data. The key metric is numRowsDroppedByWatermark, which lives inside query.lastProgress \u2014 a Python dict updated after each micro-batch.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import json\nprint(json.dumps(query.lastProgress, indent=2))\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The relevant structure in the output:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"batchId\": 42,\n  \"numInputRows\": 50000,\n  \"eventTime\": {\n    \"max\": \"2024-01-15T10:30:05Z\",\n    \"watermark\": \"2024-01-15T10:20:05Z\"\n  },\n  \"stateOperators\": &#91;{\n    \"numRowsTotal\": 50000,\n    \"numRowsUpdated\": 4821,\n    \"numRowsDroppedByWatermark\": 142\n  }]\n}\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">To access it directly:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>dropped = query.lastProgress&#91;\"stateOperators\"]&#91;0]&#91;\"numRowsDroppedByWatermark\"]\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Interpreting the number:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Zero consistently \u2192 watermark may be unnecessarily large; consider reducing for lower latency<\/li>\n\n\n\n<li>Less than 0.1% of input rows \u2192 healthy, catching only extreme outliers<\/li>\n\n\n\n<li>Growing steadily \u2192 watermark too small for your lag spikes; increase it<\/li>\n\n\n\n<li>Sudden spike \u2192 upstream outage caused a burst of late data; investigate Kafka lag<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Important caveat:<\/strong> numRowsDroppedByWatermark counts rows dropped from state \u2014 Spark received them but the target window was already closed. It does not count events so late they never matched a window at all. True data loss may be slightly higher. Treat it as a floor, not a ceiling.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For continuous monitoring, attach a StreamingQueryListener:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from pyspark.sql.streaming import StreamingQueryListener\n\nclass WatermarkMonitor(StreamingQueryListener):\n    def onQueryProgress(self, event):\n        for op in event.progress.stateOperators:\n            if op.numRowsDroppedByWatermark &gt; 0:\n                pct = op.numRowsDroppedByWatermark \/ op.numRowsTotal * 100\n                print(f\"\u26a0 Dropped: {op.numRowsDroppedByWatermark} rows ({pct:.2f}%)\")\n\nspark.streams.addListener(WatermarkMonitor())\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Saving Late Rows for Later<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">When late data matters for correctness, you don&#8217;t have to accept the loss. Split your stream before the watermark \u2014 route on-time events through the normal aggregation pipeline, and capture late events into a separate Delta table for reprocessing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pattern A \u2014 Quarantine + Merge<\/strong> (best when late arrivals are frequent and near-realtime correction is needed):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>on_time = df.filter(col(\"lag_seconds\") &lt;= 600)\nlate    = df.filter(col(\"lag_seconds\") &gt;  600)\n\n# Main pipeline\non_time \\\n  .withWatermark(\"event_time\", \"10 minutes\") \\\n  .groupBy(window(\"event_time\", \"5 minutes\")).count() \\\n  .writeStream.outputMode(\"append\").format(\"delta\").table(\"metrics_main\")\n\n# Quarantine late rows as-is\nlate \\\n  .writeStream.outputMode(\"append\").format(\"delta\").table(\"metrics_late_arrivals\")\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">A scheduled batch job then re-aggregates and merges them back in:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>late_agg = spark.table(\"metrics_late_arrivals\") \\\n  .groupBy(window(\"event_time\", \"5 minutes\")).count() \\\n  .withColumnRenamed(\"count\", \"late_count\")\n\nDeltaTable.forName(spark, \"metrics_main\") \\\n  .alias(\"m\").merge(late_agg.alias(\"l\"), \"m.window = l.window\") \\\n  .whenMatchedUpdate(set={\"count\": col(\"m.count\") + col(\"l.late_count\")}) \\\n  .whenNotMatchedInsertAll() \\\n  .execute()\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pattern B \u2014 Batch Replay<\/strong> (simpler; use when late data is rare and eventual consistency is acceptable):<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>spark.read.table(\"raw_events\") \\\n  .filter(\"event_time &gt;= now() - INTERVAL 24 HOURS\") \\\n  .withWatermark(\"event_time\", \"4 hours\") \\\n  .groupBy(window(\"event_time\", \"5 minutes\")).count() \\\n  .write.format(\"delta\").mode(\"overwrite\") \\\n  .option(\"replaceWhere\", \"window &gt;= now() - INTERVAL 24 HOURS\") \\\n  .saveAsTable(\"metrics_main\")\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><code>replaceWhere<\/code> only overwrites the affected time partitions \u2014 not the entire table.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">TL;DR<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Windowing<\/strong> groups stream events into time buckets. Use tumbling for simple aggregations, sliding for rolling metrics, session for user activity.<\/li>\n\n\n\n<li><strong>Watermarking<\/strong> tells Spark how long to wait for late events before closing a window and freeing state. Without it, state grows forever.<\/li>\n\n\n\n<li>Set your watermark to 2\u20133\u00d7 your P95 lag, measured from real traffic \u2014 not a guess.<\/li>\n\n\n\n<li>Monitor query.lastProgress[&#8220;stateOperators&#8221;][0][&#8220;numRowsDroppedByWatermark&#8221;] to see how many late events are being discarded each batch.<\/li>\n\n\n\n<li>To recover late rows: split the stream before aggregation, quarantine late events in a Delta table, and merge them back on a schedule.<\/li>\n\n\n\n<li><strong>Always store raw events.<\/strong> Streams are your fast path. A raw Delta table is your source of truth for reprocessing.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Windowing &amp; Watermarking in Databricks Streaming What Is Windowing? In a continuous stream, data never stops arriving. Windowing is the mechanism that groups streaming events into finite, time-bounded buckets so you can compute meaningful aggregations \u2014 counts, averages, sums \u2014 over a defined period rather than over all data since the beginning of time. Spark [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-5203","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Windowing and Watermarking in Databricks Streaming - Rana Nasri Ghazzi<\/title>\n<meta name=\"description\" content=\"Explore Rana Ghazzi&#039;s data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI &amp; Python.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ranaghazzi.com\/?p=5203\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Windowing and Watermarking in Databricks Streaming - Rana Nasri Ghazzi\" \/>\n<meta property=\"og:description\" content=\"Explore Rana Ghazzi&#039;s data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI &amp; Python.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ranaghazzi.com\/?p=5203\" \/>\n<meta property=\"og:site_name\" content=\"Rana Nasri Ghazzi\" \/>\n<meta property=\"article:published_time\" content=\"2026-05-31T21:31:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-05-31T22:14:30+00:00\" \/>\n<meta name=\"author\" content=\"Rana Ghazzi\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Rana Ghazzi\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203\"},\"author\":{\"name\":\"Rana Ghazzi\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"headline\":\"Windowing and Watermarking in Databricks Streaming\",\"datePublished\":\"2026-05-31T21:31:11+00:00\",\"dateModified\":\"2026-05-31T22:14:30+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203\"},\"wordCount\":890,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203\",\"name\":\"Windowing and Watermarking in Databricks Streaming - Rana Nasri Ghazzi\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#website\"},\"datePublished\":\"2026-05-31T21:31:11+00:00\",\"dateModified\":\"2026-05-31T22:14:30+00:00\",\"description\":\"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5203#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/ranaghazzi.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Windowing and Watermarking in Databricks Streaming\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#website\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/\",\"name\":\"Rana Nasri Ghazzi\",\"description\":\"Turning Data into Decisions\",\"publisher\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ranaghazzi.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\",\"name\":\"Rana Ghazzi\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"contentUrl\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"width\":1024,\"height\":1024,\"caption\":\"Rana Ghazzi\"},\"logo\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\"},\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/?author=2\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Windowing and Watermarking in Databricks Streaming - Rana Nasri Ghazzi","description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ranaghazzi.com\/?p=5203","og_locale":"en_US","og_type":"article","og_title":"Windowing and Watermarking in Databricks Streaming - Rana Nasri Ghazzi","og_description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","og_url":"https:\/\/ranaghazzi.com\/?p=5203","og_site_name":"Rana Nasri Ghazzi","article_published_time":"2026-05-31T21:31:11+00:00","article_modified_time":"2026-05-31T22:14:30+00:00","author":"Rana Ghazzi","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Rana Ghazzi","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ranaghazzi.com\/?p=5203#article","isPartOf":{"@id":"https:\/\/ranaghazzi.com\/?p=5203"},"author":{"name":"Rana Ghazzi","@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"headline":"Windowing and Watermarking in Databricks Streaming","datePublished":"2026-05-31T21:31:11+00:00","dateModified":"2026-05-31T22:14:30+00:00","mainEntityOfPage":{"@id":"https:\/\/ranaghazzi.com\/?p=5203"},"wordCount":890,"commentCount":0,"publisher":{"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ranaghazzi.com\/?p=5203#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ranaghazzi.com\/?p=5203","url":"https:\/\/ranaghazzi.com\/?p=5203","name":"Windowing and Watermarking in Databricks Streaming - Rana Nasri Ghazzi","isPartOf":{"@id":"https:\/\/ranaghazzi.com\/#website"},"datePublished":"2026-05-31T21:31:11+00:00","dateModified":"2026-05-31T22:14:30+00:00","description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","breadcrumb":{"@id":"https:\/\/ranaghazzi.com\/?p=5203#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ranaghazzi.com\/?p=5203"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ranaghazzi.com\/?p=5203#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ranaghazzi.com\/"},{"@type":"ListItem","position":2,"name":"Windowing and Watermarking in Databricks Streaming"}]},{"@type":"WebSite","@id":"https:\/\/ranaghazzi.com\/#website","url":"https:\/\/ranaghazzi.com\/","name":"Rana Nasri Ghazzi","description":"Turning Data into Decisions","publisher":{"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ranaghazzi.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc","name":"Rana Ghazzi","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","url":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","contentUrl":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","width":1024,"height":1024,"caption":"Rana Ghazzi"},"logo":{"@id":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png"},"url":"https:\/\/ranaghazzi.com\/?author=2"}]}},"_links":{"self":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/5203","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5203"}],"version-history":[{"count":18,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/5203\/revisions"}],"predecessor-version":[{"id":5231,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/5203\/revisions\/5231"}],"wp:attachment":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5203"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5203"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5203"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}