{"id":5381,"date":"2026-06-10T16:15:23","date_gmt":"2026-06-10T16:15:23","guid":{"rendered":"https:\/\/ranaghazzi.com\/?p=5381"},"modified":"2026-06-13T19:10:54","modified_gmt":"2026-06-13T19:10:54","slug":"comprehensive-guide-data-ingestion-in-databricks-lakeflow-connect-external-ingestion","status":"publish","type":"post","link":"https:\/\/ranaghazzi.com\/?p=5381","title":{"rendered":"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect &#038; External Ingestion"},"content":{"rendered":"<p><style>\n    .light-font-container, .light-font-container p, .light-font-container h2, .light-font-container li {<br \/>\n        font-weight: #FFFFFF !important;<br \/>\n    }<br \/>\n<\/style>\n<\/p>\n<div class=\"light-font-container\" style=\"background-color: #FFFFFF; padding: 40px; border-radius: 15px;\">\n\n\n<h2 class=\"wp-block-heading\">The Six Lanes of Data Ingestion in Databricks: A Practical Guide<\/h2>\n\n\n<div class=\"wp-block-post-date\"><time datetime=\"2026-06-13T19:10:48.334Z\">June 13, 2026<\/time><\/div>\n\n\n<p class=\"wp-block-paragraph\">Every lakehouse project starts with the same unglamorous question: <em>how does the data actually get in?<\/em> Databricks offers a surprisingly rich set of answers, and that&#8217;s both a blessing and a source of confusion. Auto Loader, COPY INTO, Lakeflow Connect, Partner Connect, Kafka connectors, custom Python \u2014 they all &#8220;ingest data,&#8221; but they solve very different problems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This post lays out a mental model I find useful: <strong>six lanes of ingestion<\/strong>, each defined by what your source looks like and how much engineering you want to own. By the end, you should be able to look at any new data source and know which lane it belongs in within seconds.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lane 1: Auto Loader \u2014 for files that keep coming<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Auto Loader (the cloudFiles source) is the workhorse for incremental file ingestion from cloud object storage \u2014 S3, ADLS, or GCS. You point it at a directory, and it detects and processes new files as they arrive, without you ever tracking what&#8217;s already been loaded.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">python<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>df = (spark.readStream\n  .format(\"cloudFiles\")\n  .option(\"cloudFiles.format\", \"json\")\n  .option(\"cloudFiles.schemaLocation\", \"\/checkpoints\/schema\")\n  .load(\"s3:\/\/bucket\/raw\/\"))<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What makes it the default choice for ongoing file feeds<\/strong>: it&#8217;s built on Structured Streaming, so it runs continuously or in cost-efficient triggered batches (Trigger.AvailableNow). Checkpointing gives you exactly-once guarantees \u2014 reruns never duplicate data. It infers and evolves schemas, capturing unexpected fields in a rescued-data column instead of failing. And for extreme volumes, file-notification mode taps cloud events (SQS, Event Grid) to scale to millions of files.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reach for it when:<\/strong> logs, exports, IoT dumps, or vendor drops land in cloud storage on an ongoing basis. It is the standard bronze-layer ingestion tool and slots naturally into Lakeflow Declarative Pipelines.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lane 2: Direct ingestion \u2014 for batch and ad-hoc loads<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Not everything is a pipeline. Sometimes you just need to load some files, and Databricks has built-in paths for exactly that.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>COPY INTO<\/strong> is an idempotent SQL command that loads files into a Delta table and remembers what it has loaded, so reruns skip already-processed files:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">sql<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>COPY INTO my_table\nFROM 's3:\/\/bucket\/raw\/'\nFILEFORMAT = CSV\nFORMAT_OPTIONS ('header' = 'true');<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Alongside it sit the <strong>Add Data UI<\/strong> (drag-and-drop a CSV or Excel file straight into a table) and CTAS \/ read_files() for one-off loads or querying files in place.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reach for it when:<\/strong> ingestion is occasional, batch-oriented, SQL-first, or exploratory. The usual rule of thumb: COPY INTO for thousands of files, Auto Loader once you&#8217;re into high volume or continuous arrival.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lane 3: Lakeflow Connect \u2014 managed connectors for apps and databases<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Lakeflow Connect is Databricks&#8217; fully managed connector service for enterprise applications and databases \u2014 Salesforce, Workday, ServiceNow, Google Analytics, SharePoint, SQL Server (with change-data-capture-based incremental sync), and a growing list of others.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There&#8217;s no pipeline code at all. You pick a source in the UI or API, authenticate through a Unity Catalog connection object, select the tables or objects you want, and set a schedule. Databricks handles the incremental extraction, CDC mechanics, and writes everything into governed Delta tables.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reach for it when:<\/strong> you want SaaS or database data in the lakehouse with minimal engineering effort and a native connector exists. It&#8217;s Databricks&#8217; answer to &#8220;why pay a third party for simple ELT?&#8221;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lane 4: Partner Connect \u2014 when the long tail calls<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No platform covers every source natively. Through Partner Connect, Databricks integrates with third-party ELT vendors \u2014 Fivetran, Qlik, Informatica, Rivery, Hevo, and more \u2014 and automates the plumbing: it provisions the SQL warehouse, service principal, and credentials so the partner tool can write into your lakehouse in a few clicks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The value is connector breadth. A tool like Fivetran has hundreds of pre-built connectors covering long-tail SaaS apps, plus years of accumulated handling for schema drift, API quirks, and CDC edge cases.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reach for it when:<\/strong> your source isn&#8217;t covered by Lakeflow Connect, or your organization already standardizes on one of these tools. The trade-off is extra licensing cost and one more vendor in the stack.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lane 5: Streaming event buses \u2014 Kafka, Kinesis, and friends<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">When data arrives as individual events rather than files, you skip object storage entirely. Spark has native Structured Streaming connectors for Kafka, Kinesis, Azure Event Hubs, Pulsar, and Pub\/Sub, and you read the stream directly into Delta:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">python<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>df = (spark.readStream\n  .format(\"kafka\")\n  .option(\"kafka.bootstrap.servers\", \"broker:9092\")\n  .option(\"subscribe\", \"orders\")\n  .load())<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Records arrive with the payload as binary plus metadata (key, partition, offset, timestamp); the first transformation is almost always from_json on the value column. Checkpointing tracks offsets, giving effectively exactly-once delivery into Delta even across restarts.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Two architectures are common here. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The direct path (bus \u2192 Structured Streaming \u2192 bronze) gives the lowest latency but ties you to the bus&#8217;s retention window. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The staged path (bus \u2192 Firehose or sink connector \u2192 cloud storage \u2192 Auto Loader) accepts a few minutes of delay in exchange for a durable raw archive and resilience to pipeline downtime. Many teams happily take that trade.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reach for it when:<\/strong> clickstreams, telemetry, fraud signals, or CDC event feeds (think Debezium publishing database changes to Kafka) need seconds-level freshness.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lane 6: Custom Python \u2014 the escape hatch<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">When nothing pre-built fits, you write it yourself \u2014 and there&#8217;s a clear maturity ladder.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Tier one<\/strong> is plain Python in a scheduled job: requests or a vendor SDK pulls from an API, the results become a DataFrame, and you write to Delta. You own everything the managed tools normally handle \u2014 secrets in secret scopes, watermark tables for incrementality, MERGE INTO for idempotency, retries and rate limiting.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Tier two<\/strong> is Spark&#8217;s JDBC reader for databases without a managed connector, with partitionColumn options to parallelize the read and watermark subqueries for incremental pulls.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Tier three<\/strong> is the PySpark <strong>Python Data Source API<\/strong> (DBR 15.3+ \/ Spark 4), which lets you package ingestion logic as a first-class Spark format: implement a DataSource and reader class, register it, and then spark.read.format(&#8220;myapi&#8221;) works everywhere \u2014 batch, streaming with proper offset tracking, even inside Lakeflow Declarative Pipelines. You are, in effect, building your own mini-connector with the same semantics the Kafka connector has.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reach for it when:<\/strong> the source is a proprietary internal API, an obscure SaaS, SFTP drops, mainframe extracts \u2014 anything without an existing connector. The common path is to prototype in a notebook, harden into a watermarked job, and eventually refactor into a custom data source (or retire it the day an official connector ships).<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The decision flow<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Strip away the product names and the choice comes down to what your source looks like:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Your source is&#8230;<\/th><th>Your lane<\/th><\/tr><\/thead><tbody><tr><td>Files arriving continuously in cloud storage<\/td><td><strong>1 \u2014 Auto Loader<\/strong><\/td><\/tr><tr><td>Files loaded occasionally or ad hoc<\/td><td><strong>2 \u2014 COPY INTO \/ UI \/ CTAS<\/strong><\/td><\/tr><tr><td>A SaaS app or database with a native connector<\/td><td><strong>3 \u2014 Lakeflow Connect<\/strong><\/td><\/tr><tr><td>A source only a third-party tool covers<\/td><td><strong>4 \u2014 Partner Connect<\/strong><\/td><\/tr><tr><td>A real-time event stream<\/td><td><strong>5 \u2014 Kafka \/ Kinesis via Structured Streaming<\/strong><\/td><\/tr><tr><td>Anything else<\/td><td><strong>6 \u2014 Custom Python<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">One destination, six roads<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Here&#8217;s the part that keeps the architecture sane: no matter which lane you take, everything converges on the same landing zone \u2014 Delta tables in Unity Catalog, typically a bronze layer. From there, Lakeflow Declarative Pipelines or jobs take over for the silver and gold transformations, and the downstream architecture is identical regardless of how the data arrived.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The ingestion lane only decides <em>how data gets to bronze<\/em>. Choose the lane that matches your source&#8217;s shape and your team&#8217;s appetite for owning pipeline code, and let the lakehouse handle the rest. Most real organizations end up using three or four lanes at once \u2014 Lakeflow Connect for Salesforce, Auto Loader for log files, Kafka for clickstream, a custom Python job for that one internal API \u2014 and that&#8217;s not a smell. It&#8217;s the design working as intended.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The Six Lanes of Data Ingestion in Databricks: A Practical Guide Every lakehouse project starts with the same unglamorous question: how does the data actually get in? Databricks offers a surprisingly rich set of answers, and that&#8217;s both a blessing and a source of confusion. Auto Loader, COPY INTO, Lakeflow Connect, Partner Connect, Kafka connectors, [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-5381","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect &amp; External Ingestion - Rana Nasri Ghazzi<\/title>\n<meta name=\"description\" content=\"Explore Rana Ghazzi&#039;s data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI &amp; Python.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ranaghazzi.com\/?p=5381\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect &amp; External Ingestion - Rana Nasri Ghazzi\" \/>\n<meta property=\"og:description\" content=\"Explore Rana Ghazzi&#039;s data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI &amp; Python.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ranaghazzi.com\/?p=5381\" \/>\n<meta property=\"og:site_name\" content=\"Rana Nasri Ghazzi\" \/>\n<meta property=\"article:published_time\" content=\"2026-06-10T16:15:23+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-06-13T19:10:54+00:00\" \/>\n<meta name=\"author\" content=\"Rana Ghazzi\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Rana Ghazzi\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381\"},\"author\":{\"name\":\"Rana Ghazzi\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"headline\":\"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect &#038; External Ingestion\",\"datePublished\":\"2026-06-10T16:15:23+00:00\",\"dateModified\":\"2026-06-13T19:10:54+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381\"},\"wordCount\":1253,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381\",\"name\":\"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect & External Ingestion - Rana Nasri Ghazzi\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#website\"},\"datePublished\":\"2026-06-10T16:15:23+00:00\",\"dateModified\":\"2026-06-13T19:10:54+00:00\",\"description\":\"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=5381#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/ranaghazzi.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect &#038; External Ingestion\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#website\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/\",\"name\":\"Rana Nasri Ghazzi\",\"description\":\"Turning Data into Decisions\",\"publisher\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ranaghazzi.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\",\"name\":\"Rana Ghazzi\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"contentUrl\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"width\":1024,\"height\":1024,\"caption\":\"Rana Ghazzi\"},\"logo\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\"},\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/?author=2\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect & External Ingestion - Rana Nasri Ghazzi","description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ranaghazzi.com\/?p=5381","og_locale":"en_US","og_type":"article","og_title":"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect & External Ingestion - Rana Nasri Ghazzi","og_description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","og_url":"https:\/\/ranaghazzi.com\/?p=5381","og_site_name":"Rana Nasri Ghazzi","article_published_time":"2026-06-10T16:15:23+00:00","article_modified_time":"2026-06-13T19:10:54+00:00","author":"Rana Ghazzi","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Rana Ghazzi","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ranaghazzi.com\/?p=5381#article","isPartOf":{"@id":"https:\/\/ranaghazzi.com\/?p=5381"},"author":{"name":"Rana Ghazzi","@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"headline":"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect &#038; External Ingestion","datePublished":"2026-06-10T16:15:23+00:00","dateModified":"2026-06-13T19:10:54+00:00","mainEntityOfPage":{"@id":"https:\/\/ranaghazzi.com\/?p=5381"},"wordCount":1253,"commentCount":0,"publisher":{"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ranaghazzi.com\/?p=5381#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ranaghazzi.com\/?p=5381","url":"https:\/\/ranaghazzi.com\/?p=5381","name":"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect & External Ingestion - Rana Nasri Ghazzi","isPartOf":{"@id":"https:\/\/ranaghazzi.com\/#website"},"datePublished":"2026-06-10T16:15:23+00:00","dateModified":"2026-06-13T19:10:54+00:00","description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","breadcrumb":{"@id":"https:\/\/ranaghazzi.com\/?p=5381#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ranaghazzi.com\/?p=5381"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ranaghazzi.com\/?p=5381#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ranaghazzi.com\/"},{"@type":"ListItem","position":2,"name":"Comprehensive Guide: Data Ingestion in Databricks \u2014 Lakeflow Connect &#038; External Ingestion"}]},{"@type":"WebSite","@id":"https:\/\/ranaghazzi.com\/#website","url":"https:\/\/ranaghazzi.com\/","name":"Rana Nasri Ghazzi","description":"Turning Data into Decisions","publisher":{"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ranaghazzi.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc","name":"Rana Ghazzi","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","url":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","contentUrl":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","width":1024,"height":1024,"caption":"Rana Ghazzi"},"logo":{"@id":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png"},"url":"https:\/\/ranaghazzi.com\/?author=2"}]}},"_links":{"self":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/5381","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5381"}],"version-history":[{"count":14,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/5381\/revisions"}],"predecessor-version":[{"id":5495,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/5381\/revisions\/5495"}],"wp:attachment":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5381"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5381"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5381"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}