{"id":4999,"date":"2026-05-25T15:53:36","date_gmt":"2026-05-25T15:53:36","guid":{"rendered":"https:\/\/ranaghazzi.com\/?p=4999"},"modified":"2026-10-03T15:12:01","modified_gmt":"2026-10-03T15:12:01","slug":"ci-cd-on-databricks","status":"publish","type":"post","link":"https:\/\/ranaghazzi.com\/?p=4999","title":{"rendered":"CI\/CD on Databricks"},"content":{"rendered":"<p><style>\n    .light-font-container, .light-font-container p, .light-font-container h2, .light-font-container li {<br \/>\n        font-weight: #FFFFFF !important;<br \/>\n    }<br \/>\n<\/style>\n<\/p>\n<div class=\"light-font-container\" style=\"background-color: #FFFFFF ; padding: 40px; border-radius: 15px;\">\n\n\n<h2 class=\"wp-block-heading\">CI\/CD on Databricks: Declarative Automation Bundles (DABs)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">CI\/CD on Databricks centers on automating the testing and deployment of data pipelines, notebooks, and jobs across environments using <strong>Declarative Automation Bundles (DABs)<\/strong> \u2014 formerly known as Databricks Asset Bundles \u2014 as the primary infrastructure-as-code tool, with everything version-controlled in Git. The rename (March 2026) reflects that bundles now cover full declarative automation workflows, not just &#8220;assets&#8221;; it&#8217;s non-breaking, so existing databricks.yml files and CLI commands keep working as-is.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>CI<\/strong> covers linting, unit testing (pytest or Nutter), and bundle validation on every code push.<\/li>\n\n\n\n<li><strong>CD<\/strong> automates promotion from dev \u2192 staging \u2192 prod using platform tools like GitHub Actions or Azure DevOps.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Credentials are managed through service principals and Databricks Secrets \u2014 never hardcoded \u2014 and Unity Catalog provides clean environment separation at the data layer.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Key best practices<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Idempotent jobs<\/li>\n\n\n\n<li>Separate workspaces per environment<\/li>\n\n\n\n<li>All deploys run through the pipeline (never from a local machine)<\/li>\n\n\n\n<li>Post-deploy observability via job run history, Delta table history, and external monitoring tools<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The recommended modern stack combines <strong>GitHub Actions + DABs + Unity Catalog + MLflow<\/strong> for end-to-end automated delivery of Databricks workloads.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Declarative Automation Bundles (DABs)<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The recommended modern approach \u2014 replaces legacy databricks-cli deploys.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Version-controlled as YAML in your repo<\/li>\n\n\n\n<li>Define everything in databricks.yml<\/li>\n\n\n\n<li>Supports multi-environment targets (dev, staging, prod)<\/li>\n\n\n\n<li>Deploy: databricks bundle deploy &#8211;target prod<\/li>\n\n\n\n<li>Run: databricks bundle run &lt;job-name><\/li>\n\n\n\n<li>Supports: Jobs, DLT\/Lakeflow pipelines, notebooks, Python wheels, Model Serving endpoints<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What&#8217;s new:<\/strong> the default bundle templates (e.g. default-python) now follow Spark Declarative Pipelines on Lakeflow conventions \u2014 defining pipelines as plain source files plus pyproject.toml, rather than notebooks. This pushes bundle-based projects further toward standard software engineering practice (testable modules, dependency management) and away from notebook-driven pipeline code.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\">Sources:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/docs.databricks.com\/aws\/en\/release-notes\/dev-tools\/bundles\" target=\"_blank\" rel=\"noopener\">Declarative Automation Bundles feature release notes<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/rebricked.org\/databricks\/declarative-automation-bundles\/\" target=\"_blank\" rel=\"noopener\">Declarative Automation Bundles (Databricks) &#8211; formerly Databricks Asset Bundles<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/docs.databricks.com\/aws\/en\/dev-tools\/bundles\/faqs\" target=\"_blank\" rel=\"noopener\">Declarative Automation Bundles FAQs<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>CI\/CD on Databricks: Declarative Automation Bundles (DABs) CI\/CD on Databricks centers on automating the testing and deployment of data pipelines, notebooks, and jobs across environments using Declarative Automation Bundles (DABs) \u2014 formerly known as Databricks Asset Bundles \u2014 as the primary infrastructure-as-code tool, with everything version-controlled in Git. The rename (March 2026) reflects that bundles [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-4999","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>CI\/CD on Databricks - Rana Nasri Ghazzi<\/title>\n<meta name=\"description\" content=\"Explore Rana Ghazzi&#039;s data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI &amp; Python.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ranaghazzi.com\/?p=4999\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"CI\/CD on Databricks - Rana Nasri Ghazzi\" \/>\n<meta property=\"og:description\" content=\"Explore Rana Ghazzi&#039;s data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI &amp; Python.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ranaghazzi.com\/?p=4999\" \/>\n<meta property=\"og:site_name\" content=\"Rana Nasri Ghazzi\" \/>\n<meta property=\"article:published_time\" content=\"2026-05-25T15:53:36+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-10-03T15:12:01+00:00\" \/>\n<meta name=\"author\" content=\"Rana Ghazzi\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Rana Ghazzi\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999\"},\"author\":{\"name\":\"Rana Ghazzi\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"headline\":\"CI\\\/CD on Databricks\",\"datePublished\":\"2026-05-25T15:53:36+00:00\",\"dateModified\":\"2026-10-03T15:12:01+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999\"},\"wordCount\":307,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999\",\"name\":\"CI\\\/CD on Databricks - Rana Nasri Ghazzi\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#website\"},\"datePublished\":\"2026-05-25T15:53:36+00:00\",\"dateModified\":\"2026-10-03T15:12:01+00:00\",\"description\":\"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/?p=4999#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/ranaghazzi.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"CI\\\/CD on Databricks\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#website\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/\",\"name\":\"Rana Nasri Ghazzi\",\"description\":\"Turning Data into Decisions\",\"publisher\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ranaghazzi.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/#\\\/schema\\\/person\\\/d8ee34f53cb0df9faaf816fb5363a4cc\",\"name\":\"Rana Ghazzi\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"contentUrl\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\",\"width\":1024,\"height\":1024,\"caption\":\"Rana Ghazzi\"},\"logo\":{\"@id\":\"https:\\\/\\\/ranaghazzi.com\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/logo.png\"},\"url\":\"https:\\\/\\\/ranaghazzi.com\\\/?author=2\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"CI\/CD on Databricks - Rana Nasri Ghazzi","description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ranaghazzi.com\/?p=4999","og_locale":"en_US","og_type":"article","og_title":"CI\/CD on Databricks - Rana Nasri Ghazzi","og_description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","og_url":"https:\/\/ranaghazzi.com\/?p=4999","og_site_name":"Rana Nasri Ghazzi","article_published_time":"2026-05-25T15:53:36+00:00","article_modified_time":"2026-10-03T15:12:01+00:00","author":"Rana Ghazzi","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Rana Ghazzi","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ranaghazzi.com\/?p=4999#article","isPartOf":{"@id":"https:\/\/ranaghazzi.com\/?p=4999"},"author":{"name":"Rana Ghazzi","@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"headline":"CI\/CD on Databricks","datePublished":"2026-05-25T15:53:36+00:00","dateModified":"2026-10-03T15:12:01+00:00","mainEntityOfPage":{"@id":"https:\/\/ranaghazzi.com\/?p=4999"},"wordCount":307,"commentCount":0,"publisher":{"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ranaghazzi.com\/?p=4999#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ranaghazzi.com\/?p=4999","url":"https:\/\/ranaghazzi.com\/?p=4999","name":"CI\/CD on Databricks - Rana Nasri Ghazzi","isPartOf":{"@id":"https:\/\/ranaghazzi.com\/#website"},"datePublished":"2026-05-25T15:53:36+00:00","dateModified":"2026-10-03T15:12:01+00:00","description":"Explore Rana Ghazzi's data analytics portfolio \u2014 dashboards, visualizations, and insights built with Tableau, Power BI & Python.","breadcrumb":{"@id":"https:\/\/ranaghazzi.com\/?p=4999#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ranaghazzi.com\/?p=4999"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/ranaghazzi.com\/?p=4999#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ranaghazzi.com\/"},{"@type":"ListItem","position":2,"name":"CI\/CD on Databricks"}]},{"@type":"WebSite","@id":"https:\/\/ranaghazzi.com\/#website","url":"https:\/\/ranaghazzi.com\/","name":"Rana Nasri Ghazzi","description":"Turning Data into Decisions","publisher":{"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ranaghazzi.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/ranaghazzi.com\/#\/schema\/person\/d8ee34f53cb0df9faaf816fb5363a4cc","name":"Rana Ghazzi","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","url":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","contentUrl":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png","width":1024,"height":1024,"caption":"Rana Ghazzi"},"logo":{"@id":"https:\/\/ranaghazzi.com\/wp-content\/uploads\/2025\/11\/logo.png"},"url":"https:\/\/ranaghazzi.com\/?author=2"}]}},"_links":{"self":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/4999","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4999"}],"version-history":[{"count":23,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/4999\/revisions"}],"predecessor-version":[{"id":5772,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=\/wp\/v2\/posts\/4999\/revisions\/5772"}],"wp:attachment":[{"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4999"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4999"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ranaghazzi.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4999"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}