Databricks Asset Bundles resources

Databricks Asset Bundles allows you to specify information about the Azure Databricks resources used by the bundle in the resources mapping in the bundle configuration. See resources reference.

This page provides configuration reference for all supported resource types for bundles and provides details and an example for each supported type. For additional examples, see Bundle configuration examples.

The JSON schema for bundles that is used to validate YAML configuration is in the Databricks CLI GitHub repository.

Supported resources

The following table lists supported resource types for bundles (YAML and Python, where applicable). Some resources can be created by defining them in a bundle and deploying the bundle, and some resources can only be created by referencing an existing asset to include in the bundle.

Resource configuration defines a Databricks object that corresponds to a Databricks REST API object. The REST API object's supported create request fields, expressed as YAML, are the resource's supported keys. Links to documentation for each resource's corresponding object are in the table below.

alert

Type: Map

The alert resource defines a SQL alert (v2).

Added in Databricks CLI version 0.279.0

alerts:
  <alert-name>:
    <alert-field-name>: <alert-field-value>

alert.evaluation

Type: Map

The evaluation configuration for the alert.

alert.evaluation.notification

Type: Map

The user or other destination to notify when the alert is triggered.

alert.evaluation.notification.subscriptions

Type: Sequence

An unordered list of notification subscriptions.

Each item in the list is an AlertSubscription:

alert.evaluation.source

Type: Map

Source column from result to use to evaluate the alert.

alert.evaluation.threshold

Type: Map

Threshold to use for alert evaluation, can be a column or a value.

alert.evaluation.threshold.value

Type: Map

Literal value to use as the threshold. Specify one of the following value types.

alert.schedule

Type: Map

The schedule configuration for the alert.

Examples

The following example configuration defines an alert with a simple evaluation:

resources:
  alerts:
    my_alert:
      display_name: my_alert
      evaluation:
        comparison_operator: EQUAL
        source:
          name: '1'
        threshold:
          value:
            double_value: 2
      query_text: select 2
      schedule:
        quartz_cron_schedule: '44 19 */1 * * ?'
        timezone_id: Europe/Amsterdam
      warehouse_id: 799f096837fzzzz4

The following example configuration defines an alert with permissions that evaluates using aggregation and sends notifications:

resources:
  alerts:
    my_alert:
      permissions:
        - level: CAN_MANAGE
          user_name: someone@example.com
      custom_summary: 'My alert'
      display_name: 'My alert'
      evaluation:
        comparison_operator: 'EQUAL'
        notification:
          notify_on_ok: false
          retrigger_seconds: 1
        source:
          aggregation: 'MAX'
          display: '1'
          name: '1'
        threshold:
          value:
            double_value: 2
      query_text: 'select 2'
      schedule:
        pause_status: 'UNPAUSED'
        quartz_cron_schedule: '44 19 */1 * * ?'
        timezone_id: 'Europe/Amsterdam'
      warehouse_id: 799f096837fzzzz4

app

Type: Map

The app resource defines a Databricks app. For information about Databricks Apps, see Databricks Apps.

To add an app, specify the settings to define the app, including the required source_code_path.

databricks bundle init https://github.com/databricks/bundle-examples --template-dir contrib/templates/streamlit-app

Added in Databricks CLI version 0.239.0

apps:
  <app-name>:
    <app-field-name>: <app-field-value>

app.resources

Type: Sequence

A list of compute resources for the app.

Each item in the list is an AppResource:

app.resources.database

Type: Map

The settings that identify the Lakebase database to use.

app.resources.genie_space

Type: Map

The settings that identify the Genie space to use.

app.resources.job

Type: Map

The settings that identify the job resource to use.

app.resources.secret

Type: Map

The settings that identify the Azure Databricks secret resource to use.

app.resources.serving_endpoint

Type: Map

The settings that identify the model serving endpoint resource to use.

app.resources.sql_warehouse

Type: Map

The settings that identify the SQL warehouse to use.

app.resources.uc_securable

Type: Map

The settings that identify the Unity Catalog volume to use.

Example

The following example creates an app named my_app that manages a job created by the bundle:

resources:
  jobs:
    # Define a job in the bundle
    hello_world:
      name: hello_world
      tasks:
        - task_key: task
          spark_python_task:
            python_file: ../src/main.py
          environment_key: default

      environments:
        - environment_key: default
          spec:
            environment_version: '2'

  # Define an app that manages the job in the bundle
  apps:
    job_manager:
      name: 'job_manager_app'
      description: 'An app which manages a job created by this bundle'

      # The location of the source code for the app
      source_code_path: ../src/app

      # The resources in the bundle which this app has access to. This binds the resource in the app with the bundle resource.
      resources:
        - name: 'app-job'
          job:
            id: ${resources.jobs.hello_world.id}
            permission: 'CAN_MANAGE_RUN'

The corresponding app.yaml defines the configuration for running the app:

command:
  - flask
  - --app
  - app
  - run
  - --debug
env:
  - name: JOB_ID
    valueFrom: 'app-job'

For the complete Databricks app example bundle, see the bundle-examples GitHub repository.

cluster

Type: Map

The cluster resource defines a cluster.

Added in Databricks CLI version 0.229.0

clusters:
  <cluster-name>:
    <cluster-field-name>: <cluster-field-value>

cluster.autoscale

Type: Map

Parameters for automatically scaling clusters up and down based on load.

cluster.aws_attributes

Type: Map

Attributes related to clusters running on Amazon Web Services.

cluster.azure_attributes

Type: Map

Attributes related to clusters running on Microsoft Azure.

cluster.azure_attributes.log_analytics_info

Type: Map

The configuration for Azure Log Analytics agent.

cluster.gcp_attributes

Type: Map

Attributes related to clusters running on Google Cloud Platform.

cluster.cluster_log_conf

The configuration for delivering Spark logs to a long-term storage destination.

cluster.cluster_log_conf.dbfs

Type: Map

DBFS location for cluster log delivery.

cluster.cluster_log_conf.s3

Type: Map

S3 location for cluster log delivery.

cluster.cluster_log_conf.volumes

Type: Map

Volumes location for cluster log delivery.

cluster.docker_image

Type: Map

The custom Docker image configuration.

cluster.docker_image.basic_auth

Type: Map

Basic authentication for Docker repository.

cluster.init_scripts

Type: Map

The configuration for storing init scripts. At least one location type must be specified.

cluster.init_scripts.dbfs

Type: Map

DBFS location of init script.

cluster.init_scripts.workspace

Type: Map

Workspace location of init script.

cluster.init_scripts.s3

Type: Map

S3 location of init script.

cluster.init_scripts.abfss

Type: Map

ABFSS location of init script.

cluster.init_scripts.gcs

Type: Map

GCS location of init script.

cluster.init_scripts.volumes

Type: Map

Volumes location of init script.

cluster.workload_type

Type: Map

Cluster attributes showing cluster workload types.

cluster.workload_type.clients

Type: Map

The type of clients for this compute workload.

Examples

The following example creates a dedicated (single-user) cluster for the current user with Databricks Runtime 15.4 LTS and a cluster policy:

resources:
  clusters:
    my_cluster:
      num_workers: 0
      node_type_id: 'i3.xlarge'
      driver_node_type_id: 'i3.xlarge'
      spark_version: '15.4.x-scala2.12'
      spark_conf:
        'spark.executor.memory': '2g'
      autotermination_minutes: 60
      enable_elastic_disk: true
      single_user_name: ${workspace.current_user.userName}
      policy_id: '000128DB309672CA'
      enable_local_disk_encryption: false
      data_security_mode: SINGLE_USER
      runtime_engine: STANDARD

This example creates a simple cluster my_cluster and sets that as the cluster to use to run the notebook in my_job:

bundle:
  name: clusters

resources:
  clusters:
    my_cluster:
      num_workers: 2
      node_type_id: 'i3.xlarge'
      autoscale:
        min_workers: 2
        max_workers: 7
      spark_version: '13.3.x-scala2.12'
      spark_conf:
        'spark.executor.memory': '2g'

  jobs:
    my_job:
      tasks:
        - task_key: test_task
          notebook_task:
            notebook_path: './src/my_notebook.py'
          existing_cluster_id: ${resources.clusters.my_cluster.id}

dashboard

Type: Map

The dashboard resource allows you to manage AI/BI dashboards in a bundle. For information about AI/BI dashboards, see Dashboards.

If you deployed a bundle that contains a dashboard from your local environment and then use the UI to modify that dashboard, modifications made through the UI are not applied to the dashboard JSON file in the local bundle unless you explicitly update it using bundle generate. You can use the --watch option to continuously poll and retrieve changes to the dashboard. See databricks bundle generate.

In addition, if you attempt to deploy a bundle from your local environment that contains a dashboard JSON file that is different than the one in the remote workspace, an error will occur. To force the deploy and overwrite the dashboard in the remote workspace with the local one, use the --force option. See databricks bundle deploy.

Added in Databricks CLI version 0.232.0

sync:
  exclude:
    - src/*.lvdash.json

dashboards:
  <dashboard-name>:
    <dashboard-field-name>: <dashboard-field-value>

Example

The following example includes and deploys the sample NYC Taxi Trip Analysis dashboard to the Databricks workspace.

resources:
  dashboards:
    nyc_taxi_trip_analysis:
      display_name: 'NYC Taxi Trip Analysis'
      file_path: ../src/nyc_taxi_trip_analysis.lvdash.json
      warehouse_id: ${var.warehouse_id}

database_catalog

Type: Map

The database catalog resource allows you to define database catalogs that correspond to database instances in a bundle. A database catalog is a Lakebase database that is registered as a Unity Catalog catalog.

For information about database catalogs, see Create a catalog.

Added in Databricks CLI version 0.265.0

database_catalogs:
  <database_catalog-name>:
    <database_catalog-field-name>: <database_catalog-field-value>

Example

The following example defines a database instance with a corresponding database catalog:

resources:
  database_instances:
    my_instance:
      name: my-instance
      capacity: CU_1
  database_catalogs:
    my_catalog:
      database_instance_name: ${resources.database_instances.my_instance.name}
      name: example_catalog
      database_name: my_database
      create_database_if_not_exists: true

database_instance

Type: Map

The database instance resource allows you to define database instances in a bundle. A Lakebase database instance manages storage and compute resources and provides the endpoints that users connect to.

For information about database instances, see What is a database instance?.

Added in Databricks CLI version 0.265.0

database_instances:
  <database_instance-name>:
    <database_instance-field-name>: <database_instance-field-value>

database_instance.parent_instance_ref

Type: Map

The reference of the parent instance. This is only available if the instance is child instance.

Example

The following example defines a database instance with a corresponding database catalog:

resources:
  database_instances:
    my_instance:
      name: my-instance
      capacity: CU_1
  database_catalogs:
    my_catalog:
      database_instance_name: ${resources.database_instances.my_instance.name}
      name: example_catalog
      database_name: my_database
      create_database_if_not_exists: true

For an example bundle that demonstrates how to define a database instance and corresponding database catalog, see the bundle-examples GitHub repository.

experiment

Type: Map

The experiment resource allows you to define MLflow experiments in a bundle. For information about MLflow experiments, see Organize training runs with MLflow experiments.

Added in Databricks CLI version 0.229.0

experiments:
  <experiment-name>:
    <experiment-field-name>: <experiment-field-value>

Example

The following example defines an experiment that all users can view:

resources:
  experiments:
    experiment:
      name: /Workspace/Users/someone@example.com/my_experiment
      permissions:
        - level: CAN_READ
          group_name: users
      description: MLflow experiment used to track runs

job

Type: Map

Jobs are supported in Python for Databricks Asset Bundles. See databricks.bundles.jobs.

The job resource allows you to define jobs and their corresponding tasks in your bundle.

For information about jobs, see Lakeflow Jobs. For a tutorial that uses a Databricks Asset Bundles template to create a job, see Develop a job with Databricks Asset Bundles.

Added in Databricks CLI version 0.229.0

jobs:
  <job-name>:
    <job-field-name>: <job-field-value>

job.continuous

Type: Map

Configuration for continuous job execution.

job.deployment

Type: Map

Deployment information for jobs managed by external sources.

job.email_notifications

Type: Map

Email notification settings for job runs.

job.environments

Type: Sequence

A list of task execution environment specifications that can be referenced by serverless tasks of a job.

Each item in the list is a JobEnvironment:

job.environments.spec

Type: Map

The entity that represents a serverless environment.

job.git_source

Type: Map

Git repository configuration for job source code.

job.git_source.git_snapshot

Type: Map

Read-only commit information snapshot.

job.health

Type: Map

Health monitoring configuration for the job.

job.health.rules

Type: Sequence

A list of job health rules.

Each item in the list is a JobHealthRule:

job.job_clusters

Type: Sequence

A list of job cluster specifications that can be shared and reused by tasks of this job. Libraries cannot be declared in a shared job cluster. You must declare dependent libraries in task settings.

Each item in the list is a JobCluster:

job.notification_settings

Type: Map

Notification settings that apply to all notifications for the job.

job.queue

Type: Map

Queue settings for the job.

job.schedule

Type: Map

Schedule configuration for periodic job execution.

job.trigger

Type: Map

Trigger configuration for event-driven job execution.

job.trigger.file_arrival

Type: Map

Trigger configuration based on file arrival.

job.trigger.table

Type: Map

Trigger configuration based on a table.

job.trigger.table_update

Type: Map

Trigger configuration based on table updates.

job.trigger.periodic

Type: Map

Periodic trigger configuration.

job.webhook_notifications

Type: Map

Webhook notification settings for job runs.

Examples

The following example defines a job with the resource key hello-job with one notebook task:

resources:
  jobs:
    hello-job:
      name: hello-job
      tasks:
        - task_key: hello-task
          notebook_task:
            notebook_path: ./hello.py

The following example defines a job with a SQL notebook:

resources:
  jobs:
    job_with_sql_notebook:
      name: 'Job to demonstrate using a SQL notebook with a SQL warehouse'
      tasks:
        - task_key: notebook
          notebook_task:
            notebook_path: ./select.sql
            warehouse_id: 799f096837fzzzz4

For additional job configuration examples, see Job configuration.

For information about defining job tasks and overriding job settings, see:

• Add tasks to jobs in Databricks Asset Bundles
• Job task settings override

model (legacy)

Type: Map

The model resource allows you to define legacy models in bundles. Databricks recommends you use Unity Catalog registered models instead.

Added in Databricks CLI version 0.229.0

model_serving_endpoint

Type: Map

The model_serving_endpoint resource allows you to define model serving endpoints. See Manage model serving endpoints.

Added in Databricks CLI version 0.229.0

model_serving_endpoints:
  <model_serving_endpoint-name>:
    <model_serving_endpoint-field-name>: <model_serving_endpoint-field-value>

model_serving_endpoint.email_notifications

Type: Map

Email notifications configuration for the serving endpoint.

model_serving_endpoint.ai_gateway

Type: Map

AI Gateway configuration for the serving endpoint.

model_serving_endpoint.ai_gateway.fallback_config

Type: Map

Configuration for traffic fallback which auto fallbacks to other served entities if a request fails with certain error codes.

model_serving_endpoint.ai_gateway.guardrails

Type: Map

The AI gateway guardrails configuration.

model_serving_endpoint.ai_gateway.inference_table_config

Type: Map

Configuration for inference logging to Unity Catalog tables.

model_serving_endpoint.ai_gateway.usage_tracking_config

Type: Map

The AI gateway configuration for tracking usage.

model_serving_endpoint.config

Type: Map

The core configuration of the serving endpoint.

model_serving_endpoint.config.traffic_config

Type: Map

The traffic config defining how invocations to the serving endpoint should be routed.

model_serving_endpoint.config.auto_capture_config

Type: Map

Configuration for Inference Tables which automatically logs requests and responses to Unity Catalog.

Example

The following example defines a Unity Catalog model serving endpoint:

resources:
  model_serving_endpoints:
    uc_model_serving_endpoint:
      name: 'uc-model-endpoint'
      config:
        served_entities:
          - entity_name: 'myCatalog.mySchema.my-ads-model'
            entity_version: '10'
            workload_size: 'Small'
            scale_to_zero_enabled: 'true'
        traffic_config:
          routes:
            - served_model_name: 'my-ads-model-10'
              traffic_percentage: '100'
      tags:
        - key: 'team'
          value: 'data science'

pipeline

Type: Map

Pipelines are supported in Python for Databricks Asset Bundles. See databricks.bundles.pipelines.

The pipeline resource allows you to create pipelines. For information about pipelines, see Lakeflow Spark Declarative Pipelines. For a tutorial that uses the Databricks Asset Bundles template to create a pipeline, see Develop Lakeflow Spark Declarative Pipelines with Databricks Asset Bundles.

Added in Databricks CLI version 0.229.0

pipelines:
  <pipeline-name>:
    <pipeline-field-name>: <pipeline-field-value>

pipeline.deployment

Type: Map

Deployment type configuration for the pipeline.

pipeline.environment

Type: Map

Environment specification for installing dependencies on serverless compute.

pipeline.event_log

Type: Map

Event log configuration for the pipeline.

pipeline.filters

Type: Map

Filters that determine which pipeline packages to include in the deployed graph.

pipeline.ingestion_definition

Type: Map

Configuration for a managed ingestion pipeline. These settings cannot be used with the libraries, schema, target, or catalog settings.

SchemaSpec

Type: Map

Schema object specification for ingesting all tables from a schema.

TableSpec

Type: Map

Table object specification for ingesting a specific table.

ReportSpec

Type: Map

Report object specification for ingesting analytics reports.

pipeline.ingestion_definition.source_configurations

Type: Map

Configuration for source.

pipeline.ingestion_definition.source_configuration.catalog

Type: Map

Catalog-level source configuration parameters

pipeline.ingestion_definition.table_configuration

Type: Map

Configuration options for ingestion tables.

pipeline.libraries

Type: Sequence

Defines the list of libraries or code needed by this pipeline.

Each item in the list is a definition:

pipeline.libraries.file

Type: Map

The path to a file that defines a pipeline and is stored in the Databricks Repos.

pipeline.libraries.glob

Type: Map

The unified field to include source code. Each entry can be a notebook path, a file path, or a folder path that ends /**. This field cannot be used together with notebook or file.

pipeline.libraries.notebook

Type: Map

The path to a notebook that defines a pipeline and is stored in the Databricks workspace.

pipeline.notifications

Type: Sequence

The notification settings for this pipeline. Each item in the sequence is a notification configuration.

Example

The following example defines a pipeline with the resource key hello-pipeline:

resources:
  pipelines:
    hello-pipeline:
      name: hello-pipeline
      clusters:
        - label: default
          num_workers: 1
      development: true
      continuous: false
      channel: CURRENT
      edition: CORE
      photon: false
      libraries:
        - notebook:
            path: ./pipeline.py

For additional pipeline configuration examples, see Pipeline configuration.

postgres_branch

Type:Map

The Postgres branch resource allows you to define Lakebase branches in a bundle. You must also define corresponding Postgres projects and compute endpoints.

Added in Databricks CLI version 0.287.0

postgres_branches:
  <postgres_branch-name>:
    <postgres_branch-field-name>: <postgres_branches-field-value>

Example

See postgres_projects example.

postgres_endpoint

Type: Map

The postgres_endpoints resource allows you to define Lakebase compute endpoints in a bundle. You must also define corresponding Lakebase projects and Lakebase branches.

Added in Databricks CLI version 0.287.0

postgres_endpoints:
  <postgres_endpoint-name>:
    <postgres_endpoint-field-name>: <postgres_endpoint-field-value>

Example

See postgres_projects example.

postgres_project

Type: Map

The Postgres project resource allows you to define Lakebase Autoscaling Postgres database projects in a bundle. You must also define corresponding Postgres branches and compute endpoints.

Added in Databricks CLI version 0.287.0

postgres_projects:
  <postgres_project-name>:
    <postgres_project-field-name>: <postgres_project-field-value>

Example

resources:
  postgres_projects:
    my_db:
      project_id: test-prod-app
      display_name: 'Production Database'
      pg_version: 17

  postgres_branches:
    main:
      parent: ${resources.postgres_projects.my_db.id}
      branch_id: main
      is_protected: false
      no_expiry: true

  postgres_endpoints:
    primary:
      parent: ${resources.postgres_branches.main.id}
      endpoint_id: primary
      endpoint_type: ENDPOINT_TYPE_READ_WRITE
      autoscaling_limit_min_cu: 0.5
      autoscaling_limit_max_cu: 4

postgres_project.default_endpoint_settings

Type: Map

quality_monitor (Unity Catalog)

Type: Map

The quality_monitor resource allows you to define a Unity Catalog table monitor. For information about monitors, see Data profiling.

Added in Databricks CLI version 0.229.0

quality_monitors:
  <quality_monitor-name>:
    <quality_monitor-field-name>: <quality_monitor-field-value>

quality_monitor.custom_metrics

Type: Sequence

A list of custom metric definitions.

Each item in the list is a CustomMetric:

quality_monitor.inference_log

Type: Map

Configuration for monitoring inference logs.

quality_monitor.notifications

Type: Map

Notification settings for the monitor.

quality_monitor.notifications.on_failure

Type: Map

Notification settings when the monitor fails.

quality_monitor.notifications.on_new_classification_tag_detected

Type: Map

Notification settings when new classification tags are detected.

quality_monitor.schedule

Type: Map

Schedule for automatically updating and refreshing metric tables.

quality_monitor.snapshot

Type: Map

Configuration for monitoring snapshot tables.

quality_monitor.time_series

Configuration for monitoring time series tables.

Examples

The following examples define quality monitors for InferenceLog, TimeSeries, and Snapshot profile types.

# InferenceLog profile type
resources:
  quality_monitors:
    my_quality_monitor:
      table_name: dev.mlops_schema.predictions
      output_schema_name: ${bundle.target}.mlops_schema
      assets_dir: /Workspace/Users/${workspace.current_user.userName}/databricks_lakehouse_monitoring
      inference_log:
        granularities: [1 day]
        model_id_col: model_id
        prediction_col: prediction
        label_col: price
        problem_type: PROBLEM_TYPE_REGRESSION
        timestamp_col: timestamp
      schedule:
        quartz_cron_expression: 0 0 8 * * ? # Run Every day at 8am
        timezone_id: UTC

# TimeSeries profile type
resources:
  quality_monitors:
    my_quality_monitor:
      table_name: dev.mlops_schema.predictions
      output_schema_name: ${bundle.target}.mlops_schema
      assets_dir: /Workspace/Users/${workspace.current_user.userName}/databricks_lakehouse_monitoring
      time_series:
        granularities: [30 minutes]
        timestamp_col: timestamp
      schedule:
        quartz_cron_expression: 0 0 8 * * ? # Run Every day at 8am
        timezone_id: UTC

# Snapshot profile type
resources:
  quality_monitors:
    my_quality_monitor:
      table_name: dev.mlops_schema.predictions
      output_schema_name: ${bundle.target}.mlops_schema
      assets_dir: /Workspace/Users/${workspace.current_user.userName}/databricks_lakehouse_monitoring
      snapshot: {}
      schedule:
        quartz_cron_expression: 0 0 8 * * ? # Run Every day at 8am
        timezone_id: UTC

The following example configures a quality monitor and a corresponding model retraining job based on the monitoring:

# Quality monitoring workflow
resources:
  quality_monitors:
    mlops_quality_monitor:
      table_name: ${bundle.target}.mlops_demo.predictions
      output_schema_name: ${bundle.target}.mlops_demo
      assets_dir: /Users/${workspace.current_user.userName}/databricks_lakehouse_monitoring
      inference_log:
        granularities: [1 hour]
        model_id_col: model_version
        prediction_col: prediction
        label_col: fare_amount
        problem_type: PROBLEM_TYPE_REGRESSION
        timestamp_col: inference_timestamp
      schedule:
        quartz_cron_expression: 57 0 14 * * ? # refresh monitoring metrics every day at 7 am PT
        timezone_id: UTC
  jobs:
    retraining_job:
      name: ${bundle.target}-mlops_demo-monitoring-retraining-job
      tasks:
        - task_key: monitored_metric_violation_check
          notebook_task:
            notebook_path: ../monitoring/notebooks/MonitoredMetricViolationCheck.py
            base_parameters:
              env: ${bundle.target}
              table_name_under_monitor: ${bundle.target}.mlops_demo.predictions
              metric_to_monitor: r2_score
              metric_violation_threshold: 0.7
              num_evaluation_windows: 24
              num_violation_windows: 5 # 5 out of the past 24 windows have metrics lower than threshold

        - task_key: is_metric_violated
          depends_on:
            - task_key: monitored_metric_violation_check
          condition_task:
            op: EQUAL_TO
            left: '{{tasks.monitored_metric_violation_check.values.is_metric_violated}}'
            right: 'true'

        - task_key: trigger_retraining
          depends_on:
            - task_key: is_metric_violated
              outcome: 'true'
          run_job_task:
            job_id: ${resources.jobs.model_training_job.id}

      schedule:
        quartz_cron_expression: '0 0 15 * * ?' # daily at 8 am PDT
        timezone_id: UTC

      # To get notifications, provide a list of emails to the on_failure argument.
      #
      #  email_notifications:
      #    on_failure:
      #      - someone@example.com

registered_model (Unity Catalog)

Type: Map

The registered model resource allows you to define models in Unity Catalog. For information about Unity Catalog registered models, see Manage model lifecycle in Unity Catalog.

Added in Databricks CLI version 0.229.0

registered_models:
  <registered_model-name>:
    <registered_model-field-name>: <registered_model-field-value>

registered_model.aliases

Type: Sequence

A list of aliases associated with the registered model.

Each item in the list is an Alias:

Example

The following example defines a registered model in Unity Catalog:

resources:
  registered_models:
    model:
      name: my_model
      catalog_name: ${bundle.target}
      schema_name: mlops_schema
      comment: Registered model in Unity Catalog for ${bundle.target} deployment target
      grants:
        - privileges:
            - EXECUTE
          principal: account users

schema (Unity Catalog)

Type: Map

Schemas are supported in Python for Databricks Asset Bundles. See databricks.bundles.schemas.

The schema resource type allows you to define Unity Catalog schemas for tables and other assets in your workflows and pipelines created as part of a bundle. A schema, different from other resource types, has the following limitations:

• The owner of a schema resource is always the deployment user, and cannot be changed. If run_as is specified in the bundle, it will be ignored by operations on the schema.
• Only fields supported by the corresponding Schemas object create API are available for the schema resource. For example, enable_predictive_optimization is not supported as it is only available on the update API.

Added in Databricks CLI version 0.229.0

schemas:
  <schema-name>:
    <schema-field-name>: <schema-field-value>

Examples

The following example defines a pipeline with the resource key my_pipeline that creates a Unity Catalog schema with the key my_schema as the target:

resources:
  pipelines:
    my_pipeline:
      name: test-pipeline-{{.unique_id}}
      libraries:
        - notebook:
            path: ../src/nb.ipynb
        - file:
            path: ../src/range.sql
      development: true
      catalog: ${resources.schemas.my_schema.catalog_name}
      target: ${resources.schemas.my_schema.id}

  schemas:
    my_schema:
      name: test-schema-{{.unique_id}}
      catalog_name: main
      comment: This schema was created by Databricks Asset Bundles.

A top-level grants mapping is not supported by Databricks Asset Bundles, so if you want to set grants for a schema, define the grants for the schema within the schemas mapping. For more information about grants, see Show, grant, and revoke privileges.

The following example defines a Unity Catalog schema with grants:

resources:
  schemas:
    my_schema:
      name: test-schema
      grants:
        - principal: users
          privileges:
            - SELECT
        - principal: my_team
          privileges:
            - CAN_MANAGE
      catalog_name: main

secret_scope

Type: Map

The secret_scope resource allows you to define secret scopes in a bundle. For information about secret scopes, see Secret management.

Added in Databricks CLI version 0.252.0

secret_scopes:
  <secret_scope-name>:
    <secret_scope-field-name>: <secret_scope-field-value>

secret_scope.keyvault_metadata

Type: Map

The metadata for Azure Key Vault-backed secret scopes.

Examples

The following example defines a secret scope that uses a key vault backend:

resources:
  secret_scopes:
    secret_scope_azure:
      name: test-secrets-azure-backend
      backend_type: 'AZURE_KEYVAULT'
      keyvault_metadata:
        resource_id: my_azure_keyvault_id
        dns_name: my_azure_keyvault_dns_name

The following example sets a custom ACL using secret scopes and permissions:

resources:
  secret_scopes:
    my_secret_scope:
      name: my_secret_scope
      permissions:
        - user_name: admins
          level: WRITE
        - user_name: users
          level: READ

For an example bundle that demonstrates how to define a secret scope and a job with a task that reads from it in a bundle, see the bundle-examples GitHub repository.

sql_warehouse

Type: Map

The SQL warehouse resource allows you to define a SQL warehouse in a bundle. For information about SQL warehouses, see Data warehousing on Azure Databricks.

Added in Databricks CLI version 0.260.0

sql_warehouses:
  <sql-warehouse-name>:
    <sql-warehouse-field-name>: <sql-warehouse-field-value>

sql_warehouse.channel

Type: Map

The channel configuration for the SQL warehouse.

Example

The following example defines a SQL warehouse:

resources:
  sql_warehouses:
    my_sql_warehouse:
      name: my_sql_warehouse
      cluster_size: X-Large
      enable_serverless_compute: true
      max_num_clusters: 3
      min_num_clusters: 1
      auto_stop_mins: 60
      warehouse_type: PRO

synced_database_table

Type: Map

The synced database table resource allows you to define Lakebase database tables in a bundle.

For information about synced database tables, see What is a database instance?.

Added in Databricks CLI version 0.266.0

synced_database_tables:
  <synced_database_table-name>:
    <synced_database_table-field-name>: <synced_database_table-field-value>

synced_database_table.spec

Type: Map

The database table specification.

Added in Databricks CLI version 0.266.0

synced_database_table.spec.new_pipeline_spec

Type: Map

The specification for a new pipeline used by the synced database table.

Examples

The following example defines a synced database table within a corresponding database catalog:

resources:
  database_instances:
    my_instance:
      name: my-instance
      capacity: CU_1
  database_catalogs:
    my_catalog:
      database_instance_name: my-instance
      database_name: 'my_database'
      name: my_catalog
      create_database_if_not_exists: true
  synced_database_tables:
    my_synced_table:
      name: ${resources.database_catalogs.my_catalog.name}.${resources.database_catalogs.my_catalog.database_name}.my_destination_table
      database_instance_name: ${resources.database_catalogs.my_catalog.database_instance_name}
      logical_database_name: ${resources.database_catalogs.my_catalog.database_name}
      spec:
        source_table_full_name: 'my_source_table'
        scheduling_policy: SNAPSHOT
        primary_key_columns:
          - my_pk_column
        new_pipeline_spec:
          storage_catalog: 'my_delta_catalog'
          storage_schema: 'my_delta_schema'

The following example defines a synced database table inside a standard catalog:

resources:
  synced_database_tables:
    my_synced_table:
      name: 'my_standard_catalog.public.synced_table'
      # database_instance_name is required for synced tables created in standard catalogs.
      database_instance_name: 'my-database-instance'
      # logical_database_name is required for synced tables created in standard catalogs:
      logical_database_name: ${resources.database_catalogs.my_catalog.database_name}
      spec:
        source_table_full_name: 'source_catalog.schema.table'
        scheduling_policy: SNAPSHOT
        primary_key_columns:
          - my_pk_column
        create_database_objects_if_missing: true
        new_pipeline_spec:
          storage_catalog: 'my_delta_catalog'
          storage_schema: 'my_delta_schema'

This example creates a synced database table and customizes the pipeline schedule for it. It assumes you already have:

• A database instance named my-database-instance
• A standard catalog named my_standard_catalog
• A schema in the standard catalog named default
• A source delta table named source_delta.schema.customer with the primary key c_custkey

resources:
  synced_database_tables:
    my_synced_table:
      name: 'my_standard_catalog.default.my_synced_table'
      database_instance_name: 'my-database-instance'
      logical_database_name: 'test_db'
      spec:
        source_table_full_name: 'source_delta.schema.customer'
        scheduling_policy: SNAPSHOT
        primary_key_columns:
          - c_custkey
        create_database_objects_if_missing: true
        new_pipeline_spec:
          storage_catalog: 'source_delta'
          storage_schema: 'schema'

  jobs:
    sync_pipeline_schedule_job:
      name: sync_pipeline_schedule_job
      description: 'Job to schedule synced database table pipeline.'
      tasks:
        - task_key: synced-table-pipeline
          pipeline_task:
            pipeline_id: ${resources.synced_database_tables.my_synced_table.data_synchronization_status.pipeline_id}
      schedule:
        quartz_cron_expression: '0 0 0 * * ?'

volume (Unity Catalog)

Type: Map

Volumes are supported in Python for Databricks Asset Bundles. See databricks.bundles.volumes.

The volume resource type allows you to define and create Unity Catalog volumes as part of a bundle. When deploying a bundle with a volume defined, note that:

• A volume cannot be referenced in the artifact_path for the bundle until it exists in the workspace. Hence, if you want to use Databricks Asset Bundles to create the volume, you must first define the volume in the bundle, deploy it to create the volume, then reference it in the artifact_path in subsequent deployments.
• Volumes in the bundle are not prepended with the dev_${workspace.current_user.short_name} prefix when the deployment target has mode: development configured. However, you can manually configure this prefix. See Custom presets.

Added in Databricks CLI version 0.236.0

volumes:
  <volume-name>:
    <volume-field-name>: <volume-field-value>

Example

The following example creates a Unity Catalog volume with the key my_volume_id:

resources:
  volumes:
    my_volume_id:
      catalog_name: main
      name: my_volume
      schema_name: my_schema

For an example bundle that runs a job that writes to a file in Unity Catalog volume, see the bundle-examples GitHub repository.

Common objects

grant

Type: Map

Defines the principal and privileges to grant to that principal. For more information about grants, see Show, grant, and revoke privileges.

Added in Databricks CLI version 0.229.0

Example

The following example defines a Unity Catalog schema with grants:

resources:
  schemas:
    my_schema:
      name: test-schema
      grants:
        - principal: users
          privileges:
            - SELECT
        - principal: my_team
          privileges:
            - CAN_MANAGE
      catalog_name: main

lifecycle

Type: Map

Contains the lifecycle settings for a resource. It controls the behavior of the resource when it is deployed or destroyed.

Added in Databricks CLI version 0.268.0