Configure Databricks Serverless Private Git

Databricks Serverless Private Git lets you connect a Databricks workspace to a private Git server using serverless compute and Azure Private Link. A Git server is private if internet users can't access it.

The following diagram illustrates the overall system architecture:

Why use Serverless Private Git?

Compared to Git server proxy, Serverless Private Git offers the following advantages:

• Serverless Private Git acquires serverless compute only when it receives a Git request, and it can be inactive when not in use. In contrast, the Git proxy requires the proxy cluster to be active when the user submits a Git request.
• Serverless Private Git uses Azure Private Link to securely connect to the private Git instance.

Requirements

• Enable serverless compute for the workspace.
• Place the private Git server in the same Azure VNet as the Standard Load Balancer.
• Ensure the private Git server has a signed certificate and valid HTTPS fully qualified domain name (FQDN).
• Configure the VNet with a Standard Load Balancer (SLB) for the Private Link service.

Set up Serverless Private Git

1. Follow the steps to Configure private connectivity to resources in your VNet. This allows you to create an Azure Private Link connection from Serverless to backends in your network behind an SLB.
2. Create a network connectivity configuration (NCC) to configure egress to a standard load balancer.
   • You can configure only one NCC per workspace for private Git. If the workspace connects to multiple private Git servers, they must all use the same NCC.
   • For NCC limitations, such as regional limits and workspace attachment limits, see Requirements.

3. Obtain an account API token using a service principal with account-level access.

curl --location 'https://accounts.azuredatabricks.net/oidc/accounts/{accountid}/v1/token' \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data-urlencode 'client_id=SP_CLIENT_ID_HERE' \
  --data-urlencode 'grant_type=client_credentials' \
  --data-urlencode 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default' \
  --data-urlencode 'client_secret=SP_CLIENT_SECRET_HERE'

The response contains an access token:

{ "access_token": "...", "scope": "all-apis", "token_type": "Bearer", "expires_in": 3600 }

Alternatively, you can use a Microsoft Entra ID access token:

BEARER_TOKEN=$(az account get-access-token --resource \
  2ff814a6-3304-4ab8-85cb-cd0e6f879c1d --query "accessToken" -o tsv)

4. Add a private endpoint rule to define DNS logic using the API.

In the example, specify the following:

• Account ID
• NCC ID
• Account OAuth Token
• Private Link service fesource ID
• Git server’s FQDN in the domain_name list

curl --location 'https://accounts.azuredatabricks.net/api/2.0/accounts/{accountid}/network-connectivity-configs/{nccid}/private-endpoint-rules' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer BEARER_TOKEN' \
  --data '{
    "resource_id": "/subscriptions/3f262328b/resourceGroups/rg/providers/Microsoft.Network/privateLinkServices/example",
    "domain_names": ["git-server.contoso.com"]
  }'

The response contains the private endpoint rule details:

{
  "rule_id": "843ba2e5-bbbb-bbbb-bbbb-7f0d55555215",
  "network_connectivity_config_id": "5a9bdc5f-c43d-41cd-9a6d-1b653e20c7d2",
  "resource_id": "/subscriptions/3f262328b/resourceGroups/rg/providers/Microsoft.Network/privateLinkServices/example",
  "endpoint_name": "databricks-5a9bdc5f-c43d-41cd-9a6d-1b653e20c7d2-pe-99cbbac3",
  "connection_state": "PENDING",
  "creation_time": 1740000647980,
  "updated_time": 1740000647949,
  "domain_names": ["git-server.contoso.com"]
}

5. Wait a few minutes after setting up the NCC private endpoint rules. The NCC displays the private endpoint rule with a pending status.
6. Approve the pending private endpoint connection in the Private Link service you configured in Step 1.

7. Return to the NCC inside the account console and check that it was established.
8. Perform a Git operation in the workspace. A UI indicator confirms that Serverless Private Git is active. The indicator might take a few seconds to appear while serverless compute starts.

After you configure it, Serverless Private Git takes precedence over other forms of private Git connectivity you already provisioned, such as classic Git proxy. If you have a classic Git proxy cluster running, terminate it after setting up Serverless Private Git.

Additional configurations

Customize Git operations using a configuration file.

1. Create a configuration file at /Workspace/.git_settings/config.json following the specification below.
2. Grant all Git users View permissions to the configuration file and any CA certificate files it references.
3. Validate connectivity to the Git remote by performing a Git operation, such as cloning a Git folder.
4. The system can take up to one minute to apply configuration file changes.

Top-level config file structure

{
  "default": { ... },  // Optional global settings
  "remotes": [ ... ]   // Optional list of per-remote settings
}

default section (optional)

Global defaults apply to all Git operations unless a specific remote overrides them.

remotes section (optional)

A list of objects defining settings for individual remote Git servers. These settings override the default block on a per-remote basis.

Example configuration with no remote-specific configuration

{
  "default": {
    "sslVerify": false
  }
}

Full configuration example

{
  "default": {
    "sslVerify": true,
    "caCertPath": "/Workspace/my_ca_cert.pem",
    "httpProxy": "https://git-proxy-server.company.com",
    "customHttpPort": "8080"
  },
  "remotes": [
    {
      "urlPrefix": "https://my-private-git.company.com/",
      "caCertPath": "/Workspace/my_ca_cert_2.pem"
    },
    {
      "urlPrefix": "https://another-git-server.com/project.git",
      "sslVerify": false
    }
  ]
}

Notes

• The default section must be at least partially present.
• The remotes section is optional. If included, each entry must include a urlPrefix field.
• Unspecified fields use their default values.
• Unknown fields are ignored.

Limitations

• Serverless proxy logs aren't available.
• Serverless Private Git is available only in Azure serverless regions.