Initial implementation of "direct" deployment backend #2926
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced May 22, 2025
Merged
github-merge-queue bot
pushed a commit
that referenced
this pull request
May 23, 2025
## Changes - Support editing pipelines in testserver. - Rewrite pipeline code to match best practice in testserver (separate file in libs/testserver and separate block in handlers.go). ## Why Enables new acceptance tests, I'm using this in terraform-removal branch #2926 ## Tests New edit functionality is tested via #2926
denik
force-pushed
the
denik/remove-terraform2
branch
from
0107675 to
0d353f4
Compare
github-merge-queue bot
pushed a commit
that referenced
this pull request
May 23, 2025
## Changes A new helper to update diags from multiple threads. Also includes shortcuts for frequently used Error and Errorf. ## Why Simpler and less error-prone code in mutators. Using this in #2926 ## Tests Existing tests.
denik
force-pushed
the
denik/remove-terraform2
branch
from
0d353f4 to
1c15208
Compare
denik
force-pushed
the
denik/remove-terraform2
branch
2 times, most recently
from
42ff5c9 to
e1b1e62
Compare
denik
force-pushed
the
denik/remove-terraform2
branch
from
e1b1e62 to
343e13d
Compare
denik
added a commit
that referenced
this pull request
May 26, 2025
## Changes - Update testserver pipeline to populate fields same way as real server does it. - Add new pipeline server that exercises Create and Update. - Add new script read_id.py to get id from the state. ## Why Using this test in #2926 ## Tests New acceptance local + integration test.
denik
force-pushed
the
denik/remove-terraform2
branch
from
1f0fd3f to
46e42b5
Compare
denik
force-pushed
the
denik/remove-terraform2
branch
from
f969bed to
2a7df40
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Add new deployment method: “direct” which deploys without terraform. It can be enabled with new env var DATABRICKS_CLI_DEPLOYMENT set to “direct”. There is partial support for jobs, pipelines, schemas and apps.
This is not intended for end users at this moment, it's a first PR in a series.
Compared to previous iteration #2791 the resources are implemented using type-safe SDK clients, not calling HTTP endpoints directly.
Why
This is going to be first developed into feature matching "terraform" and then (after deprecation period) become the only method of deployment of DABs. This deployment method addresses issues with relying on terraform such as license of terraform, having to download terraform binary and provider from the internet and in general will give us more control over deployment.
Tests
All acceptance tests by default are now run with two settings of DATABRICKS_CLI_DEPLOYMENT: “terraform” and “direct”. Some tests override this by disabling direct method.
Running all tests within a single github job caused the job time to grow non-linearly, so I’ve split each job in two - one for terraform and one for direct deployment, using ENVFILTER option to testrunner, see #2944