Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 15 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,41 +56,25 @@ cd <project_name>
manifold add <data_project_name>
```

3. **Generate BigQuery Resource Definitions**
3. **Generate BigQuery Resource Definitions and Terraform Configuration**

After you fill out the manifold.yml file, this command generates the necessary BigQuery schema files based on the specified dimensions and metrics.
After you fill out the manifold.yml file, this command generates the necessary BigQuery schema files and Terraform configurations based on the specified dimensions and metrics.

```bash
# Generate with submodule configuration (default)
manifold generate
```

4. **Generate Terraform Configuration (Optional)**

Manifold can optionally generate Terraform configurations for managing your BigQuery resources. To generate both BigQuery schemas and Terraform configurations, use the `--tf` flag:

```bash
# Generate standard Terraform configuration
manifold generate --tf

# Generate Terraform configuration as a submodule (skips provider configuration)
manifold generate --tf --submodule
# Generate with provider configuration (if needed)
manifold generate --no-submodule
```

This will create:

- A root `main.tf.json` file that sets up the Google Cloud provider and workspace modules (unless using `--submodule`)
- A root `main.tf.json` file that sets up workspace modules (by default) or includes provider configuration (with --no-submodule)
- Individual workspace configurations in `workspaces/<workspace_name>/main.tf.json`
- Dataset and table definitions that reference your generated BigQuery schemas

The generated Terraform configurations use the Google Cloud provider and expect a `project_id` variable to be set. When using the standard configuration (without `--submodule`), you can apply these configurations directly:

```bash
terraform init
terraform plan -var="project_id=your-project-id"
terraform apply -var="project_id=your-project-id"
```

When using `--submodule`, the generated configuration skips provider setup, allowing you to include the project as a module in a larger Terraform configuration:
By default, the generated Terraform configurations are designed to be included as a module in your larger Terraform configuration:

```hcl
module "my_manifold_project" {
Expand All @@ -99,6 +83,14 @@ module "my_manifold_project" {
}
```

If you need to use the configuration standalone (with `--no-submodule`), you can apply it directly:

```bash
terraform init
terraform plan -var="project_id=your-project-id"
terraform apply -var="project_id=your-project-id"
```

## Manifold Configuration

### Vectors
Expand Down
9 changes: 4 additions & 5 deletions lib/manifold/cli.rb
Original file line number Diff line number Diff line change
Expand Up @@ -45,16 +45,15 @@ def add(name)
logger.info "Added workspace '#{name}' with tables and routines directories."
end

desc "generate", "Generate BigQuery schema for all workspaces in the project"
method_option :tf, type: :boolean, desc: "Generate Terraform configurations"
method_option :submodule, type: :boolean, default: false,
desc "generate", "Generate BigQuery schema and Terraform configurations for all workspaces in the project"
method_option :submodule, type: :boolean, default: true,
desc: "Generate Terraform configurations as a submodule (skips provider configuration)"
def generate
path = Pathname.pwd
name = path.basename.to_s
project = API::Project.new(name, directory: path, logger:)
project.generate(with_terraform: options[:tf], is_submodule: options[:submodule])
logger.info "Generated BigQuery schema for all workspaces in the project."
project.generate(with_terraform: true, is_submodule: options[:submodule])
logger.info "Generated BigQuery schema and Terraform configurations for all workspaces in the project."
end
end
end
2 changes: 1 addition & 1 deletion lib/manifold/version.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# frozen_string_literal: true

module Manifold
VERSION = "0.0.10"
VERSION = "0.0.11"
end
29 changes: 16 additions & 13 deletions spec/manifold/cli_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -46,32 +46,35 @@
allow(mock_project).to receive(:generate)
end

context "when called with --tf option" do
it "generates terraform configurations" do
cli.options = { tf: true, submodule: false }
cli.generate
expect(mock_project).to have_received(:generate)
.with(with_terraform: true, is_submodule: false)
context "with default options" do
before do
cli.options = { submodule: true }
end
end

context "when called with --tf and --submodule options" do
it "generates terraform configurations as a submodule" do
cli.options = { tf: true, submodule: true }
cli.generate
expect(mock_project).to have_received(:generate)
.with(with_terraform: true, is_submodule: true)
end
end

context "when called without --tf option" do
it "does not generate terraform configurations" do
cli.options = { tf: false, submodule: false }
context "with --no-submodule option" do
before do
cli.options = { submodule: false }
end

it "generates terraform configurations with provider" do
cli.generate
expect(mock_project).to have_received(:generate)
.with(with_terraform: false, is_submodule: false)
.with(with_terraform: true, is_submodule: false)
end
end

it "logs the generation" do
cli.generate
expect(null_logger).to have_received(:info)
.with("Generated BigQuery schema and Terraform configurations for all workspaces in the project.")
end
end

describe "#add" do
Expand Down