Skip to content

Commit 64d61a4

Browse files
magreenbaummagreenbaum
authored
feat: S3 Table Bucket Support (#323)
1 parent 4c7f358 commit 64d61a4

File tree

16 files changed

+710
-0
lines changed

16 files changed

+710
-0
lines changed

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ These features of S3 bucket configurations are supported:
1818
- ALB/NLB log delivery bucket policy
1919
- Account-level Public Access Block
2020
- S3 Directory Bucket
21+
- S3 Table Bucket
2122

2223
## Usage
2324

@@ -124,6 +125,7 @@ Users of Terragrunt can achieve similar results by using modules provided in the
124125
- [S3 Inventory](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/s3-inventory) - S3 bucket Inventory configuration.
125126
- [S3 Account-level Public Access Block](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/account-public-access) - Manage S3 account-level Public Access Block.
126127
- [S3 Directory Bucket](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/directory-bucket) - S3 Directory Bucket configuration.
128+
- [S3 Table Bucket](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/table-bucket) - S3 Table Bucket configuration.
127129

128130
<!-- BEGIN_TF_DOCS -->
129131
## Requirements

‎examples/table-bucket/README.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# S3 Table Bucket
2+
3+
Configuration in this directory creates S3 table bucket with bucket policy and S3 Tables with table policies.
4+
5+
## Usage
6+
7+
To run this example you need to execute:
8+
9+
```bash
10+
$ terraform init
11+
$ terraform plan
12+
$ terraform apply
13+
```
14+
15+
Note that this example may create resources which cost money. Run `terraform destroy` when you don't need these resources.
16+
17+
<!-- BEGIN_TF_DOCS -->
18+
## Requirements
19+
20+
| Name | Version |
21+
|------|---------|
22+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
23+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.83 |
24+
| <a name="requirement_random"></a> [random](#requirement\_random) | >= 2.0 |
25+
26+
## Providers
27+
28+
| Name | Version |
29+
|------|---------|
30+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.83 |
31+
| <a name="provider_random"></a> [random](#provider\_random) | >= 2.0 |
32+
33+
## Modules
34+
35+
| Name | Source | Version |
36+
|------|--------|---------|
37+
| <a name="module_table_bucket"></a> [table\_bucket](#module\_table\_bucket) | ../../modules/table-bucket | n/a |
38+
39+
## Resources
40+
41+
| Name | Type |
42+
|------|------|
43+
| [aws_s3tables_namespace.namespace](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3tables_namespace) | resource |
44+
| [random_pet.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
45+
| [aws_caller_identity.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
46+
47+
## Inputs
48+
49+
No inputs.
50+
51+
## Outputs
52+
53+
| Name | Description |
54+
|------|-------------|
55+
| <a name="output_owner_account_id"></a> [owner\_account\_id](#output\_owner\_account\_id) | Account ID of the account that owns the table bucket. |
56+
| <a name="output_s3_table_arns"></a> [s3\_table\_arns](#output\_s3\_table\_arns) | The table ARNs. |
57+
| <a name="output_s3_table_bucket_arn"></a> [s3\_table\_bucket\_arn](#output\_s3\_table\_bucket\_arn) | ARN of the table bucket. |
58+
| <a name="output_s3_table_bucket_created_at"></a> [s3\_table\_bucket\_created\_at](#output\_s3\_table\_bucket\_created\_at) | Date and time when the bucket was created. |
59+
| <a name="output_s3_table_created_at"></a> [s3\_table\_created\_at](#output\_s3\_table\_created\_at) | Dates and times when the tables were created. |
60+
| <a name="output_s3_table_created_by"></a> [s3\_table\_created\_by](#output\_s3\_table\_created\_by) | Account IDs of the accounts that created the tables |
61+
| <a name="output_s3_table_metadata_locations"></a> [s3\_table\_metadata\_locations](#output\_s3\_table\_metadata\_locations) | Locations of table metadata. |
62+
| <a name="output_s3_table_modified_at"></a> [s3\_table\_modified\_at](#output\_s3\_table\_modified\_at) | Dates and times when the tables was last modified. |
63+
| <a name="output_s3_table_modified_by"></a> [s3\_table\_modified\_by](#output\_s3\_table\_modified\_by) | Account IDs of the accounts that last modified the tables. |
64+
| <a name="output_s3_table_owner_account_ids"></a> [s3\_table\_owner\_account\_ids](#output\_s3\_table\_owner\_account\_ids) | Account IDs of the accounts that owns the tables. |
65+
| <a name="output_s3_table_types"></a> [s3\_table\_types](#output\_s3\_table\_types) | Types of the tables. One of customer or aws. |
66+
| <a name="output_s3_table_version_tokens"></a> [s3\_table\_version\_tokens](#output\_s3\_table\_version\_tokens) | Identifiers for the current version of table data. |
67+
| <a name="output_s3_table_warehouse_locations"></a> [s3\_table\_warehouse\_locations](#output\_s3\_table\_warehouse\_locations) | S3 URIs pointing to the S3 Bucket that contains the table data. |
68+
<!-- END_TF_DOCS -->

‎examples/table-bucket/main.tf‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
provider "aws" {
2+
region = "eu-west-1"
3+
4+
# Make it faster by skipping something
5+
skip_metadata_api_check = true
6+
skip_region_validation = true
7+
skip_credentials_validation = true
8+
}
9+
10+
locals {
11+
bucket_name = "s3-table-bucket-${random_pet.this.id}"
12+
}
13+
14+
data "aws_caller_identity" "this" {}
15+
16+
module "table_bucket" {
17+
source = "../../modules/table-bucket"
18+
19+
table_bucket_name = local.bucket_name
20+
21+
maintenance_configuration = {
22+
iceberg_unreferenced_file_removal = {
23+
status = "enabled"
24+
25+
settings = {
26+
non_current_days = 7
27+
unreferenced_days = 3
28+
}
29+
}
30+
}
31+
32+
create_table_bucket_policy = true
33+
table_bucket_policy_statements = [
34+
{
35+
effect = "Allow"
36+
principals = [{
37+
type = "AWS"
38+
identifiers = [data.aws_caller_identity.this.account_id]
39+
}]
40+
actions = [
41+
"s3tables:GetTableData",
42+
"s3tables:GetTableMetadataLocation"
43+
]
44+
}
45+
]
46+
47+
tables = {
48+
table1 = {
49+
format = "ICEBERG"
50+
namespace = aws_s3tables_namespace.namespace.namespace
51+
52+
maintenance_configuration = {
53+
iceberg_compaction = {
54+
status = "enabled"
55+
settings = {
56+
target_file_size_mb = 64
57+
}
58+
}
59+
iceberg_snapshot_management = {
60+
status = "enabled"
61+
settings = {
62+
max_snapshot_age_hours = 40
63+
min_snapshots_to_keep = 3
64+
}
65+
}
66+
}
67+
68+
create_table_policy = true
69+
policy_statements = [
70+
{
71+
sid = "DeleteTable"
72+
effect = "Allow"
73+
principals = [{
74+
type = "AWS"
75+
identifiers = [data.aws_caller_identity.this.account_id]
76+
}]
77+
actions = [
78+
"s3tables:DeleteTable",
79+
"s3tables:UpdateTableMetadataLocation",
80+
"s3tables:PutTableData",
81+
"s3tables:GetTableMetadataLocation"
82+
]
83+
}
84+
]
85+
}
86+
table2 = {
87+
format = "ICEBERG"
88+
name = "table2"
89+
namespace = aws_s3tables_namespace.namespace.namespace
90+
}
91+
table3 = {
92+
format = "ICEBERG"
93+
namespace = aws_s3tables_namespace.namespace.namespace
94+
}
95+
}
96+
}
97+
98+
resource "random_pet" "this" {
99+
length = 2
100+
}
101+
102+
resource "aws_s3tables_namespace" "namespace" {
103+
namespace = "example_namespace"
104+
table_bucket_arn = module.table_bucket.s3_table_bucket_arn
105+
}

‎examples/table-bucket/outputs.tf‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
output "s3_table_bucket_arn" {
2+
description = "ARN of the table bucket."
3+
value = module.table_bucket.s3_table_bucket_arn
4+
}
5+
6+
output "s3_table_bucket_created_at" {
7+
description = "Date and time when the bucket was created."
8+
value = module.table_bucket.s3_table_bucket_created_at
9+
}
10+
11+
output "owner_account_id" {
12+
description = "Account ID of the account that owns the table bucket."
13+
value = module.table_bucket.s3_table_bucket_owner_account_id
14+
}
15+
16+
output "s3_table_arns" {
17+
description = "The table ARNs."
18+
value = module.table_bucket.s3_table_arns
19+
}
20+
21+
output "s3_table_created_at" {
22+
description = "Dates and times when the tables were created."
23+
value = module.table_bucket.s3_table_created_at
24+
}
25+
26+
output "s3_table_created_by" {
27+
description = "Account IDs of the accounts that created the tables"
28+
value = module.table_bucket.s3_table_created_by
29+
}
30+
31+
output "s3_table_metadata_locations" {
32+
description = "Locations of table metadata."
33+
value = module.table_bucket.s3_table_metadata_locations
34+
}
35+
36+
output "s3_table_modified_at" {
37+
description = "Dates and times when the tables was last modified."
38+
value = module.table_bucket.s3_table_modified_at
39+
}
40+
41+
output "s3_table_modified_by" {
42+
description = "Account IDs of the accounts that last modified the tables."
43+
value = module.table_bucket.s3_table_modified_by
44+
}
45+
46+
output "s3_table_owner_account_ids" {
47+
description = "Account IDs of the accounts that owns the tables."
48+
value = module.table_bucket.s3_table_owner_account_ids
49+
}
50+
51+
output "s3_table_types" {
52+
description = "Types of the tables. One of customer or aws."
53+
value = module.table_bucket.s3_table_types
54+
}
55+
56+
output "s3_table_version_tokens" {
57+
description = "Identifiers for the current version of table data."
58+
value = module.table_bucket.s3_table_version_tokens
59+
}
60+
61+
output "s3_table_warehouse_locations" {
62+
description = "S3 URIs pointing to the S3 Bucket that contains the table data."
63+
value = module.table_bucket.s3_table_warehouse_locations
64+
}

‎examples/table-bucket/variables.tf‎

Whitespace-only changes.

‎examples/table-bucket/versions.tf‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
terraform {
2+
required_version = ">= 1.0"
3+
4+
required_providers {
5+
aws = {
6+
source = "hashicorp/aws"
7+
version = ">= 5.83"
8+
}
9+
random = {
10+
source = "hashicorp/random"
11+
version = ">= 2.0"
12+
}
13+
}
14+
}

‎modules/table-bucket/README.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# S3 Table Bucket
2+
3+
Creates S3 Table Bucket and Tables with various configurations.
4+
5+
<!-- BEGIN_TF_DOCS -->
6+
## Requirements
7+
8+
| Name | Version |
9+
|------|---------|
10+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
11+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.83 |
12+
13+
## Providers
14+
15+
| Name | Version |
16+
|------|---------|
17+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.83 |
18+
19+
## Modules
20+
21+
No modules.
22+
23+
## Resources
24+
25+
| Name | Type |
26+
|------|------|
27+
| [aws_s3tables_table.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3tables_table) | resource |
28+
| [aws_s3tables_table_bucket.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3tables_table_bucket) | resource |
29+
| [aws_s3tables_table_bucket_policy.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3tables_table_bucket_policy) | resource |
30+
| [aws_s3tables_table_policy.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3tables_table_policy) | resource |
31+
| [aws_iam_policy_document.table_bucket_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
32+
| [aws_iam_policy_document.table_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
33+
34+
## Inputs
35+
36+
| Name | Description | Type | Default | Required |
37+
|------|-------------|------|---------|:--------:|
38+
| <a name="input_create"></a> [create](#input\_create) | Whether to create s3 table resources | `bool` | `true` | no |
39+
| <a name="input_create_table_bucket_policy"></a> [create\_table\_bucket\_policy](#input\_create\_table\_bucket\_policy) | Whether to create s3 table bucket policy | `bool` | `false` | no |
40+
| <a name="input_maintenance_configuration"></a> [maintenance\_configuration](#input\_maintenance\_configuration) | Map of table bucket maintenance configurations | `any` | `{}` | no |
41+
| <a name="input_table_bucket_name"></a> [table\_bucket\_name](#input\_table\_bucket\_name) | Name of the table bucket. Must be between 3 and 63 characters in length. Can consist of lowercase letters, numbers, and hyphens, and must begin and end with a lowercase letter or number | `string` | `null` | no |
42+
| <a name="input_table_bucket_override_policy_documents"></a> [table\_bucket\_override\_policy\_documents](#input\_table\_bucket\_override\_policy\_documents) | List of IAM policy documents that are merged together into the exported document. In merging, statements with non-blank `sid`s will override statements with the same `sid` | `list(string)` | `[]` | no |
43+
| <a name="input_table_bucket_policy"></a> [table\_bucket\_policy](#input\_table\_bucket\_policy) | Amazon Web Services resource-based policy document in JSON format | `string` | `null` | no |
44+
| <a name="input_table_bucket_policy_statements"></a> [table\_bucket\_policy\_statements](#input\_table\_bucket\_policy\_statements) | A map of IAM policy [statements](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document#statement) for custom permission usage | `any` | `{}` | no |
45+
| <a name="input_table_bucket_source_policy_documents"></a> [table\_bucket\_source\_policy\_documents](#input\_table\_bucket\_source\_policy\_documents) | List of IAM policy documents that are merged together into the exported document. Statements must have unique `sid`s | `list(string)` | `[]` | no |
46+
| <a name="input_tables"></a> [tables](#input\_tables) | Map of table configurations | `any` | `{}` | no |
47+
48+
## Outputs
49+
50+
| Name | Description |
51+
|------|-------------|
52+
| <a name="output_s3_table_arns"></a> [s3\_table\_arns](#output\_s3\_table\_arns) | The table ARNs. |
53+
| <a name="output_s3_table_bucket_arn"></a> [s3\_table\_bucket\_arn](#output\_s3\_table\_bucket\_arn) | ARN of the table bucket. |
54+
| <a name="output_s3_table_bucket_created_at"></a> [s3\_table\_bucket\_created\_at](#output\_s3\_table\_bucket\_created\_at) | Date and time when the bucket was created. |
55+
| <a name="output_s3_table_bucket_owner_account_id"></a> [s3\_table\_bucket\_owner\_account\_id](#output\_s3\_table\_bucket\_owner\_account\_id) | Account ID of the account that owns the table bucket. |
56+
| <a name="output_s3_table_created_at"></a> [s3\_table\_created\_at](#output\_s3\_table\_created\_at) | Dates and times when the tables were created. |
57+
| <a name="output_s3_table_created_by"></a> [s3\_table\_created\_by](#output\_s3\_table\_created\_by) | Account IDs of the accounts that created the tables |
58+
| <a name="output_s3_table_metadata_locations"></a> [s3\_table\_metadata\_locations](#output\_s3\_table\_metadata\_locations) | Locations of table metadata. |
59+
| <a name="output_s3_table_modified_at"></a> [s3\_table\_modified\_at](#output\_s3\_table\_modified\_at) | Dates and times when the tables was last modified. |
60+
| <a name="output_s3_table_modified_by"></a> [s3\_table\_modified\_by](#output\_s3\_table\_modified\_by) | Account IDs of the accounts that last modified the tables. |
61+
| <a name="output_s3_table_owner_account_ids"></a> [s3\_table\_owner\_account\_ids](#output\_s3\_table\_owner\_account\_ids) | Account IDs of the accounts that owns the tables. |
62+
| <a name="output_s3_table_types"></a> [s3\_table\_types](#output\_s3\_table\_types) | Types of the tables. One of customer or aws. |
63+
| <a name="output_s3_table_version_tokens"></a> [s3\_table\_version\_tokens](#output\_s3\_table\_version\_tokens) | Identifiers for the current version of table data. |
64+
| <a name="output_s3_table_warehouse_locations"></a> [s3\_table\_warehouse\_locations](#output\_s3\_table\_warehouse\_locations) | S3 URIs pointing to the S3 Bucket that contains the table data. |
65+
<!-- END_TF_DOCS -->

0 commit comments

Comments
 (0)