'AttributeError: 'str' object has no attribute 'append''

[rslotte]

Description

Getting the following error when running tf-compliance after checks:

11 features (0 passed)
16 scenarios (0 passed)
67 steps (0 passed)
Run 1698062751 finished within a moment
! ERROR: Hook 'load_terraform_data' from /home/runner/.local/lib/python3.10/site-packages/terraform_compliance/steps/terrain.py:9 raised: 'AttributeError: 'str' object has no attribute 'append''

Traceback (most recent call last):
  File "/home/runner/.local/lib/python3.10/site-packages/radish/hookregistry.py", line 132, in call
    func(model, *args, **kwargs)
  File "/home/runner/.local/lib/python3.10/site-packages/terraform_compliance/steps/terrain.py", line 11, in load_terraform_data
    world.config.terraform = TerraformParser(world.config.user_data['plan_file'])
  File "/home/runner/.local/lib/python3.10/site-packages/terraform_compliance/extensions/terraform.py", line 59, in __init__
    self.parse()
  File "/home/runner/.local/lib/python3.10/site-packages/terraform_compliance/extensions/terraform.py", line 566, in parse
    self._mount_references()
  File "/home/runner/.local/lib/python3.10/site-packages/terraform_compliance/extensions/terraform.py", line 518, in _mount_references
    self._mount_resources(source=source_resources,
  File "/home/runner/.local/lib/python3.10/site-packages/terraform_compliance/extensions/terraform.py", line 327, in _mount_resources
    self.resources[target_resource]['values'][ref_type].append(resource)
AttributeError: 'str' object has no attribute 'append'

Error: Process completed with exit code 1.

The same Terraform code works fine in a different environment:

 11 features (3 passed, 8 skipped)
16 scenarios (5 passed, 11 skipped)
67 steps (21 passed, 11 skipped)
Run 1698058284 finished within a moment

and here it runs the scenarios as expected.

To Reproduce

Is there a secure location I can upload the plan to?

Tested Versions:

• terraform-compliance version: v1.3.43
• terraform version: v1.2.x & v1.5.4
• python version: 3.10.12

[jkrauze]

The scenario to reproduce the issue

File structure:

% tree terraform
terraform
├── main.tf
├── my-module
│   ├── module.tf
│   ├── my-data-module
│   │   └── output.tf
│   ├── my-nested-module
│   │   └── sg.tf
│   └── sg.tf
└── terraform.tf

test.zip

Lets say our terraform module uses a my-module module

main.tf

module "component" {
  source = "./my-module"
}

the my-module module creates its own security group using the my-data-module as a data source

my-module/sg.tf

module "data-common" {
  source = "./my-data-module"
}

resource "aws_security_group" "example_sg" {
  name        = "example_sg"
  description = "Example security group"

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = module.data-common.cidr_blocks
  }
}

also it uses the my-nested-module to create another security group

my-module/module.tf

module "nested-component" {
  source = "./my-nested-module"
}

my-module/my-nested-module/sg.tf

resource "aws_security_group" "example_sg_nested_host" {
  name        = "example_sg_nested_host"
  description = "Example security group nested"
}

resource "aws_security_group" "example_sg_nested_client" {
  name        = "example_sg_nested_client"
  description = "Example security group nested"
}

resource "aws_security_group_rule" "example_sg_nested_host_ingress_client" {
  source_security_group_id = aws_security_group.example_sg_nested_client.id
  security_group_id        = aws_security_group.example_sg_nested_host.id
  type                     = "ingress"
  from_port                = 443
  to_port                  = 443
  protocol                 = "tcp"
}

resource "aws_security_group_rule" "example_sg_nested_client_egress_host" {
  source_security_group_id = aws_security_group.example_sg_nested_host.id
  security_group_id        = aws_security_group.example_sg_nested_client.id
  type                     = "egress"
  from_port                = 443
  to_port                  = 443
  protocol                 = "tcp"
}

This setup causes the error described in the issue and a following warning

❗ WARNING (mounting): The reference "module.data-common" in resource aws_security_group.example_sg is ambiguous. It will not be mounted.

Observation

If we stop using my-data-module the issue disappears. After replacing the my-module/sg.tf file with a following content

my-module/sg.tf

resource "aws_security_group" "example_sg" {
  name        = "example_sg"
  description = "Example security group"

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

it starts working fine again.

[jkrauze]

By adding a debug log in the if statement here

cli/terraform_compliance/extensions/terraform.py

Lines 335 to 336 in e9f37e7

	if parameter not in self.resources[source_resource]['values']:
	self.resources[source_resource]['values'][parameter] = target_resource

like this

                    if parameter not in self.resources[source_resource]['values']:
                        self.resources[source_resource]['values'][parameter] = target_resource
                        defaults = Defaults()
                        console_write('{} {}: {}'.format(defaults.warning_icon,
                                    defaults.warning_colour('WARNING (test)'),
                                    defaults.info_colour('Injecting string into "{}" parameter... Source resource {}, target resource {} ref type "{}".'
                                                        ''.format(parameter, source_resource, target_resource, ref_type))))

and another one just before the failing line here

cli/terraform_compliance/extensions/terraform.py

Lines 325 to 326 in e9f37e7

	if ref_type not in self.resources[target_resource]['values']:
	self.resources[target_resource]['values'][ref_type] = []

like this

                    if ref_type in self.resources[target_resource]['values'] and not isinstance(self.resources[target_resource]['values'][ref_type], list):
                        defaults = Defaults()
                        console_write('{} {}: {}'.format(defaults.warning_icon,
                                    defaults.warning_colour('WARNING'),
                                    defaults.info_colour('Source resource {}, target resource {} ref type "{}" is not a list. Parameter: {} '
                                                         'The value is: "{}"'.format(source_resource, target_resource, ref_type, parameter, self.resources[target_resource]['values'][ref_type]))))

we can see that the failing run logs following warnings

❗ WARNING (mounting): The reference "module.data-common" in resource module.component.aws_security_group.example_sg is ambiguous. It will not be mounted.
❗ WARNING (test): Injecting string into "ingress" parameter... Source resource module.component.module.nested-component.aws_security_group.example_sg_nested_client, target resource module.component.aws_security_group.example_sg ref type "aws_security_group".
❗ WARNING (test): Injecting string into "ingress" parameter... Source resource module.component.module.nested-component.aws_security_group.example_sg_nested_host, target resource module.component.aws_security_group.example_sg ref type "aws_security_group".
❗ WARNING (test): Injecting string into "ingress" parameter... Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host, target resource module.component.aws_security_group.example_sg ref type "aws_security_group_rule".
❗ WARNING (test): Injecting string into "ingress" parameter... Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_host_ingress_client, target resource module.component.aws_security_group.example_sg ref type "aws_security_group_rule".
❗ WARNING (test): Injecting string into "security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host, target resource module.component.module.nested-component.aws_security_group.example_sg_nested_client ref type "egress".
❗ WARNING (test): Injecting string into "source_security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host, target resource module.component.module.nested-component.aws_security_group.example_sg_nested_host ref type "egress".
❗ WARNING (test): Injecting string into "security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group.example_sg_nested_client, target resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host ref type "aws_security_group".
❗ WARNING (test): Injecting string into "source_security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group.example_sg_nested_host, target resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host ref type "aws_security_group".
❗ WARNING: Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_host_ingress_client, target resource module.component.module.nested-component.aws_security_group.example_sg_nested_host ref type "ingress" is not a list. Parameter: security_group_id The value is: "module.component.aws_security_group.example_sg"

where the successful run logs following

❗ WARNING (test): Injecting string into "security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host, target resource module.component.module.nested-component.aws_security_group.example_sg_nested_client ref type "egress".
❗ WARNING (test): Injecting string into "source_security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host, target resource module.component.module.nested-component.aws_security_group.example_sg_nested_host ref type "egress".
❗ WARNING (test): Injecting string into "security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group.example_sg_nested_client, target resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host ref type "aws_security_group".
❗ WARNING (test): Injecting string into "source_security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group.example_sg_nested_host, target resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_client_egress_host ref type "aws_security_group".
❗ WARNING (test): Injecting string into "security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_host_ingress_client, target resource module.component.module.nested-component.aws_security_group.example_sg_nested_host ref type "ingress".
❗ WARNING (test): Injecting string into "source_security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_host_ingress_client, target resource module.component.module.nested-component.aws_security_group.example_sg_nested_client ref type "ingress".
❗ WARNING (test): Injecting string into "security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group.example_sg_nested_host, target resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_host_ingress_client ref type "aws_security_group".
❗ WARNING (test): Injecting string into "source_security_group_id" parameter... Source resource module.component.module.nested-component.aws_security_group.example_sg_nested_client, target resource module.component.module.nested-component.aws_security_group_rule.example_sg_nested_host_ingress_client ref type "aws_security_group".

We can quickly tell that when we're using the my-data-module the ingress parameter is injected into all resources from my-nested-module (including security groups) and this is probably what's causing the issue as on the subsequent runs we do not expect string here

cli/terraform_compliance/extensions/terraform.py

Line 328 in e9f37e7

self.resources[target_resource]['values'][ref_type].append(resource)