tflint installation failing on GitLab CI

[mahela-aws]

Summary

tflint installation failing with below error

Unpacking /tmp/tflint.XXXXJBKDMj/tflint.zip ...
unzip:  cannot find or open /tmp/tflint.XXXXJBKDMj/tflint.zip, /tmp/tflint.XXXXJBKDMj/tflint.zip.zip or /tmp/tflint.XXXXJBKDMj/tflint.zip.ZIP.
Installing /tmp/tflint.XXXXJBKDMj/tflint to /usr/local/bin/ ...
install: can't stat '/tmp/tflint.XXXXJBKDMj/tflint': No such file or directory
Failed to install tflint

we are using this command to install tflint

curl -s https://raw.githubusercontent.com/terraform-linters/tflint/master/install_linux.sh | bash

Command

curl -s https://raw.githubusercontent.com/terraform-linters/tflint/master/install_linux.sh | bash

Terraform Configuration

this doesn't matter

TFLint Configuration

rule "aws_resource_missing_tags" {
  enabled = true
  tags = [
    "Name",
    "Terraform",
  ]
}

Output

Unpacking /tmp/tflint.XXXXJBKDMj/tflint.zip ...
unzip:  cannot find or open /tmp/tflint.XXXXJBKDMj/tflint.zip, /tmp/tflint.XXXXJBKDMj/tflint.zip.zip or /tmp/tflint.XXXXJBKDMj/tflint.zip.ZIP.
Installing /tmp/tflint.XXXXJBKDMj/tflint to /usr/local/bin/ ...
install: can't stat '/tmp/tflint.XXXXJBKDMj/tflint': No such file or directory
Failed to install tflint

### TFLint Version

0.47.0

### Terraform Version

1.3.7

### Operating System

- [X] Linux
- [ ] macOS
- [ ] Windows

[wata727]

I haven't been able to reproduce this.

$ docker run --rm -it alpine:3.16
/ # apk add curl
/ # apk add bash
/ # curl -s https://raw.githubusercontent.com/terraform-linters/tflint/master/install_linux.sh | bash
arch=amd64
os=linux_amd64


====================================================
Looking up the latest version ...
Downloading TFLint v0.47.0
Downloaded successfully


====================================================
Unpacking /tmp/tflint.XXXXkmMjBg/tflint.zip ...
Archive:  /tmp/tflint.XXXXkmMjBg/tflint.zip
  inflating: tflint
Installing /tmp/tflint.XXXXkmMjBg/tflint to /usr/local/bin/ ...
'/tmp/tflint.XXXXkmMjBg/tflint' -> '/usr/local/bin/tflint'
Cleaning temporary downloaded files directory /tmp/tflint.XXXXkmMjBg ...


====================================================
Current tflint version
TFLint version 0.47.0
+ ruleset.terraform (0.4.0-bundled)

[mahela-aws]

@wata727 we're using this in a gitlab pipeline not the way you were trying to reproduce it. not sure if you try to reproduce this in a gitlab pipeline. we're using our own aws runners, but you can test this on shared runners if you can

[bendrucker]

No we definitely can't get involved with running things in GitLab to try to diagnose this for you, especially for commercial use. We'll need you to either find a way to reproduce this in a container or fully debug this yourself and contribute a fix.

Based on the output unzip can't find the file. curl should bail on a server error with --fail so it's hard to say what's happening. Unless mktemp is somehow behaving unexpectedly in your environment.

Either way, the next steps for you are to fork the installation script and start adding debug statements (e.g., using ls to print directory contents) to figure out where things are going wrong.

[mahela-aws]

@bendrucker we were able to find out the issue here, we are doing this install multiple times within our gitlab pipeline in different steps in the gitlab runner environment. It appears this is causing troubles. We managed to find a way to do the installation single time within the pipeline. will monitor it for sometimes and see how it goes

[bendrucker]

Ok, interesting, that sounds like there might be a bug or at least a feature opportunity to be more aggressive about erroring when the install script fails due to conflicts. That or we should ensure conflict avoidance using temporary directories, etc., such that the script is idempotent.

[bendrucker]

I'm still unable to reproduce this. If you are able to open a PR with a failing test, I'm happy to continue looking at this. Otherwise, I'm out of ideas.

https://github.com/terraform-linters/tflint/actions/runs/5490479959/jobs/10005974411?pr=1801

That or we should ensure conflict avoidance using temporary directories

This is already what's happening. By always downloading to a unique temporary directory, there should be no conflict between runs.

So I'm still no closer to any indication of why your ZIP file might be missing.

[mahela-aws]

we were under the impression it's getting downloaded to the same tmp directory. so it looks like it wasn't the case. these failures weren't consistent but intermittent. no idea what's happening

[bendrucker]

Ok, I'm happy to look at a public GitLab reproduction but without that I don't think there's anything else to discuss/try here.