Skip to content
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/actions/build-website/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,15 @@ runs:
tag: v0.18.0
cache: enable

- name: Install Atmos
uses: cloudposse/github-action-setup-atmos@v2
with:
install-wrapper: false

- name: "Render Documentation for Terraform Components"
shell: bash
env:
PUBLIC_REPO_ACCESS_TOKEN: ${{ inputs.repo_access_token }}
PUBLIC_REPO_ACCESS_TOKEN: ${{ inputs.repo_access_token }}
run: |
./scripts/render-docs-for-components.sh

Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,7 @@ content/reference-architecture/**
!content/components/library/aws/_category_.json
!content/github-actions/library/actions/_category_.json
changelog/*

.atmos
.python-version
yarn.lock
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,9 @@ real-clean:

lint:
npx docusaurus-mdx-checker --cwd docs

readme/build:
@atmos docs generate readme

readme:
@atmos docs generate readme
50 changes: 39 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@

<!-- markdownlint-disable -->
# Developer Documentation <a href="https://cpco.io/homepage?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/docs&utm_content="><img align="right" src="https://cloudposse.com/logo-300x69.svg" width="150" /></a>
<a href="https://github.com/cloudposse/docs/releases"><img src="https://img.shields.io/github/release/cloudposse/docs.svg?style=for-the-badge" alt="Latest Release"/></a><a href="https://slack.cloudposse.com"><img src="https://slack.cloudposse.com/for-the-badge.svg" alt="Slack Community"/></a>


<a href="https://github.com/cloudposse/docs/releases"><img src="https://img.shields.io/github/release/cloudposse/docs.svg?style=for-the-badge" alt="Latest Release"/></a><a href="https://slack.cloudposse.com"><img src="https://slack.cloudposse.com/for-the-badge.svg" alt="Slack Community"/></a><a href="https://cloudposse.com/support/"><img src="https://img.shields.io/badge/Get_Support-success.svg?style=for-the-badge" alt="Get Support"/></a>


<!-- markdownlint-restore -->

<!--
Expand Down Expand Up @@ -55,17 +59,9 @@ make start



<!-- markdownlint-disable -->
## Makefile Targets
```text
Available targets:

help Help screen
help/all Display help for all targets
help/short This help short screen

```
<!-- markdownlint-restore -->



## Related Projects
Expand Down Expand Up @@ -100,6 +96,38 @@ In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow.

**NOTE:** Be sure to merge the latest changes from "upstream" before making a pull request!


## Running Terraform Tests

We use [Atmos](https://atmos.tools) to streamline how Terraform tests are run. It centralizes configuration and wraps common test workflows with easy-to-use commands.

All tests are located in the [`test/`](test) folder.

Under the hood, tests are powered by Terratest together with our internal [Test Helpers](https://github.com/cloudposse/test-helpers) library, providing robust infrastructure validation.

Setup dependencies:
- Install Atmos ([installation guide](https://atmos.tools/install/))
- Install Go [1.24+ or newer](https://go.dev/doc/install)
- Install Terraform or OpenTofu

To run tests:

- Run all tests:
```sh
atmos test run
```
- Clean up test artifacts:
```sh
atmos test clean
```
- Explore additional test options:
```sh
atmos test --help
```
The configuration for test commands is centrally managed. To review what's being imported, see the [`atmos.yaml`](https://raw.githubusercontent.com/cloudposse/.github/refs/heads/main/.github/atmos/terraform-module.yaml) file.

Learn more about our [automated testing in our documentation](https://docs.cloudposse.com/community/contribute/automated-testing/) or implementing [custom commands](https://atmos.tools/core-concepts/custom-commands/) with atmos.

### 🌎 Slack Community

Join our [Open Source Community](https://cpco.io/slack?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/docs&utm_content=slack) on Slack. It's **FREE** for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build totally *sweet* infrastructure.
Expand Down Expand Up @@ -137,7 +165,7 @@ All other trademarks referenced herein are the property of their respective owne


---
Copyright © 2017-2024 [Cloud Posse, LLC](https://cpco.io/copyright)
Copyright © 2017-2025 [Cloud Posse, LLC](https://cpco.io/copyright)


<a href="https://cloudposse.com/readme/footer/link?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/docs&utm_content=readme_footer_link"><img alt="README footer" src="https://cloudposse.com/readme/footer/img"/></a>
Expand Down
3 changes: 0 additions & 3 deletions README.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ description: |-

Let's jump right in! Here's how to get started with our documentation.

include:
- "docs/targets.md"

usage: |-
1. Build dependencies
```
Expand Down
12 changes: 12 additions & 0 deletions atmos.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Atmos Configuration — powered by https://atmos.tools
#
# This configuration enables centralized, DRY, and consistent project scaffolding using Atmos.
#
# Included features:
# - Organizational custom commands: https://atmos.tools/core-concepts/custom-commands
# - Automated README generation: https://atmos.tools/cli/commands/docs/generate
#

# Import shared configuration used by all modules
import:
- https://raw.githubusercontent.com/cloudposse/.github/refs/heads/main/.github/atmos/default.yaml
7 changes: 5 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions scripts/docs-collator/AbstractFetcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
DOCS_DIR = "docs"
TARGETS_MD = "targets.md"
README_YAML = "README.yaml"
ATMOS_YAML = "atmos.yaml"


class MissingReadmeYamlException(Exception):
Expand All @@ -29,3 +30,7 @@ def _fetch_docs(self, repo, module_download_dir, submodule_dir=""):
continue

self.github_provider.fetch_file(repo, remote_file, module_download_dir)

def _fetch_atmos_yaml(self, repo, module_download_dir):
self.github_provider.fetch_file(repo, ATMOS_YAML, module_download_dir)

8 changes: 8 additions & 0 deletions scripts/docs-collator/GitHubActionFetcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
ACTIONS_DIR = "actions"
README_YAML = "README.yaml"
README_MD = "README.md"
ACTION_YAML = "action.yml"


class GitHubActionFetcher(AbstractFetcher):
Expand All @@ -23,12 +24,19 @@ def fetch(self, repo):

self._fetch_readme_yaml(repo, repo_download_dir)

self._fetch_action_yaml(repo, repo_download_dir)

self._fetch_atmos_yaml(repo, repo_download_dir)

if DOCS_DIR in remote_files:
self._fetch_docs(repo, repo_download_dir)

if ACTIONS_DIR in remote_files:
self.__fetch_sub_actions(repo, repo_download_dir)

def _fetch_action_yaml(self, repo, module_download_dir):
self.github_provider.fetch_file(repo, ACTION_YAML, module_download_dir)

def __fetch_sub_actions(self, repo, module_download_dir):
remote_files = self.github_provider.list_repo_dir(repo, ACTIONS_DIR)

Expand Down
20 changes: 10 additions & 10 deletions scripts/docs-collator/GitHubActionRenderer.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@
from utils import rendering, templating

TARGETS_MD = "targets.md"
README_YAML = "README.yaml"
README_MD = "README.md"
ATMOS_YAML = "atmos.yaml"
INDEX_CATEGORY_JSON = "_category_.json"
README_TEMPLATE = "readme.md"
DOC_SUBFOLDER = "actions"

SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
Expand Down Expand Up @@ -45,25 +44,26 @@ def render(self, repo):
self._copy_extra_resources_for_docs(repo_download_dir, module_docs_dir)

def __render_readme(self, module_download_dir, module_docs_dir):
readme_yaml_file = os.path.join(module_download_dir, README_YAML)
readme_md_file = os.path.join(module_download_dir, README_MD)
readme_tmpl_file = os.path.join(TEMPLATES_DIR, README_TEMPLATE)
atmos_yaml_file = os.path.join(TEMPLATES_DIR, ATMOS_YAML)

io.create_dirs(module_docs_dir)

response = subprocess.run(
[
"make",
"atmos",
"--config",
f"{atmos_yaml_file}",
"docs",
"generate",
"readme",
f"README_TEMPLATE_FILE={readme_tmpl_file}",
f"README_FILE={readme_md_file}",
f"README_YAML={readme_yaml_file}",
f"README_TEMPLATE_YAML={readme_yaml_file}",
f"README_INCLUDES={module_download_dir}",
],
capture_output=True,
cwd=module_download_dir,
)

logging.info(response.stderr.decode("utf-8"))

if response.returncode != 0:
error_message = response.stderr.decode("utf-8")
raise TerraformDocsRenderingError(error_message)
Expand Down
4 changes: 4 additions & 0 deletions scripts/docs-collator/ModuleFetcher.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import os

from AbstractFetcher import AbstractFetcher, MissingReadmeYamlException
import logging

DOCS_DIR = "docs"
IMAGES_DIR = "images"
Expand Down Expand Up @@ -56,12 +57,15 @@ def __fetch_submodules(self, repo, module_download_dir):
for readme_file in readme_files.values():
self.github_provider.fetch_file(repo, readme_file, module_download_dir)
if os.path.basename(readme_file) == README_YAML:
logging.info(f"Fetching submodule docs for: {os.path.dirname(readme_file)}")
self._fetch_docs(
repo,
module_download_dir,
submodule_dir=os.path.dirname(readme_file),
)

self.__fetch_terraform_files(repo, module_download_dir, remote_files)

def __fetch_terraform_files(self, repo, module_download_dir, remote_files):
for remote_file in remote_files:
# We only need variables and output files to render terraform-docs.
Expand Down
38 changes: 18 additions & 20 deletions scripts/docs-collator/ModuleRenderer.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,10 @@
DOCS_DIR = "docs"
IMAGES_DIR = "images"
SUBMODULES_DIR = "modules"
TARGETS_MD = "targets.md"
ATMOS_YAML = "atmos.yaml"
README_YAML = "README.yaml"
README_MD = "README.md"
INDEX_CATEGORY_JSON = "_category_.json"
MODULES_README_TEMPLATE = "readme.md"

SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
TEMPLATES_DIR = os.path.join(SCRIPT_DIR, "templates/modules")
Expand Down Expand Up @@ -62,42 +61,41 @@ def render(self, repo):
def __render_readme(self, module_download_dir, module_docs_dir):
readme_yaml_file = os.path.join(module_download_dir, README_YAML)
readme_md_file = os.path.join(module_download_dir, README_MD)
readme_tmpl_file = os.path.join(TEMPLATES_DIR, MODULES_README_TEMPLATE)
atmos_yaml_file = os.path.join(TEMPLATES_DIR, ATMOS_YAML)

io.create_dirs(module_docs_dir)

# Re-render terraform docs with this repo's terraform-docs template for modules.
# This replaces docs/terraform.md for the given module in place
logging.debug(f"Rendering terraform docs for: {module_download_dir}")
rendering.render_terraform_docs(
module_download_dir, os.path.join(TEMPLATES_DIR, "terraform-docs.yml")
)

# Run the make readme command in the module directory to compile README.md
logging.debug(f"Rendering README.md for: {module_download_dir}")
logging.debug(f"make readme")
logging.debug(f"README_TEMPLATE_FILE: {readme_tmpl_file}")
logging.debug(f"atmos docs generate readme --config {atmos_yaml_file}")
logging.debug(f"ATMOS_YAML_FILE: {atmos_yaml_file}")
logging.debug(f"README_FILE: {readme_md_file}")
logging.debug(f"README_YAML: {readme_yaml_file}")
logging.debug(f"README_TEMPLATE_YAML: {readme_yaml_file}")
logging.debug(f"README_INCLUDES: {module_download_dir}")
logging.debug(f"README_MD: {readme_md_file}")
response = subprocess.run(
[
"make",
"atmos",
"--config",
f"{atmos_yaml_file}",
"docs",
"generate",
"readme",
f"README_TEMPLATE_FILE={readme_tmpl_file}",
f"README_FILE={readme_md_file}",
f"README_YAML={readme_yaml_file}",
f"README_TEMPLATE_YAML={readme_yaml_file}",
f"README_INCLUDES={module_download_dir}",
],
capture_output=True,
cwd=module_download_dir,
)

if response.returncode != 0:
error_message = response.stderr.decode("utf-8")
raise TerraformDocsRenderingError(error_message)

# Re-render terraform docs with this repo's terraform-docs template for modules.
# This replaces docs/terraform.md for the given module in place
logging.debug(f"Rendering terraform docs for: {module_download_dir}")
rendering.render_terraform_docs(
module_download_dir, os.path.join(TEMPLATES_DIR, "terraform-docs.yml")
)

logging.info(f"Rendered: {readme_md_file}")

def __copy_extra_resources_for_images(self, module_download_dir, module_docs_dir):
Expand Down
1 change: 1 addition & 0 deletions scripts/docs-collator/component/fetcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ def fetch(self, repo):

self._fetch_readme_yaml(repo, module_download_dir)
self._fetch_readme_md(repo, module_download_dir)
self._fetch_atmos_yaml(repo, module_download_dir)

if CHANGELOG_MD in remote_files:
self._fetch_changelog_md(repo, module_download_dir)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
docs:
generate:
readme:
base-dir: .
input:
- "./README.yaml"
- get_support: true
template: "../../../scripts/docs-collator/component/renderer/templates/components/yml/readme.md"
output: "./README.md"
terraform:
source: ./src
enabled: true
format: "markdown table"
show_providers: false
show_inputs: true
show_outputs: true
sort_by: "name"
hide_empty: false
indent_level: 2
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
{{- defineDatasource "config" .Env.README_YAML | regexp.Replace ".*" "" -}}
{{- defineDatasource "includes" .Env.README_INCLUDES -}}
{{- $deprecated := has (ds "config") "deprecated" -}}
{{- $fullModuleName := (ds "config").name -}}
{{- $shortModuleName := ( conv.Default $fullModuleName (conv.Join (coll.GoSlice ($fullModuleName | strings.SplitN "-" 3) 1) "-")) -}}
---
title: {{ $fullModuleName }}
sidebar_label: {{ $fullModuleName }}
Expand Down Expand Up @@ -71,6 +69,14 @@ This module is no longer actively maintained
{{(ds "config").examples }}
{{ end }}

{{ if has (ds "config") "terraform_docs" }}
<!-- markdownlint-disable -->
<!-- BEGIN_TF_DOCS -->
{{ (ds "config").terraform_docs }}
<!-- END_TF_DOCS -->
<!-- markdownlint-restore -->
{{ end }}

{{ if has (ds "config") "include" }}
{{ range $file := (datasource "config").include -}}
{{ (include "includes" $file) }}
Expand Down
Loading
Loading