|
| 1 | +--- |
| 2 | +title: "Concepts" |
| 3 | +description: "Learn more about the core concepts and domain model that make up the SweetOps methodology." |
| 4 | +weight: 3 |
| 5 | +--- |
| 6 | + |
| 7 | +SweetOps is built on top of a number of high-level concepts and terminology that are critical to understanding prior to getting started. In this document, we break down these concepts to help you get a leg up prior to going through your first tutorial. |
| 8 | + |
| 9 | +## Layers |
| 10 | + |
| 11 | + |
| 12 | +SweetOps breaks down cloud infrastructure into 4 discrete layers: |
| 13 | + |
| 14 | +1. **Foundational**: Your AWS Accounts, VPCs, IAM, and DNS architecture. |
| 15 | +1. **Platform**: Your EKS / ECS Clusters, Load Balancers, IAM Service Accounts, and Certificates. |
| 16 | +1. **Shared Services**: Your CI/CD pipelines, BeyondCorp solution, and Monitoring and Logging tooling. |
| 17 | +1. **Application**: Your Backend and Frontend Applications. |
| 18 | + |
| 19 | +We delineate between these layers as they have different Software Development Life Cycles (SDLC) and tools that are responsible for managing them. For example, Terraform is great for building your Foundational layer, but other tools might be better for managing the continuous delivery of the Application layer (e.g. ArgoCD). It's also important to note that each layer builds on the previous. The lower layers are less likely to change over time. At the bottom, you won't frequently change your AWS accounts and VPCs in your Foundational layer, just like when operating a platform you won't be rebuilding EKS clusters in your Platform layer without disrupting all tenants of the platform. You might, however, add HashiCorp Vault to your Shared Services layer to increase your security posture or add a new API microservice to your Application layer. The Application Layer is ultimately the most important layer of them all: it's what drives your business. It's where your applications live, which is why it will change continuously. |
| 20 | + |
| 21 | +## Components |
| 22 | + |
| 23 | +Components are opinionated, self-contained units of infrastructure as code that solve one, specific problem or use-case. SweetOps has two flavors of components: |
| 24 | + |
| 25 | +1. **Terraform:** Stand-alone root modules that implement some piece of your infrastructure. For example, typical components might be an EKS cluster, RDS cluster, EFS filesystem, S3 bucket, DynamoDB table, etc. You can find the [full library of SweetOps Terraform components on GitHub](https://github.com/cloudposse/terraform-aws-components). |
| 26 | +1. **Helmfiles**: Stand-alone, applications deployed using `helmfile` to Kubernetes. For example, typical helmfiles might deploy the DataDog agent, cert-manager controller, nginx-ingress controller, etc. Similarly, the [full library of SweetOps Helmfile components is on GitHub](https://github.com/cloudposse/helmfiles). |
| 27 | + |
| 28 | +One important distinction about components that is worth noting: components are opinionated "root" modules that typically call other child modules. Components are the building-blocks of your infrastructure. This is where you define all the business logic for how to provision some common piece of infrastructure like ECR repos or EKS clusters. Our convention is only stick components in the `components/terraform` directory and to use `modules/` when referring to child modules intended to be called by other components. We do not recommend consuming one terraform component inside of another as that would defeat the purpose; each component is intended to be a loosely coupled unit of IaC with its own lifecycle. |
| 29 | + |
| 30 | +## Stacks |
| 31 | + |
| 32 | +Stacks are a way to express the complete infrastructure needed for an environment using a standard YAML configuration format that has been developed by SweetOps. Stacks consist of components and the variables inputs into those components. For example, you configure a stack for each AWS account and then reference the components which comprise that stack. The more modular the components, the easier it is to quickly define a stack without writing any new code. |
| 33 | + |
| 34 | +Here is an example stack defined for a Dev environment in the us-west-2 region: |
| 35 | + |
| 36 | +```yaml |
| 37 | +# Filename: stacks/uw2-dev.yaml |
| 38 | +import: |
| 39 | + - eks/eks-defaults |
| 40 | + |
| 41 | +vars: |
| 42 | + stage: dev |
| 43 | + |
| 44 | +terraform: |
| 45 | + vars: {} |
| 46 | + |
| 47 | +helmfile: |
| 48 | + vars: |
| 49 | + account_number: "1234567890" |
| 50 | + |
| 51 | +components: |
| 52 | + terraform: |
| 53 | + |
| 54 | + dns-delegated: |
| 55 | + vars: |
| 56 | + request_acm_certificate: true |
| 57 | + zone_config: |
| 58 | + - subdomain: dev |
| 59 | + zone_name: example.com |
| 60 | + |
| 61 | + vpc: |
| 62 | + vars: |
| 63 | + cidr_block: "10.122.0.0/18" |
| 64 | + |
| 65 | + eks: |
| 66 | + vars: |
| 67 | + cluster_kubernetes_version: "1.19" |
| 68 | + region_availability_zones: ["us-west-2b", "us-west-2c", "us-west-1d"] |
| 69 | + public_access_cidrs: ["72.107.0.0/24"] |
| 70 | + |
| 71 | + aurora-postgres: |
| 72 | + vars: |
| 73 | + instance_type: db.r4.large |
| 74 | + cluster_size: 2 |
| 75 | + |
| 76 | + mq-broker: |
| 77 | + vars: |
| 78 | + apply_immediately: true |
| 79 | + auto_minor_version_upgrade: true |
| 80 | + deployment_mode: "ACTIVE_STANDBY_MULTI_AZ" |
| 81 | + engine_type: "ActiveMQ" |
| 82 | + |
| 83 | + helmfile: |
| 84 | + |
| 85 | + external-dns: |
| 86 | + vars: |
| 87 | + installed: true |
| 88 | + |
| 89 | + datadog: |
| 90 | + vars: |
| 91 | + installed: true |
| 92 | + datadogTags: |
| 93 | + - "env:uw2-dev" |
| 94 | + - "region:us-west-2" |
| 95 | + - "stage:dev" |
| 96 | +``` |
| 97 | +
|
| 98 | +Great, so what can you do with a stack? Stacks are meant to be a language and tool agnostic way to describe infrastructure, but how to use the stack configuration is up to you. SweetOps provides the following ways to utilize stacks today: |
| 99 | +
|
| 100 | +1. [atmos](https://github.com/cloudposse/atmos): atmos is a command-line tool that enables CLI-driven stack utilization and supports workflows around `terraform`, `helmfile`, and many other commands. |
| 101 | +1. [Terraform Cloud](https://www.terraform.io/docs/cloud/index.html): By using the [terraform-tfe-cloud-infrastructure-automation module](https://github.com/cloudposse/terraform-tfe-cloud-infrastructure-automation) you can provision Terraform Cloud workspaces for each component in your stack using Continuous Delivery and GitOps. |
| 102 | +1. [Spacelift](https://spacelift.io/): By using the [terraform-spacelift-cloud-infrastructure-automation module](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation) you can provision Spacelift stacks (our industry loves this word, huh?) for each component in your stack using Continuous Delivery and GitOps. |
| 103 | + |
| 104 | +## Catalogs |
| 105 | + |
| 106 | +Catalogs in SweetOps are collections of sharable and reusable configurations. These are typically YAML configurations that can be imported and provide solid baselines to configure security, monitoring, or other 3rd party tooling. Catalogs enable an organization to codify their best practices of configuration and share them. SweetOps provides many catlaogs to get you started. |
| 107 | + |
| 108 | +Today SweetOps provides a couple important catalogs: |
| 109 | + |
| 110 | +1. [DataDog Monitors](https://github.com/cloudposse/terraform-datadog-monitor/tree/master/catalog/monitors): Quickly bootstrap your SRE efforts by utilizing some of these best practice DataDog application monitors. |
| 111 | +1. [AWS Config Rules](https://github.com/cloudposse/terraform-aws-config/tree/master/catalog): Quickly bootstrap your AWS compliance efforts by utilizing hundreds of [AWS Config](https://aws.amazon.com/config/) rules that automate security checks against many common services. |
| 112 | +1. [AWS Service Control Policies](https://github.com/cloudposse/terraform-aws-service-control-policies/tree/master/catalog): define what permissions in your organization you want to permit or deny in member accounts. |
| 113 | + |
| 114 | +In the future, you're likely to see additional open-source catalogs for OPA rules and tools to make sharing configurations even easier. But it is important to note that how you use catalogs is really up to you to define, and the best catalogs will be specific to your organization. |
| 115 | + |
| 116 | +## Primary vs Delegated |
| 117 | + |
| 118 | +Primary vs Delegated is a common implementation pattern in SweetOps. This is most easily described when looking at the example of domain and DNS usage in a mutli-account AWS organization: SweetOps takes the approach that the root domain (e.g. `example.com`) is owned by a **primary** AWS account where the apex zone resides. Subdomains on that domain (e.g. `dev.example.com`) are then **delegated** to the other AWS accounts via an `NS` record on the primary hosted zone which points to the delegated hosted zone's name servers. |
| 119 | + |
| 120 | +You can see examples of this pattern in the [dns-primary](https://github.com/cloudposse/terraform-aws-components/tree/master/modules/dns-primary) / [dns-delegated](https://github.com/cloudposse/terraform-aws-components/tree/master/modules/dns-delegated) and [iam-primary-roles](https://github.com/cloudposse/terraform-aws-components/tree/master/modules/iam-primary-roles) / [iam-delegated-roles](https://github.com/cloudposse/terraform-aws-components/tree/master/modules/iam-delegated-roles) components. |
| 121 | + |
| 122 | +## Docker Based Infrastructure Development |
| 123 | + |
| 124 | +In the landscape of developing infrastructure, there are dozens of tools that we all need on our personal machines to do our jobs. In SweetOps, instead of having you install each tool individually, we use Docker to package all of these tools into one image that you can use as your infrastructure automation toolbox: [Geodesic]({{< relref "reference/tools.md#geodesic" >}}). We use `geodesic` as the base Docker image of our `Dockerfile` for all our DevOps tooling. |
| 125 | + |
| 126 | +Geodesic is a DevOps Linux Distribution packaged as a Docker image that provides users the ability to utilize atmos, terraform, kubectl, helmfile, AWS CLI, and many other popular tools that compromise the SweetOps methodology without having to invoke a dozen `install` commands to get started. It's intended to be used as an interactive cloud automation shell, a base image, or in CI / CD scripting to ensure that all systems are running the same set of versioned, easily accessible tools. |
| 127 | + |
| 128 | +<!-- TODO: Link to How-to on "Using Geodesic" once created. --> |
0 commit comments