Terraform AWS Large Language Model (LLM) Infrastructure

The main purpose of this repository is to create an AWS EC2 instance that will run a large language model (LLM) using an the Ollama server. Along side other resources such as AWS App Runner.

Development

Dependencies

• aws-vault
• terraform
• terragrunt
• terraform-docs this is required for terraform_docs hooks
• pre-commit

Prerequisites

1. Have a AWS account account and associated credentials.
2. You may need to request a service quote increase for example for AWS EC2 you will not be able to deploy resources until the request has been completed by AWS:
   • Running On-Demand G and VT instances
   • All G and VT Spot Instance Requests

Usage

1. Navigate to the environment you would like to deploy,

2. Create the S3 bucket for your Terraform state file:

   aws-vault exec <profile> --no-session terragrunt backend bootstrap

3. Initialize the configuration with:

   aws-vault exec <profile> --no-session terragrunt init

4. Plan your changes with:

   aws-vault exec <profile> --no-session terragrunt plan

5. If you're happy with the changes:

   aws-vault exec <profile> --no-session terragrunt apply

Note

Please note that terragrunt will create an S3 Bucket and DynamoDB table for storing the remote state. Ensure the account deploying the resources has the appropriate permissions to create or connect to these resources.

Pre-Commit hooks

Git hook scripts are very helpful for identifying simple issues before pushing any changes. Hooks will run on every commit automatically pointing out issues in the code e.g. trailing whitespace.

To help with the maintenance of these hooks, pre-commit is used, along with pre-commit-hooks.

Please following these instructions to install pre-commit locally and ensure that you have run pre-commit install to install the hooks for this project.

Additionally, once installed, the hooks can be updated to the latest available version with pre-commit autoupdate.

Documentation Generation

Code formatting and documentation for variables and outputs is generated using pre-commit-terraform hooks that in turn uses terraform-docs that will insert/update documentation. The following markers have been added to the README.md:

<!-- {BEGINNING|END} OF PRE-COMMIT-TERRAFORM DOCS HOOK --->

Requirements

Name	Version
terraform	>= 1.5.7, <= 1.13.4
aws	~> 6.19.0
http	~> 3.5.0
tls	~> 4.1.0

Providers

Name	Version
aws	6.19.0
http	3.5.0
tls	4.1.0

Modules

No modules.

Resources

Name	Type
aws_instance.ollama_instance	resource
aws_internet_gateway.main_igw	resource
aws_key_pair.ollama_developer	resource
aws_route_table.main_public_route_table	resource
aws_route_table_association.main_route_table_public_assoc	resource
aws_security_group.sg_ollama_server	resource
aws_subnet.main_subnet	resource
aws_vpc.main_vpc	resource
aws_vpc_security_group_egress_rule.allow_all_developer_nvidia_com	resource
aws_vpc_security_group_egress_rule.allow_all_github_com	resource
aws_vpc_security_group_egress_rule.allow_all_github_user_releases	resource
aws_vpc_security_group_egress_rule.allow_ssh_tcp_https	resource
aws_vpc_security_group_ingress_rule.allow_ollama_server_communication_for_users	resource
aws_vpc_security_group_ingress_rule.allow_ssh_tcp_for_users	resource
tls_private_key.ollama_developer_ssh_key	resource
aws_availability_zones.available_zones	data source
aws_caller_identity.current_caller_identity	data source
http_http.aws_check_ip	data source

Inputs

Name	Description	Type	Default	Required
additional_developer_access_ip_addresses	Map of developer name and their IP address to access various resources.	map(string)	{}	no
allowed_account_ids	List of allowed AWS account IDs to prevent you from mistakenly using an incorrect one.	list(string)	n/a	yes
aws_region	The AWS region.	string	n/a	yes
env_prefix	The prefix added to resources in the environment.	string	n/a	yes
project_name	The name of the project.	string	n/a	yes

Outputs

Name	Description
availability_zones	List of the Availability Zone names available to the account.
current_caller_identity	AWS Account ID number of the account that owns or contains the calling entity.
ec2_ollama_server_instance_public_dns	Public DNS name assigned to the instance.
ollama_developer_key_pair_name	The key pair name.
tls_ollama_developer_private_key	Private key data in PEM (RFC 1421) format for connecting to the EC2 instance hosting the Ollama server.
tls_ollama_developer_public_key	Public key data in PEM (RFC 1421) format for connecting to the EC2 instance hosting the Ollama server.