Adding AWS resources to your environment
Through the cloud-platform-environments repository, you can provision AWS resources for your environments. This is done using Terraform and more specifically, terraform modules the Cloud Platform team provides.
The documentation for the modules lives in each module’s repository.
Available modules
| Name | Description |
|---|---|
| CloudFront | Creates a CloudFront distribution to serve objects from S3 |
| Database Migration Service (DMS) | Creates a DMS replication instance to move data from another database to one inside Cloud Platform |
| DynamoDB cluster | Creates a non-global DynamoDB cluster |
| Elastic Container repository | Creates a container image repository |
| ElastiCache for Redis cluster | Creates a Redis cluster |
| Kubernetes: IAM role for service accounts (IRSA) | Creates an IAM role for a Kubernetes service account |
| Kubernetes: service account | Creates a Kubernetes service account, role, and rolebinding within a namespace |
| Kubernetes: service pod | Creates a pod in a namespace to access AWS services using the AWS CLI |
| OpenSearch | Creates an OpenSearch domain |
| Pushgateway | Creates a Prometheus Pushgateway |
| RDS Aurora cluster | Creates an RDS Aurora cluster |
| RDS database instance | Creates an RDS instance (Postgres, MySQL, MariaDB, MSSQL) |
| S3 bucket | Creates an S3 bucket |
| Secrets Manager | Creates and manages a secret in Secrets Manager |
| SNS topic | Creates an SNS topic |
| SQS queue | Creates an SQS queue |
| OpenSearch Cluster | Creates an OpenSearch Cluster |
| IRSA | IAM role and connected Kubernetes service account for use on the Cloud Platform. This is known as IRSA or IAM roles for service accounts. |
Usage
There are two ways to create Terraform resources for your namespace:
- Manually create a Terraform manifest.
- Using the cloud-platform CLI tool.
Manually create a Terraform manifest
In the cloud-platform-environments, cd into your namespace and create a new directory called resources (if it doesn’t already exist). Inside this directory create a file with a sensible filename i.e. rds.tf and create a call to the module you require.
For example, if you wish to create an RDS mssql instance or an RDS mysql instance or an RDS postgresql instance, go to the example in the module’s repository and populate your rds-.tf file using the example as a guide.
Use the cloud-platform cli
At time of writing, the following resources can be created by the CLI tool:
- rds
- s3
- ecr
- serviceaccount
Download the cloud-platform command line tool.
In the cloud-platform-environments, cd into your namespace and run the following command. This will create an RDS manifest in the resources directory:
cloud-platform environment rds create
Each example will have some global terraform configuration defined, however, this should only be declared once, regardless of the number of modules used:
terraform {
backend "s3" {}
}
provider "aws" {
region = "eu-west-1"
}
Additionally, some examples might define variables. Again, these should only be declared once per namespace:
variable "cluster_name" {}
variable "cluster_state_bucket" {}
The main README file of each module repository will list all the available configuration options that can be passed to the module.
Outputs
Each module will have its own outputs. These expose useful information, such as endpoints, credentials etc. The module examples all use a common approach: they employ the kubernetes_secret terraform resource to push the outputs straight into your namespace in the form of a Secret which you could then extract information from or directly reference in Pods.
This is currently the only supported way of accessing terraform outputs.
See this article for instructions on how to use these secrets in your applications.
Versioning
Only the latest version of a Cloud Platform terraform module may be used. A github action runs on each pr and will fail if the latest version isn’t used, you can check the failed action for the latest version.
The Cloud Platform team will upgrade all existing instances of a module, whenever a new version is released, as per this architecture decision record
Please check the version badge for the module you are using (visit the web page of the module’s github repository - the version badge will be just below the README heading), and make sure you are using the latest version of the module in your configuration, by specifying the ref attribute in the query string of the source URL:
module "my_module" {
source = "https://github.com/ministryofjustice/cloud-platform-terraform-ecr-credentials?ref=1.0"
}
Refer to the terraform documentation on modules for more information on usage.
Monitoring AWS Resources
All resources are monitored by AWS CloudWatch by default. We use CloudWatch Exporter to export a number of the metrics for the Terraform Modules we provide into Prometheus.
To view the current set of metrics available:
- Log into Prometheus
- Click on ‘insert metric at cursor’
- type ‘aws_’
You can view all metrics AWS make available here
Adding your own terraform to create AWS resources
If you need to create AWS resources that are not covered by the modules provided by the Cloud Platform team, you can contact us on the #ask-cloud-platform Slack channel or raise a support ticket to discuss your requirements.
There are some resources that we will not allow you to create, such as
Lambda functionswhich is discussed in the ADR- your own terraform module
If the decision is to allow you to create your own terraform, consider the following:
- all terraform code must be stored in the cloud-platform-environments repository
- iam policy permissions must be locked down to only allow access to those specific AWS resources
- the team who created those terraform code owns it and hence that team is responsible for maintaining it
- Cloud Platform team can provide console access if those resources support ABAC
Deploying the terraform code is the same as deploying the terraform modules provided by the Cloud Platform team. Create a PR to the cloud-platform-environments repository, make sure the checks are passed. Once the PR is merged, the concourse pipeline will run and deploy the terraform code.
Note: The pipeline may need permissions to create those resources. If that is the case, please contact us on the #ask-cloud-platform Slack channel