Accessing your RDS database
When you create an RDS instance using the RDS module, it is created inside a virtual private cloud (VPC), which will only accept network connections from within the kubernetes cluster. So, trying to connect to the RDS instance from your local machine will not work.
+--------------+ outside \ / inside +--------------+
| Your machine | -------------------X-----------------------> | RDS instance |
+--------------+ / \ +--------------+
VPC Boundary
If you need to access your database from outside the cluster (e.g. from your own development machine, or to perform a bulk data import), you can do so via the following steps:
- Run a pod inside the cluster to forward network traffic to your RDS instance
- Tell kubernetes to forward traffic from your local machine to the new pod
- Access the database as if it were running on your local machine
So, the connection from your machine to the RDS instance works like this:
+--------------+ +---------------------+ +--------------+
| Your machine |------------>| Port forwarding pod |--------->| RDS instance |
+--------------+ +---------------------+ +--------------+
1. Run a port-forward pod
There are several docker images designed to forward network traffic, but you need one which does not run as root.
The cluster pod security policy (PSP) will prevent any images from running a process as the
rootuser, so docker images which expect to do this will not work.
We will use ministryofjustice/port-forward for this example.
kubectl \
-n [your namespace] \
run port-forward-pod \
--image=ministryofjustice/port-forward \
--port=5432 \
--env="REMOTE_HOST=[your database hostname]" \
--env="LOCAL_PORT=5432" \
--env="REMOTE_PORT=5432"
Use port 1433 for MS-SQL.
Use port 3306 for My-SQL.
This creates a pod in your namespace, but without a deployment to manage it.
Manually-created pods (i.e. those which are not part of a deployment) will always be eventually deleted by regular cluster maintainance operation - node-recycling, which runs every weekday between midnight and 3 am. Based on our cluster node count at time of writing, it can take up to 15 days for every node to be recycled and thus manually deployed pods may exist this long before being removed during a node recycle.
If you need a port-forward pod to be persistent, please make it part of a deployment.
2. Forward local traffic to the port-forward-pod
Now you need to forward network traffic from a port on your local machine to the port-forward pod inside the cluster.
kubectl \
-n [your namespace] \
port-forward \
port-forward-pod 5432:5432
Use port 1433 for MS-SQL.
Use port 3306 for My-SQL.
You need to leave this running as long as you are accessing the database.
3. Access the database
Now you can connect to the database as if it were running locally, on your machine.
If you are exporting a database URL from your RDS kubernetes secret, it might have a value like this:
postgres://cpDvquXO5B:R1eDN0xEUnaH6Aqr@cloud-platform-df3589e0e7acba37.cdwm328dlye6.eu-west-2.rds.amazonaws.com:5432/dbdf3589e0e7acba37
For MS-SQL, use sqlcmd for MacOS and test with a command like
sqlcmd -S tcp:127.0.0.1,1433 -U $user -P $pass -Q 'select @@version'
For My-SQL, use mysql for MacOS and test with a command like below - note there is no space between parameter and value
mysql -h127.0.0.1 -P3306 -u$user -p$pass
You can use this URL to connect to your database via the port forward you have set up, but you need to replace the database hostname, (cloud-platform-df3589e0e7acba37.cdwm328dlye6.eu-west-2.rds.amazonaws.com in this example), with localhost.
So, if you were starting from the database URL above, the database connection command you will run on your local machine would be:
psql postgres://cpDvquXO5B:R1eDN0xEUnaH6Aqr@localhost:5432/dbdf3589e0e7acba37
If you are exporting the database credentials separately, the command would be something like this:
psql \
--host localhost \
--port 5432 \
--dbname [your database name] \
--username [your database username] \
--password
(You will be prompted to enter the database password, which you should get (and then base64 decode) from your kubernetes secret)
4. Delete the port-forward pod
Please remember to delete the port-forwarding pod when you have finished.
kubectl delete pod port-forward-pod -n [your namespace]