Skip to main content

How to use kubectl to connect to the cluster

This is a guide to installing and configuring kubectl, the official command-line tool for Kubernetes.

kubectl is used to deploy and manage applications on Kubernetes. Cloud Platform users will often use it, in addition to the Cloud Platform CLI.

After working through this guide you should be able to use kubectl to query and update all the kubernetes resources in your namespace(s) on the Cloud Platform.

Installation

To install kubectl, please follow the official docs: Install kubectl.

Kubectl version

Choose a kubectl version that is within one minor version difference of your cluster. For example, kubectl v1.20 should work with a Kubernetes cluster at v1.19, v1.20, and v1.21. (Although in practice there are rarely problems with even newer kubectl versions.)

You can check the current cluster version here:

Authentication

Next you’ll need to ensure your GitHub user is in the right organization and team, and then do authentication.

GitHub setup

The Cloud Platform uses GitHub for authentication and authorisation:

  1. Your GitHub user must be a member of the Ministry of Justice organisation. See: MOJ Technical Guidance - Member of the MoJ organisation (Note: you can only join if your user has 2FA enabled and your ...@digital.justice.gov.uk email address has been added to it)

  2. Access to each Cloud Platform environment/namespace (e.g. pvb-production, cla-staging) is granted according to GitHub team membership (e.g. ‘PVB’ and ‘CLA’ teams). To check which team is authorized for a particular environment, look in the 00-namespace.yaml e.g. 00-namespace.yaml

Get a kubeconfig file

You’ll login to the cluster using GitHub, and it will provide you with a token file, which you’ll save on your machine as ~/.kube/config

For those interested in how it works: GitHub is being used as an OIDC provider. Once you’ve logged in to GitHub, it provides a token, which is a signed JWT containing your GitHub username and list of teams you’re in. GitHub passes it to Auth0 (OIDC broker), which passes it to Cloud Platform’s login/Kuberos app, which provides it to you in a ‘kubeconfig’ file, containing the cluster API URL and token. This kubeconfig will be used when you run kubectl, to authenticate with the Kubernetes cluster in Cloud Platform.

  1. Browse to the login URL:

  2. Select ‘Sign in with GitHub’. (This page is hosted by Auth0.com)

  3. You’re redirected to github.com and will need to login, if you’ve not done so already.

  4. GitHub asks if ‘MOJ Cloud Platforms Auth0’ may access your GitHub user details. Select “Authorize”.

    GitHub Authorize MoJ

  5. Auth0.com asks if ‘live:kubernetes’ app may access ‘justice-cloud-platform account’. Select “Accept”.

  6. Select ‘Download Config File’.

  7. Move the downloaded file to directory and rename the file to config ~/.kube/config:

    mv kubecfg.yaml ~/.kube/config
    

    The above command will rename the kubecfg.yaml file to config and move it into the ~/.kube directory.

    If you’ve already got a kubeconfig, you may want to replace it, or merge with it - see Using multiple kubeconfigs.

  8. Minimize the permissions on the file:

    chmod 600 ~/.kube/config
    
  9. Tell kubectl to use this config:

    • live cluster:

      kubectl config use-context live.cloud-platform.service.justice.gov.uk
      

You should now be able to run kubectl commands. Try running kubectl get namespaces to list the namespaces in the cluster.

Troubleshooting: “current” context

If you see an error like this when executing kubectl commands:

The connection to the server localhost:8080 was refused

A common reason for this is that the “current” context has not been set. Check it by running:

kubectl config get-contexts

If current context is set correctly, you’ll see a *, like this:

CURRENT   NAME                                         CLUSTER                                      AUTHINFO                 NAMESPACE
*         live.cloud-platform.service.justice.gov.uk   live.cloud-platform.service.justice.gov.uk   <your github e-mail>

If the * is missing, then you need to set the context with the use-context command mentioned above.

Using multiple kubeconfigs

You can have two kubeconfig files, corresponding to different clusters, which is useful when working on multiple clusters. This is a guide to being able to manage multiple kubeconfigs, so you can conveniently switch the cluster that kubectl connects to.

Essentially we will merge the kubeconfigs into a single file, and use kubectl config use-context to switch between them.

  1. Assuming you already have a kubeconfig for a test cluster, download another kubeconfig for live from login.cloud-platform.service.justice.gov.uk.

  2. Move the kubeconfig to your .kube folder:

    mv ~/Downloads/kubecfg.yaml ~/.kube/config-live
    
  3. Edit ~/.kube/config-live, and change the username in two places, from <firstname>.<lastname>@digital.justice.gov.uk to <firstname>.<lastname>@live, as shown below:

    contexts:
    - context:
        cluster: live.cloud-platform.service.justice.gov.uk
        user: <firstname>.<lastname>@live
    name: live.cloud-platform.service.justice.gov.uk
    current-context: ""
    kind: Config
    preferences: {}
    users:
    - name: <firstname>.<lastname>@live
    

    Explanation: If we don’t rename the username, then the merge will overwrite one of the them and their token, which is different between the clusters.

  4. Copy your existing test cluster credentials into a separate file:

    cp ~/.kube/config ~/.kube/config-test
    
  5. Edit ~/.kube/config-test, and change the username in two places, from <firstname>.<lastname>@digital.justice.gov.uk to <firstname>.<lastname>@test, as shown below:

    contexts:
    - context:
        cluster: test.cloud-platform.service.justice.gov.uk
        user: <firstname>.<lastname>@test
    name: test.cloud-platform.service.justice.gov.uk
    current-context: ""
    kind: Config
    preferences: {}
    users:
    - name: <firstname>.<lastname>@test
    
  6. Merge the two config files together into a new config file and replace your old config with the new merged config:

    KUBECONFIG=~/.kube/config-test:~/.kube/config-live kubectl config view --flatten > /tmp/config && mv /tmp/config ~/.kube/config
    

Now using the single config file, you can switch between test and live context

For test:

kubectl config use-context test.cloud-platform.service.justice.gov.uk

For live:

kubectl config use-context live.cloud-platform.service.justice.gov.uk

Where to go from here?

Now that you’ve setup kubectl, you might want to look at this handy cheatsheet.

Once you are ready to deploy applications you will need to create a namespace first.

This page was last reviewed on 19 April 2022. It needs to be reviewed again on 19 July 2022 .
This page was set to be reviewed before 19 July 2022. This might mean the content is out of date.