Setting Up Your First Workload Cluster
Using a self-service PR automation to provision your first workload cluster
Overview
Now that you have a management cluster and your SCM connected, you can test-out our self-service provisioning using PR Automations and our terraform management system called Stacks. At a high level, this is going to:
- utilize a PR Automation (PRA) to instantiate a few CRDs into folders syncable by the root service-of-services in the
bootstrap
folder - The
InfrastructureStack
CRD created by the PRA will create a terraform stack referencing code in theterraform/modules/clusters/aws
folder to provision a new cluster. Your management cluster should already have been configured with sufficient IAM perms to create this cluster. Any future commits to that folder will also be tracked and generate new terraform runs to sync in changes to the desired infrastructure.
Warning:
This Guide will not work properly unless you've finished the tutorial Integrate with your Source Control Provider.
Enable the cluster-creator PR Automation
There should be a crd at bootstrap/pr-automation/cluster-creator.yaml
which will create the PRA that drives this tutorial. By default it references a github
SCMConnection crd, you'll need to have created that fully, and eventually the operator will also create the PR Automation in our API, and it will be visible in the UI as well.
If it's not showing, navigate to the apps
service, and you can filter on PrAutomation
resources. You should be able to see error messages in the YAML explaining what the operator is tripping on.
Create a Workload Cluster
Now that PR Automation is configured, we should be able to spawn our cluster seamlessly. The steps are:
- Navigate to
https://{your-console-domain}/pr/automations
- Click
Create a PR
on thecluster-creator
Automation Row - Fill in the Required Fields
- Name: The Name of the Cluster
- Cloud: Cloud Provider to Deploy the Cluster (Dropdown Menu)
- Fleet: The Fleet to Associate the cluster (this is arbitrary but can help you group like clusters)
- Tier: The Tier to Place the Cluster (dev/staging/prod)
- Click
Next
- Enter the Name of the Branch to Create the PR
- Click
Create
- Optionally view the PR that was created
- Merge the PR
Once this PR is merged, and our CD system syncs all the manifests, you should see a clusters
service. This will have synced in the InfrastructureStack
CRD and caused a Stack to be created at https://{your-console-domain}/stacks
.
By default these stacks require approval for safety (terraform can do the strangest things sometimes, you should always validate a terraform plan before applying). To do that:
- Navigate to
https://{your-console-domain}/stacks
and click on the run which should be inPending Approval
state - Click the
Approve
Button in the top-right- You can also see the plan in the run logs and the
Plan
subtab as well if you want to ensure the plan looks sane.
- You can also see the plan in the run logs and the
Info:
Cluster provisioning usually takes quite a while. On AWS, expect the process to take upwards of 20m, it can be more like 10m on GCP.
Overview of the GitOps Manifests Created
This PRA creates basically three GitOps manifests:
- A
ServiceDeployment
atbootstrap/clusters.yaml
. This spawns a new service syncing theservices/clusters
folder. This is meant to reduce bloat in the mainbootstrap
folder, and is technically optional. - An
InfrastructureStack
atservices/clusters/{cloud}/stacks/{name}.yaml
. This configures the Stacks api to create a new terraform stack to manage the provisioning of the underlying EKS/AKS/etc cluster. It is tracking theterraform/modules/clusters/{cloud}
folder. - A
Cluster
CRD atservices/clusters/{cloud}/clusters/{name}.yaml
. This creates a pointer CRD which is often used in other CRDs likeServiceDeployment
to reference clusters.
Warning:
The Cluster
CRD is not created by default when registering a cluster. This is why the PR Automation creates it, and if you registered clusters another way, you'll need to create an instance of it as well if you want to use the other CRDs which reference it.