Crossplane Ep 12: GitOps with ArgoCD and Crossplane

1. The GitOps Architecture

#

Before GitOps, if you wanted to change infrastructure, you ran terraform apply from your laptop, or you configured a complex Jenkins pipeline with AWS Admin credentials.

With GitOps, Git is the single source of truth.

1. The developer commits a PostgreSQLInstance YAML file to GitHub.
2. ArgoCD (running inside the Kubernetes cluster) detects the commit.
3. ArgoCD pulls the YAML from GitHub and applies it to the cluster.
4. Crossplane (also inside the cluster) detects the new object and talks to AWS.

Notice that Jenkins is entirely eliminated from the deployment phase, and AWS credentials never leave the Kubernetes cluster.

2. Installing ArgoCD

#

We must install ArgoCD into our cluster alongside Crossplane.

# Create a namespace for ArgoCD
kubectl create namespace argocd

# Install the ArgoCD manifest
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Verify it is running:

kubectl get pods -n argocd

Accessing the ArgoCD UI

#

To view the UI on your laptop:

kubectl port-forward svc/argocd-server -n argocd 8080:443

You can now open https://localhost:8080 in your browser.

3. Creating the Git Repository Structure

#

For ArgoCD to work, we need a Git repository.

A best practice in GitOps is to separate your Platform code (XRDs and Compositions) from your Application code (Deployments and Claims).

Let’s assume we have an app-infrastructure repository. It looks like this:

app-infrastructure/
├── production/
│   └── database-claim.yaml
└── staging/
    └── database-claim.yaml

The production/database-claim.yaml looks exactly like the one we wrote in Episode 4:

apiVersion: database.acmecorp.com/v1alpha1
kind: PostgreSQLInstance
metadata:
  name: prod-app-db
  namespace: production
spec:
  parameters:
    storageGB: 500
    environment: prod

Commit this to a public GitHub repository (e.g., github.com/my-org/app-infrastructure).

4. Hooking ArgoCD to Crossplane

#

Now, we must tell ArgoCD to watch that GitHub repository and apply any YAML it finds into the cluster.

We do this by creating an ArgoCD Application Custom Resource.

Create a file named argocd-crossplane-app.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: prod-infrastructure
  namespace: argocd
spec:
  project: default
  
  # 1. Where is the YAML coming from? (GitHub)
  source:
    repoURL: 'https://github.com/my-org/app-infrastructure.git'
    targetRevision: HEAD
    path: production

  # 2. Where is it going? (The local cluster)
  destination:
    server: 'https://kubernetes.default.svc'
    namespace: production

  # 3. Tell ArgoCD to apply changes automatically!
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Apply this to the cluster:

kubectl apply -f argocd-crossplane-app.yaml

The Magic of the Sync

#

Within 3 minutes (ArgoCD’s default polling interval), ArgoCD will pull the database-claim.yaml from GitHub and apply it to the cluster.

Because Crossplane is running in the exact same cluster, it instantly detects the new Claim, runs the Composition Engine, and begins provisioning the 500GB RDS instance in AWS.

If you open the ArgoCD Web UI, you will see a beautiful visual tree showing the PostgreSQLInstance turning green!

5. Handling Drift and “Self-Heal”

#

What happens if someone manually runs kubectl edit postgresqlinstance prod-app-db and changes the storage from 500 to 20?

Because we set selfHeal: true in the ArgoCD Application:

1. ArgoCD detects that the live cluster state (20GB) differs from the Git state (500GB).
2. ArgoCD immediately overwrites the manual edit, restoring it to 500GB.
3. Crossplane ensures AWS remains at 500GB.

You now have a two-layered defense against drift. ArgoCD protects the Kubernetes layer, and Crossplane protects the physical AWS layer. No unauthorized changes can survive.

Troubleshooting & Common Errors

#

1. ArgoCD shows OutOfSync and Failed to Sync

   • Root Cause: ArgoCD tried to apply the PostgreSQLInstance YAML, but the Kubernetes API rejected it. This usually happens because you forgot to install the Platform Code (the XRDs) into the cluster before applying the Application Code (the Claim).
   • Solution: Ensure Crossplane XRDs are fully installed and ESTABLISHED before ArgoCD attempts to apply Claims. (Pro-tip: You can use ArgoCD Sync Waves to deploy XRDs first, and Claims second).

2. The Claim is Green in ArgoCD, but the Database isn’t in AWS

   • Root Cause: ArgoCD only monitors the resources it applies (the Claim). It does not natively know about the physical AWS MRs that Crossplane creates behind the scenes. ArgoCD might say “Healthy”, while Crossplane is failing to authenticate with AWS.
   • Solution: You must still use the Crossplane CLI (kubectl crossplane trace) to debug AWS provisioning errors. ArgoCD only guarantees that the Claim YAML is successfully lodged in the cluster.

Conclusion & Next Steps

#

You have successfully built a fully automated Platform Engineering pipeline. Developers open Pull Requests, and physical cloud infrastructure magically appears.

However, as you onboard dozens of teams to your IDP, you will begin to hit the physical limitations of the Kubernetes Control Plane. Installing the AWS Provider injected 1,000 CRDs into your cluster, which consumes massive amounts of RAM.

In Episode 13: Provider Family and Performance, we will learn how to optimize a Crossplane cluster for enterprise scale, understanding the memory footprint of Go controllers and how to split monolithic Providers into lightweight Provider Families.