Migrating SonarQube from AWS to Tencent Cloud

We were doing a cloud migration from AWS to Tencent Cloud. One of the tasks I got was migrating SonarQube.

We use SonarQube for code scanning in our CI pipeline. It runs on Kubernetes with ArgoCD, so it is already GitOps. The same SonarQube image tag on both sides. No version upgrade, just a lift-and-shift to a different cloud.

I asked my manager: "Is downtime okay?" He said yes. I needed to know because I had to stop the SonarQube service while migrating the PV data.

The migration had five parts: database, PV data, custom plugin, ArgoCD apps, and DNS repointing. Let me walk through each one.

Database migration

I started with the database. We used Tencent DTS (Data Transfer Service). The network team already configured the intercloud connection between AWS and Tencent Cloud, so we had private connectivity. I will not talk about that here. That is a whole different story.

DTS was surprisingly easy. We used the migration with full sync mode. This means DTS does an initial full migration, then keeps replicating changes continuously. So the source and destination stay in sync.

But here is the part that took some thinking. We did a two-phase cutover.

First, before the actual cutover, I repointed the old SonarQube (still running on AWS) to use the new Tencent Cloud DB. I updated the jdbcOverwrite in the Helm values to point to the new database URL. This made sure all new scan results were written directly to the Tencent Cloud DB while the old SonarQube was still running on AWS.

Once that was confirmed working, we could safely stop the DTS incremental sync. No data loss, because all new data was already going to the new DB.

The DB schemas were the same on both sides (both PostgreSQL), so no migration issues there. Honestly, this was the smoothest part of the whole thing.

PV data migration

Next was the PV data. This one was straightforward. I created a migration pod in both clusters and mounted the PVCs. Then I used tar to pipe the data directly from the old cluster to the new cluster.

kubectl \
  --context=arn:aws:eks:ap-southeast-1:<aws-account-id>:cluster/<eks-cluster-name> \
  exec -n sonarqube sonarqube-sonarqube-migration -- \
  tar -C /mnt/sonarqube -cpf - . \
| \
kubectl \
  --context=<tencent-cluster-context> \
  exec -i -n sonarqube sonarqube-sonarqube-migration -- \
  tar -C /mnt/sonarqube -xpf -

The data was small. Less than 1GB. The pipe across clusters worked without issues.

One thing I noticed: the old PV was 50Gi. Way too big for the actual data. This was legacy. Someone probably set it once and never revisited. On Tencent Cloud, the minimum CBS volume is 10Gi, so I set it to 10Gi. Still more than enough.

I used tar for the PV migration here. It works, but it requires downtime. If I had to do this again, I would use rsync instead. Rsync can sync incrementally while the service is still running, then do a final quick sync during a brief pause. No downtime needed.

Custom plugin from S3 to COS

We had a custom SonarQube plugin stored in S3. On Tencent Cloud, the object storage is COS. COS is S3-compatible, so we could still use the AWS CLI tool to download it. I just pointed it to the COS endpoint instead.

On AWS, the init container looked like this:

extraInitContainers:
  - name: download-custom-plugin
    image: amazon/aws-cli:2.15.0
    command:
      - sh
      - -c
      - |
        set -e
        echo "Downloading plugin from S3..."
        aws s3 cp \
          s3://<s3-bucket-name>/<plugin-jar-name>.jar \
          /opt/sonarqube/extensions/plugins/<plugin-jar-name>.jar
    env:
      - name: AWS_REGION
        value: ap-southeast-1

On Tencent Cloud, it changed to:

extraInitContainers:
  - name: download-custom-plugin
    image: amazon/aws-cli:2.35.9
    command:
      - sh
      - -c
      - |
        aws configure set default.s3.addressing_style virtual
        aws s3 cp s3://<cos-bucket-name>/<plugin-jar-name>.jar \
          /opt/sonarqube/extensions/plugins/ \
          --endpoint-url https://cos.ap-jakarta.myqcloud.com
    env:
      - name: AWS_ACCESS_KEY_ID
        valueFrom:
          secretKeyRef:
            name: cos-credentials
            key: secret-id
      - name: AWS_SECRET_ACCESS_KEY
        valueFrom:
          secretKeyRef:
            name: cos-credentials
            key: secret-key
      - name: AWS_DEFAULT_REGION
        value: ap-jakarta

Two things I had to figure out.

First, COS needs virtual addressing style. Without aws configure set default.s3.addressing_style virtual, the CLI fails. I got an error on my first attempt and had to look it up. Fun fact: this is a common gotcha with S3-compatible object stores that nobody tells you about upfront.

Second, on AWS we used IRSA (IAM Roles for Service Accounts). No explicit credentials in the values. On Tencent Cloud, COS does not have the same IRSA-like integration. Or maybe it does and I was too lazy to set it up. So I used explicit credentials from a Kubernetes secret instead. It works.

I did the plugin copy before deploying the ArgoCD app. The init container downloads the plugin on pod startup, so the file needs to already be in COS before the pod tries to pull it.

ArgoCD apps in the new environment

With the DB running, PV data transferred, and plugin in COS, I deployed the Helm chart on the new cluster. This was mostly copying it and adjusting things for Tencent Cloud. Storage class changed from ebs-sc (EKS) to cbs (TKE). Different cluster, different defaults. Nothing complicated.

But here is where I made a mistake.

During the initial setup on Tencent Cloud, I minimized the resource requests and limits. The cluster was new and I did not want to waste resources since no other workloads were running yet. So I set them too low.

After cutover, I realized the SonarQube pods were struggling. The resources were too small. I had to quickly bump them up. Not a disaster, just an annoying thing I forgot to adjust. Looking back, I should have set them correctly from the start.

DNS repointing

All the infrastructure was ready. Database migrated, PV data transferred, plugin in COS, ArgoCD app running. But nobody was hitting the new SonarQube yet.

The DNS for <sonarqube-hostname> runs on Cloudflare. The record pointed to the AWS ALB and needed to change to the Tencent CLB.

There was one problem. I did not have permission to change the DNS record myself. So my manager changed it for me.

How long did this take?

The whole thing took about two weeks from planning to done. Not because it was hard. There were just many small things to remember. The actual hands-on work was maybe a few hours. The rest was waiting. Waiting for DTS sync, waiting for approvals, waiting for testing, and fixing things I forgot.

That is it. SonarQube is running on Tencent Cloud now. The pipeline is happy. Hopefully the next migration will be smoother.