> ## Content Index > Fetch the complete content index at: https://anantafatur.dev/llms.txt > Use this file to discover other available public pages before exploring further. # GitLab Backup to Tencent COS After EC2 Migration - URL: https://anantafatur.dev/gitlab-backup-tencent-cos-cvm/ - Published: 2026-09-11T21:41:18.000Z - Updated: 2026-09-11T21:42:13.000Z - Description: Configuring GitLab Omnibus backup uploads to Tencent Cloud Object Storage after migrating from AWS EC2, and the S3-compatibility settings that make it work. - Author: Ananta We recently moved our self-hosted GitLab instance from AWS EC2 to a Tencent Cloud CVM using go2tencentcloud, Tencent's lift-and-shift migration tool \[1\]. The migration itself went fine. The backup upload config, though, did not survive the move. Here is what broke and how I fixed it. ## Moving from EC2 to CVM On EC2, GitLab uploaded backups to an S3 bucket. We used an IAM role attached to the instance, so there were no access keys to manage. The config in `gitlab.rb` looked like this: ```ruby gitlab_rails['backup_upload_connection'] = { 'provider' => 'AWS', 'region' => 'ap-southeast-1', 'use_iam_profile' => true } gitlab_rails['backup_upload_remote_directory'] = '' ``` Simple. No credentials in config files, no rotation headaches. The EC2 metadata service handled everything. After moving to CVM, this stopped working. Not because the config was wrong, but because there is no EC2 metadata service on a CVM. Tencent Cloud has its own instance metadata service, but COS (Cloud Object Storage) does not support IAM roles the way S3 does in this context. I had to switch to access key and secret key authentication. --- ## First try with COS credentials I created a COS bucket, generated an access key pair, and updated the config: ```ruby gitlab_rails['backup_upload_connection'] = { 'provider' => 'AWS', 'region' => 'auto', 'aws_access_key_id' => '', 'aws_secret_access_key' => '', 'endpoint' => 'https://.cos..myqcloud.com', 'use_iam_profile' => false } gitlab_rails['backup_upload_remote_directory'] = '' ``` COS exposes an S3-compatible API, so `provider => 'AWS'` is correct. GitLab uses the `fog-aws` gem under the hood for S3 uploads, and fog-aws can talk to any S3-compatible endpoint. I ran `gitlab-ctl reconfigure` and tried a backup. It failed. The error was around AWS SDK Ruby's `Expect: 100-continue` handling. Something about "1xx responses not supported." The backup uploader was choking on the HTTP interaction between GitLab's patched Net::HTTP and the COS endpoint. ## Testing the connection in isolation Before digging into GitLab internals, I wanted to confirm the credentials and endpoint actually worked. I used GitLab's embedded Rails runner to create an AWS SDK client directly and test a `PutObject`: ```ruby sudo gitlab-rails runner ' c = Gitlab.config.backup.upload.connection.to_h s3 = Aws::S3::Client.new( region: c["region"], endpoint: c["endpoint"], access_key_id: c["aws_access_key_id"], secret_access_key: c["aws_secret_access_key"], force_path_style: c["path_style"].to_s == "true", http_continue_timeout: 0 ) s3.put_object( bucket: Gitlab.config.backup.upload.remote_directory, key: "gitlab-cos-connection-test.txt", body: "GitLab COS connection test" ) puts "SUCCESS: COS upload works" ' ``` It worked. The object showed up in the COS bucket. So credentials, endpoint, and permissions were all fine. The problem was somewhere in how GitLab's native backup uploader constructs the HTTP requests. The `http_continue_timeout: 0` in that test disables the `Expect: 100-continue` header in the AWS SDK. But this is an SDK client option, not something you can set in `gitlab.rb`. ## Two settings that made it work While searching for solutions, I found Ajay Furber's post about getting GitLab backups to work with Cloudflare R2 \[2\]. Same problem, different S3-compatible provider. The fix came down to two settings in the fog-aws configuration: `**path_style**`: S3-compatible services often use path-style addressing (`bucket-name.endpoint/bucket-name/key`) instead of virtual-hosted style (`bucket-name.endpoint/key`). Some services don't support virtual-hosted style at all. Setting this to `true` forces path-style requests. `**enable_signature_v4_streaming**`: By default, fog-aws sets this to `true`. When enabled, it uses `aws-chunked` content encoding with individually signed chunks. From what I have seen, S3-compatible services like Cloudflare R2 and Tencent COS do not implement this. Setting it to `false` disables chunked transfer and uses a single signed request instead. Neither setting was obvious from GitLab's own documentation. The `path_style` option is at least mentioned in some places, but `enable_signature_v4_streaming` is not. I only learned about it from Ajay's post, which references a fog-aws PR where it was added. So the final config looks like this: ```ruby gitlab_rails['backup_upload_connection'] = { 'provider' => 'AWS', 'region' => 'auto', 'aws_access_key_id' => '', 'aws_secret_access_key' => '', 'endpoint' => 'https://.cos..myqcloud.com', 'path_style' => 'true', 'enable_signature_v4_streaming' => 'false', 'use_iam_profile' => false } gitlab_rails['backup_upload_remote_directory'] = '' ``` Note that `'true'` and `'false'` are strings here, not booleans. That is how GitLab Omnibus passes them through to fog-aws. After updating `gitlab.rb`, I ran `sudo gitlab-ctl reconfigure` to apply the changes. ## Backup worked Then I ran `sudo gitlab-backup create`. The backup completed and the archive was uploaded directly to COS. No additional configuration was needed beyond the two settings above. The `Expect: 100-continue` error turned out to be a symptom, not the root cause. The real issue was that COS does not support v4 chunked streaming encoding, which fog-aws enables by default. Setting `enable_signature_v4_streaming` to `false` fixed the HTTP interaction entirely, and the `http_continue_timeout: 0` SDK workaround was not needed for the native backup uploader. --- ## Closing The fix was two settings in `gitlab.rb` that are not mentioned in GitLab's own backup documentation. I found the solution through Ajay's writeup about Cloudflare R2 \[2\], which is essentially the same problem with a different S3-compatible provider. If you are using any S3-compatible object storage that is not AWS S3 itself, `path_style` and `enable_signature_v4_streaming` are probably the first things to check. ## References \[1\] Tencent Cloud, "Online Migration Overview," Tencent Cloud Documentation, Aug. 13, 2026\. \[Online\]. Available: . \[Accessed: Sep. 12, 2026\]. \[2\] A. Furber, "How to get fog-aws / Gitlab Backups to work with Cloudflare R2," ajf.me.uk, Jul. 6, 2024\. \[Online\]. Available: . \[Accessed: Sep. 10, 2026\].