Recovering Jenkins When Azure AD Client Secret Expires

The problem started with a permission issue

A colleague reported they could not approve a pipeline in Jenkins. I figured it was just a missing permission, so I opened Jenkins to approve it myself. That is when I noticed something was wrong. My account was also read-only.

I am a DevOps engineer at a telecom company, managing the CI/CD pipeline. Our team has admin access in Jenkins. Seeing my own account reduced to read-only meant something deeper was broken.

I tried logging out and logging back in through Azure AD. The login page loaded, but after authenticating with Entra, Jenkins showed an error. The authentication was failing silently. No useful error message, just a redirect loop back to the login page.

That is when I realized the Azure AD client secret had probably expired. Microsoft Entra client secrets have a maximum lifetime of 24 months, and Microsoft recommends setting an expiration of less than 12 months [3]. Ours had been set a long time ago and nobody had rotated it.

The environment

Jenkins is running on an EC2 instance, installed directly on the VM (not in a container). I had SSH access to the instance with sudo. The Jenkins version is 2.492.2. Authentication is handled by the Azure AD plugin [1], with the security realm set to AzureSecurityRealm and authorization set to AzureAdMatrixAuthorizationStrategy [1].

I did not know exactly how Jenkins was installed on the VM. It has a systemd service, so it is likely the Debian/Ubuntu package. But the exact install method did not matter for the recovery. As long as I had OS-level access and could stop and start Jenkins, I could fix this.

What I ruled out

Before SSHing into the instance, I checked a few things from the Entra side. The app registration for Jenkins still existed. The redirect URIs were correct. The client ID and tenant ID had not changed. The only thing that could have expired was the client secret. Microsoft Entra client secrets are created under "Certificates and secrets" in the app registration and have a configurable expiration date [3].

I also ruled out network issues. The EC2 security group allowed my IP. The issue was purely on the authentication layer inside Jenkins.

Getting into the EC2

I SSH'd into the instance and checked that Jenkins was running:

sudo systemctl status jenkins

It was running. No crashes, no obvious errors in the systemd output. The problem was not that Jenkins was down. It was running fine, just refusing to authenticate anyone.

I found the Jenkins home directory:

sudo systemctl cat jenkins | grep JENKINS_HOME

It was /var/lib/jenkins, the default location. Before touching anything, I made a backup of the main configuration:

sudo cp /var/lib/jenkins/config.xml \
        /var/lib/jenkins/config.xml.bak.$(date +%Y%m%d-%H%M%S)

This is important. If I messed up the config, I could always restore from this backup. I was not going to lose the existing job definitions, plugin configurations, or credentials.

Temporarily disabling security

The plan was simple. Disable Jenkins security temporarily, log in without authentication, update the Entra client secret through the UI, then re-enable security. The Jenkins documentation has an official procedure for this exact scenario [2]. The plugin's FAQ also references the same approach [1].

First, I stopped Jenkins:

sudo systemctl stop jenkins

Then I edited the config to disable security. Following the Jenkins docs [2], I looked for the useSecurity element and changed it from true to false:

<useSecurity>false</useSecurity>

I also changed the authorization strategy to Unsecured so there would be no permission checks at all:

<authorizationStrategy class="hudson.security.AuthorizationStrategy$Unsecured"/>

Then I started Jenkins again:

sudo systemctl start jenkins

At this point, Jenkins was wide open. Anyone who could reach the instance could do anything. The Jenkins docs warn about this [2]. I made sure the security group only allowed access from my IP before doing this.

Creating a new Entra client secret

I opened Jenkins in the browser. No login page. I went straight to the dashboard.

I navigated to Manage Jenkins, then Security, then the Security Realm section. The Entra ID configuration was still there. The client ID and tenant ID were intact. But the client secret field was showing the old, expired secret.

I went to the Azure portal, found the app registration for Jenkins, and created a new client secret. The Microsoft Entra documentation describes this under "Certificates and secrets" where you select "New client secret," set a description and expiration, and copy the value [3]. I copied the value (not the Secret ID, the actual value). Then I pasted it into the Jenkins UI and saved.

The plugin encrypted the new secret and wrote it to config.xml. Now the file had useSecurity set to false, an Unsecured authorization strategy, but the correct AzureSecurityRealm with the new client secret.

At this point, I could have just re-enabled security and called it done. But I wanted to restore the original authorization strategy, which had all the team permissions configured through Azure AD groups and users. The plugin uses AzureAdMatrixAuthorizationStrategy to match permissions based on the Object ID of users and groups [1].

Restoring the original authorization

The original authorization was AzureAdMatrixAuthorizationStrategy. It had permissions assigned to specific Entra groups and users. I did not want to recreate those manually. I needed to restore them from the backup.

The backup had the full authorization block. Something like:

<authorizationStrategy class="com.microsoft.jenkins.azuread.AzureAdMatrixAuthorizationStrategy">
    <permission>GROUP:com.cloudbees.plugins.credentials.CredentialsProvider.Create:<devops-group-name></permission>
    <permission>USER:com.cloudbees.plugins.credentials.CredentialsProvider.Create:<user-object-id></permission>
    ...
</authorizationStrategy>

I needed to extract this block from the backup and insert it into the current config, while keeping the new securityRealm with the updated client secret.

I also needed to change useSecurity back to true.

A gotcha with self-closing XML

Here is where I made a mistake. I tried to use a simple regex approach to replace the authorization block. But the current config had a self-closing tag:

<authorizationStrategy class="hudson.security.AuthorizationStrategy$Unsecured"/>

The regex was looking for an opening and closing tag pair with content in between. It did not match the self-closing element. The replacement silently failed.

No damage was done. The backup was still intact. But I wasted some time wondering why the authorization block did not change.

I fixed it with a Python script that handles both cases. It matches either a self-closing tag or a tag pair with content, and replaces it with the authorization block from the backup:

import re

current = "/var/lib/jenkins/config.xml"
backup = "/var/lib/jenkins/config.xml.bak.20260905-102328"

with open(current, "r") as f:
    cur = f.read()

with open(backup, "r") as f:
    bak = f.read()

new = re.search(
    r'<authorizationStrategy\b[^>]*>.*?</authorizationStrategy>',
    bak,
    re.DOTALL
)

if not new:
    raise SystemExit("ERROR: Authorization block not found in backup")

pattern = r'<authorizationStrategy\b[^>]*/>|<authorizationStrategy\b[^>]*>.*?</authorizationStrategy>'

old = re.search(pattern, cur, re.DOTALL)

if not old:
    raise SystemExit("ERROR: Current authorizationStrategy not found")

cur = cur[:old.start()] + new.group(0) + cur[old.end():]

with open(current, "w") as f:
    f.write(cur)

print("Authorization strategy restored successfully.")

This script extracts the full authorization block from the backup and replaces whatever is currently in the config, whether it is self-closing or not.

After running it, I verified the result:

sudo sed -n '10,25p' /var/lib/jenkins/config.xml

The output showed the correct authorization strategy with all the original permissions intact. I also validated the XML:

sudo python3 -c 'import xml.etree.ElementTree as ET; ET.parse("/var/lib/jenkins/config.xml"); print("XML OK")'

It returned XML OK. Good.

I also confirmed the security realm was still there with the new client secret. The securityRealm block looked correct:

<securityRealm class="com.microsoft.jenkins.azuread.AzureSecurityRealm">
    <clientid>...</clientid>
    <credentialType>Secret</credentialType>
    <clientsecret>...</clientsecret>
    <tenant>...</tenant>
</securityRealm>

The new secret was there, encrypted by the plugin. The original authorization was restored. useSecurity was back to true. Everything looked right.

Verifying and restarting

I restarted Jenkins:

sudo systemctl restart jenkins

Checked the status:

sudo systemctl status jenkins --no-pager

Jenkins came up cleanly. I tested the login endpoint locally:

curl -I http://127.0.0.1:8080/login

It returned a redirect to the Azure login page. That was the expected behavior. The security realm was active and pointing to Entra.

I opened Jenkins in an incognito browser window and tested the full Azure AD login flow. I got the Microsoft login page, authenticated, and was redirected back to Jenkins with my full admin permissions. The colleague who originally reported the issue confirmed they could approve pipelines again.

The final state was exactly what I wanted:

• useSecurity set to true
• authorizationStrategy set to AzureAdMatrixAuthorizationStrategy with all original permissions
• securityRealm set to AzureSecurityRealm with the new client secret

Closing

The root cause was simple. The Entra client secret had an expiration date, and nobody remembered to rotate it before it expired. Microsoft Entra client secrets have a maximum lifetime of 24 months, with a recommendation of less than 12 months [3]. This is a classic operations problem. Secrets expire, and if you do not have monitoring or a rotation process in place, you find out when things break.

The fix itself was not complicated. Stop Jenkins, edit a config file, create a new secret, start Jenkins. The tricky part was the self-closing XML tag. I assumed the authorization strategy would always be a tag pair, and that assumption wasted time. Next time I will check the actual structure of the config before writing a regex for it.

I also learned that the Azure AD plugin's encryption is tied to the Jenkins instance. Even though I was editing config.xml directly, the new secret I entered through the UI was encrypted properly. I did not need to understand the encryption format. I just needed to use the plugin's UI to set the secret, and then restore the surrounding config from the backup.

References

[1] "Microsoft Entra ID (previously Azure AD)," Jenkins Plugins, version 711.v34046f788fd7. [Online]. Available: https://plugins.jenkins.io/azure-ad/. [Accessed: Sep. 6, 2026].

[2] "Disable Access Control," Jenkins User Documentation. [Online]. Available: https://www.jenkins.io/doc/book/security/access-control/disable/. [Accessed: Sep. 6, 2026].

[3] "Register an application with the Microsoft identity platform," Microsoft Learn, last updated Jan. 22, 2026. [Online]. Available: https://learn.microsoft.com/en-us/graph/auth-register-app-v2. [Accessed: Sep. 6, 2026].