Managed Certificate stuck in PENDING status

[djeastm]

Hi!

I've been trying to spin up a new instance of PlatformPlatform in Azure using the pp CLI and everything appears to be working except for the Managed Certificates for custom domains. It seems they get stuck in Pending for several hours before failing. I made the requisite CNAME and TXT records with my DNS provider and have retried removing/adding the certificates, both through the Azure CLI and the portal, but to no avail.

I believe it has something to do with the recent changes they've made concerning validation, referenced here: microsoft/azure-container-apps#1542. Most solutions I've found related to that change mention ensuring the app is publicly available, but even with the app-gateway container app in this instance seemingly open to the internet already, I can't find any other place to open. I have not changed anything from the default configurations.

There doesn't seem to be much in the way of visibility into the action being taken while "Pending". Presumably it's just DigiCert attempting to reach the instance.

I was wondering if anyone else has deployed a new instance recently and come across a similar issue and/or found a solution.

Thanks!

[tjementum]

Hi @djeastm

Yes... I know. This is a problem in Azure. The PlatformPlatform AppGateway is public so that is not the problem.

I have a support case running with Microsoft, who escalated it to their product team.

The Actual Problem

When deploying via Bicep, Azure Container Apps fails to create the DigiCert validation endpoint:

https://app.your-saas-product.com/.well-known/pki-validation/fileauth.txt

This endpoint is only created when manually configuring through the Azure Portal. Since August 15th, DigiCert requires this endpoint - without it, certificate issuance fails.

The Workaround (Manual Process)

For now I have a workaround that I've confirmed with two customers. But it requires a manual step.

1. Delete the app-gateway container app manually.
2. Manually create a new app-gateway pointing to the correct container registry, and select the version of app-gateway that should be deployed from the registry. Map the port to 8080 (instead of the default 80).
3. Manually add a domain and certificate to the App Gateway. This should now work... but the next deployment of infrastructure will fail. Continue to solve this problem.
4. Manually delete the certificate from the Container Apps Environment... you might need to manually unbind it from the app-gateway. The problem is that the certificate created in the Portal is not named correctly as expected by the infrastructure.
5. Go to the resource group and select "Deployments". You should see a failed deployment with the name xxxxx-prod-eu-app-gateway-managed-certificate, where xxxx is your unique resource prefix (see GitHub variable named UNIQUE_PREFIX).
6. Click Redeploy on the failed certificate in the Azure Portal. Now that you have the domain configured, it should succeed.
7. Now you should be able to browse to your domain using https.
8. Recommended: Deploy your cloud infrastructure to verify that you are able to deploy infrastructure without running into problems. Also deploy your AppGateway code, just to ensure that is also working.
9. Verify that you can browse http://app.your-saas-product.com/.well-known/pki-validation/fileauth.txt, to ensure that Azure Container Apps will be able to renew your certificate every 3 months.

I have only done this once myself, but another customer of mine managed to solve it following this guide.

PS: If you have any other questions, I will be more than happy to jump on a call and answer any questions you have.

[djeastm]

That did it. Thanks for the help!

Edit: The Cloud Infrastructure deployment does seem to be having trouble getting through the Staging/Plan step now, but I'll continue looking at that.
Edit2: It was stuck on the az deployment sub create -w part of Plan Cluster Resources. I just commented the entire Plan step out and ran the workflow with only the Deploy step and that was fine. Then uncommented the Plan step and it finished fine when re-running the entire workflow.