403 forbidden when calling Azure Service Management REST api from a worker role instance

So a couple of weeks ago I blogged about how to make use of the Service Bus Entity Metrics REST APIs and some of the things I learned. The next step of the puzzle was to create a worker role that would monitor some of my service bus queues using this API and logs the results.  A team member and I started down the path of creating this worker and thought things were going great.  We installed the certificate on our local machine that we uploaded to the Azure portal and ran the worker locally thru the emulator and all worked well.  Unfortunately, that’s were the celebration stopped.  Once we deployed the worker to a cloud service instance, we kept getting a 403 forbidden exception when the worker tried to call the management API.  After days of fighting this, we finally had success.  Here are the steps we took to make this successful.

  • First things first you need to create a self-signed certificate.  This is the really specific piece.  There are lots of articles on doing this but they all seemed to lack one specific detail.  Credit for these steps clearly go to Jeff and his post on stack overflow.  Run these two commands to create your certificate files.
    makecert -r -pe -n "CN=[name of certificate]" -sky exchange "[path to certificate].cer" -sv "[path to certificate].pvk"
    pvk2pfx -pvk "[path to certificate].pvk" -spc "[path to certificate].cer" -pfx "[path to certificate].pfx" -pi [password]
  • Upload the .cer file to the Azure Portal. This is at the subscription level
    azure portal certificate
  • Upload the .pfx file to the cloud service instance hosting your worker role
    cloud service certificate
  • Inside Visual Studio, add the certificate to the Worker Role properties
    worker properties
  • We actually wrote a small custom web client to abstract some of the details.
        public class AzureManagementWebClient : WebClient
            string _thumbprint = CloudConfig.Get("ServiceBusAPICertificateThumb");
            protected override WebRequest GetWebRequest(Uri address)
                var request = base.GetWebRequest(address);
                //The certificate must be installed on your local machine and configured to deploy with worker
                var certs = this.GetCertificate(StoreLocation.CurrentUser);
                if (certs.Count == 0)
                    certs = this.GetCertificate(StoreLocation.LocalMachine);
                if (certs.Count == 0)
                    throw new ArgumentNullException(string.Format("Certificate: {0}", _thumbprint));
                (request as HttpWebRequest).ClientCertificates.Add(certs[0]);
                (request as HttpWebRequest).Headers.Add("x-ms-version: 2013-10-01");
                (request as HttpWebRequest).Accept = "application/json";
                return request;
            private X509Certificate2Collection GetCertificate(StoreLocation location)
                var store = new X509Store(StoreName.My, location);
                store.Open(OpenFlags.ReadOnly | OpenFlags.OpenExistingOnly);
                var collection = store.Certificates.Find(X509FindType.FindByThumbprint, _thumbprint, false);
                return collection;
  • In order to run this locally with the emulator, you will need to import the pfx into your Local Machine / Personal certificate store