Cert-Manager Certificate creation stuck at Created new CertificateRequest resource

1
votes

I am using cert-manager v1.0.0 on GKE, I tried to use the staging environment for acme and it worked fine but when shifting to production I can find the created certificate stuck at Created new CertificateRequest resource and nothing changes after that

I expect to see the creation of the certificate to be succeeded and change the status of the certificate from false to true as happens in staging

Environment details::

Kubernetes version (v1.18.9): Cloud-provider/provisioner (GKE): cert-manager version (v1.0.0): Install method (helm)

Here is my clusterIssuer yaml file

apiVersion: cert-manager.io/v1

kind: ClusterIssuer

metadata:
  name: i-storage-ca-issuer-prod
  namespace: default
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: MY_EMAIL_HERE
    privateKeySecretRef:
      name: i-storage-ca-issuer-prod
    solvers:
    - http01:
        ingress:
          class: gce

And here is my ingress yaml file

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: i-storage-core
  namespace: i-storage
  annotations:
    kubernetes.io/ingress.global-static-ip-name: i-storage-core-ip
    cert-manager.io/cluster-issuer: i-storage-ca-issuer-prod
  labels:
    app: i-storage-core
spec:
  tls:
  - hosts:
    - i-storage.net
    secretName: i-storage-core-prod-cert
  rules:
  - host: i-storage.net
    http:
      paths:
      - path: /*
        backend:
          serviceName: i-storage-core-service
          servicePort: 80

describe certificateRequest output

Name:         i-storage-core-prod-cert-stb6l
Namespace:    i-storage
Labels:       app=i-storage-core
Annotations:  cert-manager.io/certificate-name: i-storage-core-prod-cert
              cert-manager.io/certificate-revision: 1
              cert-manager.io/private-key-secret-name: i-storage-core-prod-cert-2pw26
API Version:  cert-manager.io/v1
Kind:         CertificateRequest
Metadata:
  Creation Timestamp:  2020-10-31T15:44:57Z
  Generate Name:       i-storage-core-prod-cert-
  Generation:          1
  Managed Fields:
    API Version:  cert-manager.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:cert-manager.io/certificate-name:
          f:cert-manager.io/certificate-revision:
          f:cert-manager.io/private-key-secret-name:
        f:generateName:
        f:labels:
          .:
          f:app:
        f:ownerReferences:
          .:
          k:{"uid":"f3442651-3941-49af-81de-dcb937e8ba40"}:
            .:
            f:apiVersion:
            f:blockOwnerDeletion:
            f:controller:
            f:kind:
            f:name:
            f:uid:
      f:spec:
        .:
        f:issuerRef:
          .:
          f:group:
          f:kind:
          f:name:
        f:request:
      f:status:
        .:
        f:conditions:
    Manager:    controller
    Operation:  Update
    Time:       2020-10-31T15:44:57Z
  Owner References:
    API Version:           cert-manager.io/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Certificate
    Name:                  i-storage-core-prod-cert
    UID:                   f3442651-3941-49af-81de-dcb937e8ba40
  Resource Version:        18351251
  Self Link:               /apis/cert-manager.io/v1/namespaces/i-storage/certificaterequests/i-storage-core-prod-cert-stb6l
  UID:                     83412862-903f-4fff-a736-f170e840748e
Spec:
  Issuer Ref:
    Group:  cert-manager.io
    Kind:   ClusterIssuer
    Name:   i-storage-ca-issuer-prod
  Request:  LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2ZUQ0NBV1VDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU5HcQovRDRVZlRhV0xFa01GUzdsdVN1RmRlR0NNVjJ4czREcG5Pem1HbjJxSlRUTlBnS2hHbGVEd0p2TkZIaTc5WWxHCmpYcjhjNDFHU1JUT2U4UDdUS3AvWXpBSUtxSXpPMllIeHY5VzA5bEZDWWQ4MTByMUNsOG5jb2NYa3BGZlAxMzAKZURlczZ6SUkwZW9ZTW1uRXQ3cmRUNk52dHhuZ1ZZVmlnai9VcXpxSkZ4NmlLa0R6V1VHK3lNcWtQM1ZKa1lYeApZUFNTNWZsWXlTdkI4emdxb3pnNUNJUndra09KTU1aRlNoWHVxYkpNZnJvQmR2YW9nQWtEYmZYSWs0SVRIaXlYCkV4UDFBaFdieGhPbndDd2h5bXpGWmgzSkZUZHhzeFdtRDZJMmp3MzV1SXZ1WWlIWEJ4VTBCMG50K3FYMVVWaWwKSkRlOFdNcTdjT3AzWmtlT2FHa0NBd0VBQWFBNE1EWUdDU3FHU0liM0RRRUpEakVwTUNjd0dBWURWUjBSQkJFdwpENElOYVMxemRHOXlZV2RsTG01bGREQUxCZ05WSFE4RUJBTUNCYUF3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCCkFLMkhhSEQxd3dDZVFqS1diU1N0SFkxMm1Da1A1amQ0RnFmZFFYRG5XR3grK3FCWExGY0F4TVZhbVF2cStQK0gKLzExQjhvdlUydU9icGRHRktoak9aNDJsdjNNMVllRWk5UG5nS0RFdndCbER0Q0Vsa0lHQzV4T1ZENCtheVlmaApEMUI2L20vdEJsdlhYNS8zRDlyejJsTWNRSzRnSTNVQ3Mxd0Y0bmduQ3JYMEhoSDJEendheXI5d2QvY1V1clZlClloYS9HZjcyaEFCcGQxSmkrR2hKaGxzVDlGbTNVZVNUTi9OYkpVWmk4NkM1S1dTRW1DblNjV3dzWGNoVW1vVisKVHpGQmNhOEhqOUxsVFdJVVBSYVl0bFQ2TEhrUjVLUW1EL2tJRTZDajlidTNXMG9oWDZ2UC9CQ012SWdaTVZEUgoyeFVwY3lhUmJad2ttWTQ2MktNZ25wUT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==
Status:
  Conditions:
    Last Transition Time:  2020-10-31T15:44:57Z
    Message:               Waiting on certificate issuance from order i-storage/i-storage-core-prod-cert-stb6l-177980933: "pending"
    Reason:                Pending
    Status:                False
    Type:                  Ready
Events:                    <none>

describe order output

Name:         i-storage-core-prod-cert-stb6l-177980933
Namespace:    i-storage
Labels:       app=i-storage-core
Annotations:  cert-manager.io/certificate-name: i-storage-core-prod-cert
              cert-manager.io/certificate-revision: 1
              cert-manager.io/private-key-secret-name: i-storage-core-prod-cert-2pw26
API Version:  acme.cert-manager.io/v1
Kind:         Order
Metadata:
  Creation Timestamp:  2020-10-31T15:44:57Z
  Generation:          1
  Managed Fields:
    API Version:  acme.cert-manager.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:cert-manager.io/certificate-name:
          f:cert-manager.io/certificate-revision:
          f:cert-manager.io/private-key-secret-name:
        f:labels:
          .:
          f:app:
        f:ownerReferences:
          .:
          k:{"uid":"83412862-903f-4fff-a736-f170e840748e"}:
            .:
            f:apiVersion:
            f:blockOwnerDeletion:
            f:controller:
            f:kind:
            f:name:
            f:uid:
      f:spec:
        .:
        f:dnsNames:
        f:issuerRef:
          .:
          f:group:
          f:kind:
          f:name:
        f:request:
      f:status:
        .:
        f:authorizations:
        f:finalizeURL:
        f:state:
        f:url:
    Manager:    controller
    Operation:  Update
    Time:       2020-10-31T15:44:57Z
  Owner References:
    API Version:           cert-manager.io/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  CertificateRequest
    Name:                  i-storage-core-prod-cert-stb6l
    UID:                   83412862-903f-4fff-a736-f170e840748e
  Resource Version:        18351252
  Self Link:               /apis/acme.cert-manager.io/v1/namespaces/i-storage/orders/i-storage-core-prod-cert-stb6l-177980933
  UID:                     92165d9c-e57e-4d6e-803d-5d28e8f3033a
Spec:
  Dns Names:
    i-storage.net
  Issuer Ref:
    Group:  cert-manager.io
    Kind:   ClusterIssuer
    Name:   i-storage-ca-issuer-prod
  Request:  LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2ZUQ0NBV1VDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU5HcQovRDRVZlRhV0xFa01GUzdsdVN1RmRlR0NNVjJ4czREcG5Pem1HbjJxSlRUTlBnS2hHbGVEd0p2TkZIaTc5WWxHCmpYcjhjNDFHU1JUT2U4UDdUS3AvWXpBSUtxSXpPMllIeHY5VzA5bEZDWWQ4MTByMUNsOG5jb2NYa3BGZlAxMzAKZURlczZ6SUkwZW9ZTW1uRXQ3cmRUNk52dHhuZ1ZZVmlnai9VcXpxSkZ4NmlLa0R6V1VHK3lNcWtQM1ZKa1lYeApZUFNTNWZsWXlTdkI4emdxb3pnNUNJUndra09KTU1aRlNoWHVxYkpNZnJvQmR2YW9nQWtEYmZYSWs0SVRIaXlYCkV4UDFBaFdieGhPbndDd2h5bXpGWmgzSkZUZHhzeFdtRDZJMmp3MzV1SXZ1WWlIWEJ4VTBCMG50K3FYMVVWaWwKSkRlOFdNcTdjT3AzWmtlT2FHa0NBd0VBQWFBNE1EWUdDU3FHU0liM0RRRUpEakVwTUNjd0dBWURWUjBSQkJFdwpENElOYVMxemRHOXlZV2RsTG01bGREQUxCZ05WSFE4RUJBTUNCYUF3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCCkFLMkhhSEQxd3dDZVFqS1diU1N0SFkxMm1Da1A1amQ0RnFmZFFYRG5XR3grK3FCWExGY0F4TVZhbVF2cStQK0gKLzExQjhvdlUydU9icGRHRktoak9aNDJsdjNNMVllRWk5UG5nS0RFdndCbER0Q0Vsa0lHQzV4T1ZENCtheVlmaApEMUI2L20vdEJsdlhYNS8zRDlyejJsTWNRSzRnSTNVQ3Mxd0Y0bmduQ3JYMEhoSDJEendheXI5d2QvY1V1clZlClloYS9HZjcyaEFCcGQxSmkrR2hKaGxzVDlGbTNVZVNUTi9OYkpVWmk4NkM1S1dTRW1DblNjV3dzWGNoVW1vVisKVHpGQmNhOEhqOUxsVFdJVVBSYVl0bFQ2TEhrUjVLUW1EL2tJRTZDajlidTNXMG9oWDZ2UC9CQ012SWdaTVZEUgoyeFVwY3lhUmJad2ttWTQ2MktNZ25wUT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==
Status:
  Authorizations:
    Challenges:
      Token:        EMTpMo_Jt5YkITiwk_lOuL66Xu_Q38scNMf1o0LPgvs
      Type:         http-01
      URL:          https://acme-v02.api.letsencrypt.org/acme/chall-v3/8230128790/0EcdqA
      Token:        EMTpMo_Jt5YkITiwk_lOuL66Xu_Q38scNMf1o0LPgvs
      Type:         dns-01
      URL:          https://acme-v02.api.letsencrypt.org/acme/chall-v3/8230128790/9chkYQ
      Token:        EMTpMo_Jt5YkITiwk_lOuL66Xu_Q38scNMf1o0LPgvs
      Type:         tls-alpn-01
      URL:          https://acme-v02.api.letsencrypt.org/acme/chall-v3/8230128790/BaReZw
    Identifier:     i-storage.net
    Initial State:  pending
    URL:            https://acme-v02.api.letsencrypt.org/acme/authz-v3/8230128790
    Wildcard:       false
  Finalize URL:     https://acme-v02.api.letsencrypt.org/acme/finalize/100748195/5939190036
  State:            pending
  URL:              https://acme-v02.api.letsencrypt.org/acme/order/100748195/5939190036
Events:             <none>
1
sslkubernetesdevopskubernetes-ingresscert-manager
You need to put more information kubectl get certificaterequest and kubectl get order or kubectl describe order <ordername> – Joy
Another thing if staging worked fine then its most likely the challenge failed for some reason or you have faced rate limiter for prod cert – Joy
I updated the question with the needed information – tarek salem
regarding the staging, the same problem was facing me at first but after deleting the ingress and recreating it worked, I tried the same for this issue but not working – tarek salem
can you help me please? – tarek salem

1 Answers

2
votes

List all certificates that you have:

kubectl get certificate --all-namespaces

Try to figure out the problem using describe command:

kubectl describe certificate CERTIFICATE_NAME -n YOUR_NAMESPACE

The output of the above command contains the name of the associated certificate request. Dig into more details using describe command once again:

kubectl describe certificaterequest CERTTIFICATE_REQUEST_NAME -n YOUR_NAMESPACE

You may also want to troubleshoot challenges with the following command:

kubectl describe challenges --all-namespaces

In my case, to make it work, I had to replace ClusterIssuer with just Issuer for reasons explained in the comment.

Here is my Issuer manifest:

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: cert-manager-staging
  namespace: YOUR_NAMESPACE
spec:
  acme:
    # You must replace this email address with your own.
    # Let's Encrypt will use this to contact you about expiring
    # certificates, and issues related to your account.
    email: [email protected]
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      # Secret resource that will be used to store the account's private key.
      name: cert-manager-staging-private-key
    # Add a single challenge solver, HTTP01 using nginx
    solvers:
      - http01:
          ingress:
            class: nginx

Here is my simple Ingress manifest:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/issuer: cert-manager-staging
  name: YOUR_NAME
  namespace: YOUR_NAMESPACE
spec:
  tls:
    - hosts:
        - example.com
      secretName: example-com-staging-certificate
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: example.com
                port:
                  number: 80