Webhook Certificate Management

Porch webhooks require TLS certificates to function. By default, self-signed certificates are generated at deployment, which works for development and testing but doesn’t auto-rotate. For production, use cert-manager for automatic provisioning and rotation.

Default Setup (Development)

Self-signed certificates are generated by scripts/webhook-utils.sh during deployment and stored in Secret porch-controllers-webhook-tls in namespace porch-system. They’re mounted at /etc/webhook/certs in the porch-controllers pod.

The limitation of self-signed certificates is that they don’t auto-rotate before expiration, which can cause webhook outages if they are not renewed manually.

cert-manager Setup (Example)

This section shows a basic cert-manager setup using self-signed certificates. This approach is suitable for testing and development environments. For production deployments with organizational trust requirements, use a CA-backed issuer or ACME provider instead.

Use cert-manager for automatic certificate management:

1. Install cert-manager (if not already installed):

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v{{ .Site.Params.version_certmanager }}/cert-manager.yaml

2. Create an Issuer that signs webhook certificates:

apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: porch-webhook-issuer
  namespace: porch-system
spec:
  selfSigned: {}

3. Create a Certificate resource:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: porch-webhook-cert
  namespace: porch-system
spec:
  secretName: porch-controllers-webhook-tls
  commonName: porch-controllers
  dnsNames:
  - porch-controllers
  - porch-controllers.porch-system
  - porch-controllers.porch-system.svc
  - porch-controllers.porch-system.svc.cluster.local
  issuerRef:
    name: porch-webhook-issuer
    kind: Issuer
  duration: 2160h     # 90 days
  renewBefore: 360h   # Renew 15 days before expiry

4. Deploy these resources:

kubectl apply -f issuer.yaml
kubectl apply -f certificate.yaml

Once deployed, cert-manager automatically manages the certificate lifecycle. It creates and updates the Secret porch-controllers-webhook-tls, watches for expiration, and rotates before renewal.

The caBundle is automatically patched into ValidatingWebhookConfiguration resources when cert-manager’s CA injector is enabled (via the cert-manager.io/inject-ca-from annotation on the webhook configuration). This requires cert-manager’s cainjector component. Without it, you must manually patch the caBundle or use a webhook configuration that references the certificate directly. The porch-controllers pod automatically uses the updated certificate once it’s in place.

Certificate Paths

Webhook certificates are mounted from the Secret porch-controllers-webhook-tls to /etc/webhook/certs in the porch-controllers pod. The webhook server reads:

• /etc/webhook/certs/tls.crt — the certificate
• /etc/webhook/certs/tls.key — the private key

Certificate Rotation

With cert-manager: Certificates are automatically renewed. cert-manager watches the Certificate resource, rotates before expiration based on the renewBefore setting, updates the Secret, and the porch-controllers pod automatically reloads from the updated Secret. No manual intervention or downtime required.

With self-signed certificates: Manual rotation is required before expiration. Generate new certificates using scripts/webhook-utils.sh, update the Secret, redeploy webhook configurations, and restart the porch-controllers pod.

Diagnostics

The following commands help you inspect webhook and certificate state when investigating a problem.

Webhook Registration

List registered webhooks and check their configuration:

kubectl get validatingwebhookconfiguration
kubectl describe validatingwebhookconfiguration packagerevision-validating-webhook-configuration
kubectl describe validatingwebhookconfiguration repository-validating-webhook-configuration

Certificate Status

Check the certificate exists, view its details, and verify it hasn’t expired:

# View certificate details
kubectl get secret -n porch-system porch-controllers-webhook-tls -o yaml | \
  grep tls.crt | awk '{print $2}' | base64 -d | \
  openssl x509 -text -noout

# Check certificate expiry
kubectl get secret -n porch-system porch-controllers-webhook-tls -o yaml | \
  grep tls.crt | awk '{print $2}' | base64 -d | \
  openssl x509 -noout -enddate

Webhook Pod and Logs

Verify the porch-controllers pod is running and check webhook server logs:

kubectl get pods -n porch-system -l app=porch-controllers
kubectl logs -n porch-system -l app=porch-controllers | grep -i webhook
kubectl describe pod -n porch-system -l app=porch-controllers | grep -A 5 webhook-certs

Troubleshooting

Use the Diagnostics commands above to gather state, then match the symptoms below.

Certificate validation errors: Verify the certificate is mounted correctly, hasn’t expired, and its caBundle matches the current certificate in the ValidatingWebhookConfiguration.

Webhook timeouts: Ensure the porch-controllers pod is running, check logs for errors, and verify cluster network connectivity.

cert-manager Certificate not ready: Check the Certificate status with kubectl describe certificate -n porch-system porch-webhook-cert, verify the Issuer/ClusterIssuer is ready, and check cert-manager logs: kubectl logs -n cert-manager.

caBundle mismatch after rotation: For cert-manager, it should auto-patch ValidatingWebhookConfiguration. For self-signed, verify the caBundle matches the certificate after manual rotation.

Production Deployment Checklist

Before deploying Porch to production, verify:

• Webhook TLS certificates configured (using cert-manager for automatic rotation)
• Certificate Issuer ready (Issuer or ClusterIssuer deployed and verified)
• Certificate rotation tested (verify it works without causing outages)
• Webhook availability monitored (alerts for timeouts or failures)
• Certificate expiry monitored (alerts before expiration for manual rotation scenarios)
• Pod replicas >= 2 (multiple controller replicas for high availability)
• Webhook timeouts appropriate (default 30s suitable for most clusters)
• RBAC verified (service account has webhook management permissions)

Critical: Missing webhook TLS configuration is a production readiness blocker. Webhooks require valid TLS certificates to function. Without proper certificate management, webhook validation fails when certificates expire, blocking all package operations.