The DevOps Holy Grail: HTTPS and DNS Automation

Co-founder & CPTO

On this page
- Why HTTPS Matters: More Than Just a Green Lock
- The Evolution of HTTPS: From Expensive to Universal
- Automatic HTTPS with cert-manager: Set It and Forget It
- DNS Automation: Why Manual DNS is a Liability
- Automatic DNS with external-dns
- The Complete Automation Stack
- Security Best Practices
- Troubleshooting Common Issues
- What's Next?
In Part 1, we built the foundation: containerization, CI/CD, Kubernetes basics, and GitOps with FluxCD. Now it's time to add the security and reliability features that make your system production-ready.
This isn't just about checking security boxes. It's about building systems that users trust and that scale without breaking.
Why HTTPS Matters: More Than Just a Green Lock
HTTPS isn't optional anymore. It's the foundation of web security, and here's why it matters more than ever:
The Security Imperative
Data Protection: Every request without HTTPS is visible to anyone between your user and your server. Passwords, API keys, personal data—all transmitted in plain text over HTTP.
Authentication: HTTPS proves your server is actually your server, not an imposter. Without it, users have no way to know they're talking to the real you.
Integrity: HTTPS ensures data isn't modified in transit. Without it, attackers can inject malicious code into your pages.
The Business Impact
SEO Rankings: Google penalizes HTTP sites in search results. HTTPS is a ranking factor.
Browser Warnings: Modern browsers show scary warnings for HTTP sites, especially those with forms. Users abandon sites with security warnings.
API Requirements: Most modern APIs require HTTPS. Payment processors, social logins, and third-party services won't work over HTTP.
Compliance: GDPR, PCI DSS, and other regulations often require encrypted data transmission.
The Evolution of HTTPS: From Expensive to Universal
HTTPS evolved from expensive, manual certificates that only large companies could afford to free, automated certificates available to everyone thanks to Let's Encrypt in 2016. Today, encrypting your services isn't just best practice—it's essential for security, SEO, and user trust.
Automatic HTTPS with cert-manager: Set It and Forget It
cert-manager is an open-source Kubernetes add-on that automates the management and issuance of TLS certificates. Originally created by Jetstack (now part of Venafi), it's a CNCF project with strong community support and is used by thousands of organizations worldwide.
What makes cert-manager special:
- Fully automated certificate lifecycle management
- Multiple certificate authorities supported (Let's Encrypt, HashiCorp Vault, Venafi, self-signed)
- Kubernetes-native with custom resources and operators
- Production-ready with extensive monitoring and observability
- Free and open source with commercial support available
cert-manager brings Let's Encrypt automation to Kubernetes. Install it once, and never think about certificates again.
🤖 cert-manager Setup with Let's Encrypt
First, ask me about my certificate requirements before generating any code:- What is your email address for Let's Encrypt notifications?- What domains do you need certificates for?- Do you need wildcard certificates?- What DNS provider do you use (for DNS-01 challenges if needed)?- Do you have existing certificates to migrate from?
Then set up cert-manager in my Kubernetes cluster for automatic HTTPS certificates. I need:
- Complete cert-manager installation via Helm/Flux- ClusterIssuer configuration for Let's Encrypt production certificates- Updated Ingress manifests with TLS annotations for automatic certificate generation- Explanation of how cert-manager automatically renews certificates- Troubleshooting commands to check certificate status- Support for multiple domains and wildcard certificates if needed- Integration with existing GitOps workflow
Important: All Kubernetes manifests should go in the /k8s directory and use FluxCD HelmRelease format for GitOps deployment.Installing cert-manager
apiVersion: v1kind: Namespacemetadata: name: cert-manager---# k8s/cert-manager-helm-repo.yamlapiVersion: source.toolkit.fluxcd.io/v1beta2kind: HelmRepositorymetadata: name: jetstack namespace: cert-managerspec: interval: 1h url: https://charts.jetstack.io---# k8s/cert-manager.yamlapiVersion: helm.toolkit.fluxcd.io/v2beta1kind: HelmReleasemetadata: name: cert-manager namespace: cert-managerspec: interval: 30m chart: spec: chart: cert-manager version: '1.13.x' sourceRef: kind: HelmRepository name: jetstack namespace: cert-manager values: installCRDs: true global: leaderElection: namespace: cert-managerCreating a Certificate Issuer
apiVersion: cert-manager.io/v1kind: ClusterIssuermetadata: name: letsencrypt-prodspec: acme: # Let's Encrypt production server server: https://acme-v02.api.letsencrypt.org/directory email: admin@example.com privateKeySecretRef: name: letsencrypt-prod solvers: # HTTP-01 challenge for regular domains - http01: ingress: class: nginx # DNS-01 challenge for wildcard certificates - dns01: cloudflare: email: admin@example.com apiTokenSecretRef: name: cloudflare-api-token key: api-token selector: dnsNames: - '*.example.com'Using Certificates in Ingress
apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: my-app annotations: # Tell cert-manager to create a certificate cert-manager.io/cluster-issuer: letsencrypt-prod # Force HTTPS redirects nginx.ingress.kubernetes.io/ssl-redirect: 'true'spec: tls: - hosts: - myapp.example.com - api.myapp.example.com secretName: myapp-tls # cert-manager creates this automatically rules: - host: myapp.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-app port: number: 80How Automatic Renewal Works
cert-manager handles the entire certificate lifecycle:
- Issuance: When you create an Ingress with cert-manager annotations, it automatically requests certificates from Let's Encrypt
- Validation: cert-manager proves you control the domain using HTTP-01 or DNS-01 challenges
- Storage: Certificates are stored as Kubernetes Secrets
- Renewal: cert-manager automatically renews certificates 30 days before expiration
- Updates: New certificates are automatically deployed to your Ingress controllers
You never touch certificates manually again.
Monitoring Certificate Health
apiVersion: monitoring.coreos.com/v1kind: PrometheusRulemetadata: name: certificate-alertsspec: groups: - name: certificates rules: - alert: CertificateExpiringSoon expr: certmanager_certificate_expiration_timestamp_seconds - time() < 7 * 24 * 3600 for: 1h annotations: summary: 'Certificate expiring soon' description: 'Certificate {{ $labels.name }} expires in less than 7 days'
- alert: CertificateNotReady expr: certmanager_certificate_ready_status == 0 for: 10m annotations: summary: 'Certificate not ready' description: 'Certificate {{ $labels.name }} is not ready'DNS Automation: Why Manual DNS is a Liability
Managing DNS records manually doesn't scale. Here's why automation matters:
The Manual DNS Problem
Human Error: Typos in DNS records cause outages. A missing dot, wrong IP, or incorrect TTL can break everything.
Slow Changes: Manual DNS updates take time. In incident response, every minute matters.
Inconsistency: Different team members make changes differently. No standard process leads to configuration drift.
No Audit Trail: Who changed what DNS record when? Manual changes are hard to track.
Scaling Issues: Adding new services means updating DNS records. This doesn't scale with microservices.
The Business Case for DNS Automation
Faster Deployments: New services get DNS records automatically. No waiting for ops team.
Reduced Outages: Automated DNS reduces human error, the leading cause of outages.
Better Security: Automated DNS can implement security policies consistently.
Cost Savings: Less manual work means lower operational costs.
Automatic DNS with external-dns
external-dns is an open-source Kubernetes controller that automatically manages DNS records for your services. Originally developed by the Kubernetes community, it's now maintained by the Kubernetes SIG Network and is widely adopted across the ecosystem.
Key features of external-dns:
- Multi-provider support - Works with 20+ DNS providers (Cloudflare, Route53, Google DNS, Azure DNS, etc.)
- Kubernetes-native - Watches Ingress, Service, and other Kubernetes resources
- Declarative - DNS records are managed through Kubernetes annotations
- Safe by default - Configurable policies prevent accidental changes
- Open source with active community development
external-dns watches your Kubernetes Ingress and Service resources and automatically creates DNS records.
🤖 external-dns Setup for Automatic DNS Management
First, ask me about my DNS setup before generating any code:- What DNS provider do you use (Cloudflare, Route53, Google DNS, Azure DNS, etc.)?- What domains do you want external-dns to manage?- Do you have API credentials/tokens for your DNS provider?- Do you want to manage all DNS records or only specific ones?- What is your preferred DNS record TTL?
Then set up external-dns for automatic DNS management in my Kubernetes cluster. I need:
- Complete external-dns installation via Helm/Flux- Configuration for your specific DNS provider- Required API token setup and secret creation- Ingress annotations for automatic DNS record creation- Examples for your chosen DNS provider- Troubleshooting steps to verify DNS automation is working- Integration with existing GitOps workflow
Important: All Kubernetes manifests should go in the /k8s directory and use FluxCD HelmRelease format for GitOps deployment.Installing external-dns
apiVersion: v1kind: Namespacemetadata: name: external-dns---# k8s/external-dns-helm-repo.yamlapiVersion: source.toolkit.fluxcd.io/v1beta2kind: HelmRepositorymetadata: name: external-dns namespace: external-dnsspec: interval: 1h url: https://kubernetes-sigs.github.io/external-dns/---# k8s/external-dns.yamlapiVersion: helm.toolkit.fluxcd.io/v2beta1kind: HelmReleasemetadata: name: external-dns namespace: external-dnsspec: interval: 30m chart: spec: chart: external-dns version: '1.14.x' sourceRef: kind: HelmRepository name: external-dns namespace: external-dns values: provider: cloudflare env: - name: CF_API_TOKEN valueFrom: secretKeyRef: name: cloudflare-api-token key: api-token txtOwnerId: 'my-cluster' policy: sync # or 'upsert-only' for safer operation sources: - ingress - service domainFilters: - example.com # Only manage records for this domainCreating DNS Provider Secrets
# encrypted with sealed-secretsapiVersion: bitnami.com/v1alpha1kind: SealedSecretmetadata: name: cloudflare-api-token namespace: external-dnsspec: encryptedData: api-token: AgBy3i4OJSWK+PiTySYZZA9rO43cGDEQAx... template: metadata: name: cloudflare-api-token namespace: external-dnsUsing DNS Automation in Ingress
apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: my-app annotations: # cert-manager creates certificates cert-manager.io/cluster-issuer: letsencrypt-prod # external-dns creates DNS records external-dns.alpha.kubernetes.io/hostname: myapp.example.com,api.myapp.example.com external-dns.alpha.kubernetes.io/ttl: '300'spec: tls: - hosts: - myapp.example.com - api.myapp.example.com secretName: myapp-tls rules: - host: myapp.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-app port: number: 80 - host: api.myapp.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-api port: number: 80Multi-Provider DNS Setup
external-dns supports multiple DNS providers:
# For AWS Route53values: provider: aws aws: region: us-west-2 zoneType: public serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/external-dns
# For Google Cloud DNSvalues: provider: google google: project: my-project serviceAccountSecret: google-service-account
# For Azure DNSvalues: provider: azure azure: resourceGroup: my-resource-group tenantId: my-tenant-id subscriptionId: my-subscription-idThe Complete Automation Stack
When you combine cert-manager and external-dns, deploying a new service becomes trivial:
apiVersion: apps/v1kind: Deploymentmetadata: name: new-servicespec: replicas: 2 selector: matchLabels: app: new-service template: metadata: labels: app: new-service spec: containers: - name: app image: my-org/new-service:latest ports: - containerPort: 8080---apiVersion: v1kind: Servicemetadata: name: new-servicespec: selector: app: new-service ports: - port: 80 targetPort: 8080---apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: new-service annotations: cert-manager.io/cluster-issuer: letsencrypt-prod external-dns.alpha.kubernetes.io/hostname: new.example.comspec: tls: - hosts: - new.example.com secretName: new-service-tls rules: - host: new.example.com http: paths: - path: / pathType: Prefix backend: service: name: new-service port: number: 80Commit this to Git, and within minutes you have:
- A running service
- Automatic DNS record creation
- Automatic HTTPS certificate
- Automatic certificate renewal
Security Best Practices
Certificate Security
Use Production Let's Encrypt: The staging server has rate limits but issues untrusted certificates. Only use it for testing.
Monitor Certificate Health: Set up alerts for certificate expiration and renewal failures.
Secure Private Keys: cert-manager stores private keys in Kubernetes Secrets. Ensure your cluster has proper RBAC.
DNS Security
Limit Domain Scope: Use domainFilters to prevent external-dns from managing unintended domains.
Use Least Privilege: DNS provider API tokens should have minimal required permissions.
Monitor DNS Changes: Set up alerts for unexpected DNS record changes.
Backup DNS Configuration: Keep backups of your DNS zone files.
Troubleshooting Common Issues
Certificate Problems
# Check certificate statuskubectl get certificates -A
# Check certificate detailskubectl describe certificate myapp-tls
# Check cert-manager logskubectl logs -n cert-manager deployment/cert-manager
# Check certificate challengeskubectl get challenges -ADNS Problems
# Check external-dns logskubectl logs -n external-dns deployment/external-dns
# Verify DNS records were createddig myapp.example.com
# Check external-dns eventskubectl get events -n external-dnsWhat's Next?
With automatic HTTPS and DNS in place, you now have:
- Secure communications with automatic HTTPS certificates
- Reliable DNS that updates automatically with your services
- Zero-maintenance security through automated certificate renewal
- Professional appearance with proper SSL certificates and custom domains
In Part 3, we'll add enterprise-grade features:
- Self-hosted backends with Supabase
- Bulletproof secrets management
- Comprehensive monitoring that prevents incidents
- Multi-environment deployments
- Advanced deployment strategies
🚀 Complete HTTPS and DNS Automation Setup
First, ask me about my specific setup before generating any code:- What DNS provider do you use (Cloudflare, Route53, Google DNS, etc.)?- What is your domain name?- What services do you want to expose and their desired hostnames?- What is your email address for Let's Encrypt notifications?- Do you need wildcard certificates?- Do you have existing DNS records or certificates to migrate?
Then help me implement automatic HTTPS and DNS management for my Kubernetes cluster:
1. cert-manager installation via Flux with Let's Encrypt integration2. ClusterIssuer configuration for both HTTP-01 and DNS-01 challenges3. external-dns installation and configuration for your DNS provider4. Proper secrets management for DNS provider API tokens5. Ingress manifests with automatic certificate and DNS record creation6. Monitoring and alerting for certificate and DNS health7. Troubleshooting procedures for common issues8. Integration with existing GitOps workflow
Important: All Kubernetes manifests should go in the /k8s directory and use FluxCD HelmRelease format for GitOps deployment.
Provide everything needed for production-ready HTTPS and DNS automation.Ready for Part 3? We'll add the enterprise features that scale your infrastructure from startup to enterprise without breaking.