Production checklist

Comprehensive checklist to ensure your Boiler-Go deployment is production-ready

Overview

This checklist covers all critical aspects of deploying Boiler-Go to production, based on the built-in production-ready features and best practices.

Configuration

Environment variables

1. Required variables are set

   DATABASE_URL=postgres://user:pass@host:5432/db?sslmode=require
   REDIS_ADDR=redis-host:6379
   

   See internal/config/config.go:61-74 for validation logic.

2. Database URL uses SSL

   # Production
   DATABASE_URL=postgres://user:pass@host:5432/db?sslmode=require
   
   # NOT in production
   DATABASE_URL=postgres://user:pass@host:5432/db?sslmode=disable
   

   Always use sslmode=require or higher in production to encrypt database connections.

3. Redis password is configured

   REDIS_PASSWORD=your_secure_password_here
   

   Enable Redis AUTH in production to prevent unauthorized access.

4. Timeouts are appropriate

   HEALTH_CHECK_TIMEOUT=5s
   API_SHUTDOWN_TIMEOUT=30s
   WORKER_SHUTDOWN_TIMEOUT=60s
   

   Production systems should allow more time for graceful shutdown than development.

5. Logging is configured

   # For containers (Docker, Kubernetes)
   LOG_OUTPUT=stdout
   
   # For traditional servers
   LOG_OUTPUT=file
   LOG_FILE=/var/log/boiler-go/api.log
   

   Configure based on your deployment platform and log aggregation strategy.

Database configuration

1. Connection pool is tuned

   Review pool settings in internal/db/pool.go:32-36:

   poolConfig.MaxConns = 15              // Adjust for load
   poolConfig.MinConns = 2
   poolConfig.MaxConnLifetime = 30 * time.Minute
   poolConfig.MaxConnIdleTime = 5 * time.Minute
   poolConfig.HealthCheckPeriod = 1 * time.Minute
   

   Default values work for most applications. Increase MaxConns for high-traffic scenarios.

2. Migrations are applied

   make migrate-up
   # or
   migrate -path migrations -database $DATABASE_URL up
   

   Ensure database schema is up-to-date before deployment.

3. Database credentials are secured

   • Use secrets management (AWS Secrets Manager, HashiCorp Vault, Kubernetes Secrets)
   • Never commit credentials to version control
   • Rotate passwords regularly
   • Use least-privilege database users
4. Backups are configured

   • Automated daily backups
   • Point-in-time recovery enabled
   • Backup retention policy defined
   • Regular restore testing

Redis configuration

1. Persistence is enabled

   # In docker-compose.yml
   redis:
     command: redis-server --appendonly yes
   

   AOF (Append-Only File) ensures task queue durability.

2. Memory policy is configured

   redis-server --maxmemory 2gb --maxmemory-policy allkeys-lru
   

   Prevent out-of-memory errors with appropriate policies.

3. Redis is monitored

   • Memory usage
   • Connection count
   • Command statistics
   • Persistence status

Security

Network security

1. Services are not publicly exposed

   Only expose the API server. Keep database and Redis on private networks.

   # docker-compose.yml - Remove port mappings in production
   postgres:
     # ports:  # Don't expose to host
     #   - "5432:5432"
   
2. TLS is configured

   • Use HTTPS for API endpoints
   • Configure TLS certificates
   • Use reverse proxy (Nginx, Traefik) for SSL termination
   • Enable HSTS headers
3. Firewall rules are configured

   • Restrict database access to application servers only
   • Restrict Redis access to application servers only
   • Use security groups/network policies
   • Enable VPC/private networking

Application security

1. Request limits are enforced

   Built-in security from cmd/api/main.go:90-97:

   server := &http.Server{
       ReadTimeout:    10 * time.Second,
       WriteTimeout:   10 * time.Second,
       IdleTimeout:    60 * time.Second,
       MaxHeaderBytes: 1 << 20, // 1MB
   }
   

   These prevent resource exhaustion attacks.

2. CORS is properly configured

   Review CORS settings in your handler setup:

   // Configure for your specific origins
   e.Use(middleware.CORS())
   

   Don’t use wildcard (*) in production.

3. Panic recovery is enabled

   Built-in panic recovery middleware prevents crashes.

   Echo’s default recovery middleware is production-ready and handles panics gracefully.

4. Input validation is implemented

   • Validate all user inputs
   • Sanitize data before database queries
   • Use parameterized queries (sqlc provides this)
   • Implement rate limiting

Secrets management

1. Environment variables are secured

   • Never commit .env files
   • Use secrets management services
   • Inject secrets at runtime
   • Rotate credentials regularly
2. Sensitive data is encrypted

   • Encrypt data at rest
   • Encrypt data in transit
   • Use appropriate encryption algorithms
   • Manage encryption keys securely

Reliability

Graceful shutdown

1. API server shutdown is configured

   Built-in graceful shutdown in cmd/api/main.go:110-131:

   sigChan := make(chan os.Signal, 1)
   signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
   
   select {
   case sig := <-sigChan:
       logg.Info().Str("signal", sig.String()).Msg("shutdown signal received")
   }
   
   shutdownCtx, cancel := context.WithTimeout(context.Background(), cfg.APIShutdownTimeout)
   defer cancel()
   
   if err := server.Shutdown(shutdownCtx); err != nil {
       logg.Error().Err(err).Msg("server shutdown failed")
   }
   

   Handles SIGTERM and SIGINT signals gracefully.

2. Worker shutdown is configured

   Built-in graceful shutdown in cmd/worker/main.go:143-178:

   srv.Stop()  // Stop accepting new tasks
   
   shutdownCtx, cancel := context.WithTimeout(context.Background(), cfg.WorkerShutdownTimeout)
   defer cancel()
   
   done := make(chan struct{})
   go func() {
       srv.Shutdown()  // Wait for in-flight tasks
       close(done)
   }()
   
   select {
   case <-done:
       logg.Info().Msg("worker shutdown completed gracefully")
   case <-shutdownCtx.Done():
       logg.Warn().Msg("worker shutdown timed out, forcing exit")
   }
   

   Allows in-flight tasks to complete before shutdown.

3. Shutdown timeouts are appropriate

   API_SHUTDOWN_TIMEOUT=30s      # Time for requests to complete
   WORKER_SHUTDOWN_TIMEOUT=60s   # Time for tasks to complete
   

   Set timeouts longer than your longest expected request/task duration.

Health checks

1. Health endpoint is accessible

   curl http://localhost:8080/health
   

   Returns database and Redis status with response time.

2. Load balancer health checks configured

   # Example: Kubernetes liveness probe
   livenessProbe:
     httpGet:
       path: /health
       port: 8080
     initialDelaySeconds: 10
     periodSeconds: 5
   

   The /health endpoint is lightweight and safe for frequent polling - it does not enqueue background jobs.

3. Database health is monitored

   Built-in connection pool health checks:

   poolConfig.HealthCheckPeriod = 1 * time.Minute
   

   From internal/db/pool.go:36.

Error handling

1. Errors are logged properly

   Built-in error handling in cmd/worker/main.go:93-103:

   ErrorHandler: asynq.ErrorHandlerFunc(func(ctx context.Context, task *asynq.Task, err error) {
       logg.Error().
           Err(err).
           Str("task_type", task.Type()).
           Str("task_id", taskID).
           Msg("task processing failed")
   })
   

   All errors are captured with context.

2. Retry strategy is configured

   Built-in exponential backoff in cmd/worker/main.go:88-91:

   RetryDelayFunc: func(n int, e error, t *asynq.Task) time.Duration {
       // 1s, 2s, 4s, 8s, 16s...
       return time.Duration(1<<uint(n)) * time.Second
   }
   

   Failed tasks are automatically retried with increasing delays.

3. Resource cleanup is guaranteed

   defer db.Close()
   defer rdb.Close()
   defer schedulerClient.Close()
   

   Resources are cleaned up even if errors occur.

Worker configuration

1. Concurrency is tuned

   // In cmd/worker/main.go:82
   asynq.Config{
       Concurrency: 10,  // Adjust based on workload
   }
   

   Balance between throughput and resource usage.

2. Queue priorities are configured

   // In cmd/worker/main.go:84
   Queues: queue.Priorities(),  // critical:6, default:3, low:1
   

   Defined in internal/queue package.

3. Worker status is verified

   # Check worker connectivity
   curl http://localhost:8080/worker/status
   
   # Test task processing
   curl -X POST http://localhost:8080/worker/ping
   

   Verify worker is processing tasks before deploying.

Monitoring and logging

Structured logging

1. Logs are in JSON format

   Built-in structured logging with zerolog:

   {
     "level": "info",
     "request_id": "abc123",
     "method": "GET",
     "path": "/health",
     "duration": 12,
     "status": 200,
     "message": "request completed"
   }
   
2. Request IDs are tracked

   Built-in request correlation in internal/middleware/logger.go:18-28:

   reqID := c.Request().Header.Get("X-Request-ID")
   if reqID == "" {
       reqID = uuid.NewString()
   }
   
   c.Response().Header().Set("X-Request-ID", reqID)
   

   Every request has a unique ID for tracing.

3. Request IDs propagate to workers

   Worker tasks include the original request ID for end-to-end tracing.

   See cmd/worker/main.go:123-125 for correlation logging.

4. Log aggregation is configured

   • Send logs to centralized system (ELK, Splunk, Datadog)
   • Configure log retention policies
   • Set up log-based alerts
   • Enable log searching and filtering

Metrics and monitoring

1. Application metrics are collected

   Monitor:

   • Request rate and duration
   • Error rate and types
   • Database query performance
   • Worker task processing rate
   • Queue depth
2. Infrastructure metrics are collected

   Monitor:

   • CPU usage
   • Memory usage
   • Disk I/O
   • Network I/O
   • Connection counts
3. Database metrics are monitored

   Monitor:

   • Connection pool usage
   • Query duration
   • Lock waits
   • Table sizes
   • Replication lag (if applicable)
4. Redis metrics are monitored

   Monitor:

   • Memory usage
   • Key count
   • Command rate
   • Queue sizes
   • Persistence status

Alerting

1. Critical alerts are configured

   • Service down
   • Database connection failures
   • Redis connection failures
   • High error rates
   • Resource exhaustion
2. Performance alerts are configured

   • Slow response times
   • High CPU/memory usage
   • Database connection pool exhaustion
   • Queue backlog growth
   • Disk space low
3. Alert notification channels are set up

   • Email notifications
   • Slack/Teams integration
   • PagerDuty for critical issues
   • On-call rotation defined

Performance

Database optimization

1. Indexes are created

   • Index frequently queried columns
   • Index foreign keys
   • Use composite indexes where appropriate
   • Monitor index usage
2. Queries are optimized

   • Use EXPLAIN ANALYZE for slow queries
   • Avoid N+1 queries
   • Use connection pooling (built-in)
   • Implement query timeouts
3. Connection pool is monitored

   // Monitor pool stats
   stats := db.Get().Stat()
   // Check: AcquireCount, AcquireDuration, AcquiredConns, IdleConns
   

Application optimization

1. Thread safety is verified

   Built-in thread-safe components:

   • Database pool uses sync.RWMutex (internal/db/pool.go:15)
   • Config loading uses sync.Once (internal/config/config.go:43)
   • All shared resources are properly synchronized
2. Resource limits are configured

   // HTTP server limits (cmd/api/main.go:90-97)
   ReadTimeout:    10 * time.Second,
   WriteTimeout:   10 * time.Second,
   IdleTimeout:    60 * time.Second,
   MaxHeaderBytes: 1 << 20,  // 1MB
   
3. Worker concurrency is optimized

   // Adjust based on workload and available resources
   Concurrency: 10,  // cmd/worker/main.go:82
   

Caching strategy

1. Redis caching is implemented

   • Cache frequently accessed data
   • Set appropriate TTLs
   • Implement cache invalidation
   • Monitor cache hit rates
2. Cache policies are defined

   • What to cache
   • Cache duration
   • Cache eviction strategy
   • Cache warming strategy

Deployment

Deployment process

1. Zero-downtime deployment is configured

   • Rolling updates
   • Blue-green deployment
   • Canary releases
   • Graceful shutdown ensures no dropped requests
2. Rollback plan is defined

   • Quick rollback procedure
   • Database migration rollback strategy
   • Configuration rollback
   • Automated rollback triggers
3. Health checks prevent bad deployments

   • Pre-deployment health checks
   • Post-deployment verification
   • Automated rollback on health check failures

Infrastructure as code

1. Infrastructure is version controlled

   • Terraform/CloudFormation templates
   • Kubernetes manifests
   • Docker Compose files
   • CI/CD pipeline configuration
2. Configuration is managed

   • Environment-specific configs
   • Secrets management
   • Feature flags
   • Configuration validation

Documentation

1. Deployment process is documented

   • Step-by-step deployment guide
   • Environment setup instructions
   • Troubleshooting guide
   • Runbook for common issues
2. Architecture is documented

   • System architecture diagram
   • Component interactions
   • Data flow documentation
   • API documentation

Built-in production features

Boiler-Go includes these production-ready features out of the box:

Thread-safe database pool

Concurrent-safe PostgreSQL connection management with sync.RWMutex.

See internal/db/pool.go:14-15

Graceful shutdown

Proper resource cleanup and timeout handling for both API and worker.

See cmd/api/main.go:110-138 and cmd/worker/main.go:143-178

Structured logging

JSON logging with request tracing and correlation IDs.

See internal/middleware/logger.go

Configuration validation

Fail-fast validation with descriptive error messages.

See internal/config/config.go:60-92

Security hardened

Request size limits, timeouts, and panic recovery.

See cmd/api/main.go:90-97

Background jobs

Redis-based task processing with retry logic and exponential backoff.

See cmd/worker/main.go:79-104

Health monitoring

Lightweight service health monitoring with duration tracking.

Endpoint: GET /health

Worker management

API endpoints for worker status and ping testing.

Endpoints: GET /worker/status, POST /worker/ping

Pre-deployment verification

Run through this final checklist before going live:

1. Configuration review

   • All environment variables set
   • SSL enabled for database
   • Redis password configured
   • Timeouts appropriate for production
   • Logging configured correctly
2. Security review

   • Database credentials secured
   • Services not publicly exposed
   • TLS configured
   • Firewall rules in place
   • Input validation implemented
3. Reliability verification

   • Health endpoint accessible
   • Graceful shutdown tested
   • Worker processing verified
   • Database migrations applied
   • Backups configured
4. Monitoring setup

   • Logs aggregated
   • Metrics collected
   • Alerts configured
   • Dashboards created
   • On-call rotation defined
5. Performance validation

   • Load testing completed
   • Database indexes created
   • Connection pools tuned
   • Worker concurrency optimized
   • Cache strategy implemented
6. Deployment readiness

   • Rollback plan defined
   • Zero-downtime deployment configured
   • Documentation complete
   • Team trained
   • Runbook prepared

Additional resources

Docker deployment

Container configuration and best practices

Environment variables

Complete configuration reference

Architecture

System design and component overview

Health checks

Monitoring endpoints documentation