From Elastic Beanstalk to Zero-Downtime Deployments

We used to deploy to Elastic Beanstalk multiple times a week, and each deployment caused downtime. Sometimes seconds, sometimes minutes—but always downtime. Every Friday afternoon deploy came with anxiety: Will this break? Will the database migration fail? Can I rollback?

Then we built an automated CI/CD pipeline using Bitbucket and AWS CodePipeline. Zero downtime. Full automation. No anxiety.

This is how we did it.

The Problem: Manual Elastic Beanstalk Deployments

Our deployment process was:

1. Engineer pushes code to Bitbucket
2. Engineer manually triggers Elastic Beanstalk deployment
3. EB terminates old instances, launches new ones
4. Database migrations run (sometimes fail, hard to rollback)
5. Application restarts (customers see errors during transition)
6. If something goes wrong, manual rollback (30+ minutes)

Issues:

• Downtime: Every deploy caused 30-60 seconds of downtime while instances restarted
• Manual: Deployments required engineer attention. Mistakes were common
• Database risk: Migrations couldn't be rolled back instantly
• Visibility: No view into what was being deployed or progress
• Frequency: We deployed less often (multiple times/week) to reduce risk, but this meant slower feature velocity

The Solution: Bitbucket + AWS CodePipeline

Architecture

We built a fully automated pipeline:

Engineer pushes to Bitbucket (main branch)
           ↓
Bitbucket webhook triggers AWS CodePipeline
           ↓
CodeBuild: Build Docker image
           ↓
CodeBuild: Run tests (unit + integration)
           ↓
CodeBuild: Push image to ECR (Elastic Container Registry)
           ↓
CodeDeploy: Deploy to "Green" environment (ECS)
           ↓
Health checks on green environment
           ↓
Load balancer: Instant switch from Blue → Green
           ↓
Monitor metrics for 5 minutes
           ↓
If healthy: Clean up old blue environment
If issues detected: Automatic rollback to blue (60 seconds)
      

Key Insight: Blue-Green Deployment

The magic is blue-green deployment:

• Blue: Current production (handling live traffic)
• Green: New version (not receiving traffic)
• Switch: Load balancer instantly routes to green (no downtime)
• Rollback: If something breaks, flip back to blue (60 seconds)

This is the key difference from Elastic Beanstalk's in-place deployment: we run both versions simultaneously, so the switch is instant.

Implementation Details

Step 1: Docker Image Build (CodeBuild)

We define a buildspec.yml that CodeBuild runs:

version: 0.2

phases:
  pre_build:
    commands:
      - aws ecr get-login-password | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
      - COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
      - IMAGE_TAG=$COMMIT_HASH

  build:
    commands:
      - docker build -t my-app:$IMAGE_TAG .
      - docker tag my-app:$IMAGE_TAG $ECR_REPO_URL:$IMAGE_TAG
      - docker tag my-app:$IMAGE_TAG $ECR_REPO_URL:latest

  post_build:
    commands:
      - docker push $ECR_REPO_URL:$IMAGE_TAG
      - docker push $ECR_REPO_URL:latest

artifacts:
  files:
    - imagedefinitions.json
      

Step 2: ECS Deployment (CodeDeploy)

CodeDeploy updates ECS services with the new Docker image:

• Create a new ECS task definition with the new image
• Launch green tasks with the new definition
• Wait for health checks to pass
• Update load balancer target group to point to green

Step 3: Health Checks (Critical!)

Before switching traffic, we run health checks on the green environment:

GET /health

Expected response:
{
  "status": "healthy",
  "uptime": 45,
  "database": "connected",
  "version": "v2.3.4"
}
      

If the health check fails 3 times, CodeDeploy stops the deployment and rolls back to blue automatically.

Step 4: Monitoring & Auto-Rollback

After traffic switches to green, we monitor for 5 minutes:

• Error rate: If >1% for 30 seconds, automatic rollback
• Latency: If p95 > 1 second for 1 minute, automatic rollback
• CPU/Memory: If spikes >80%, automatic rollback

These are CloudWatch alarms that trigger CodeDeploy rollback automatically.

Database Migrations: The Tricky Part

The Problem

Blue-green is easy for stateless code, but database migrations are hard. You can't run two different schema versions against one database.

The Solution: Expand-Contract Pattern

We use the "expand-contract" pattern:

1. Expand phase: Before any code deployment, add new database columns (backward compatible)
2. Deploy code: Deploy blue-green; new code uses new schema
3. Contract phase: In next deployment, remove old columns (now unused)

Example:

// Deployment 1 (expand):
ALTER TABLE users ADD COLUMN email_new VARCHAR(255);
UPDATE users SET email_new = email WHERE email_new IS NULL;

// Blue code: Uses 'email' column (old)
// Green code: Uses 'email_new' column (new)
// Both work simultaneously during blue-green switch

// Deployment 2 (contract):
ALTER TABLE users DROP COLUMN email;
ALTER TABLE users RENAME COLUMN email_new TO email;
      

This way, both blue and green can run against the same database without conflicts.

Results: Before & After

Metric	Before (EB)	After (CodePipeline)
Downtime per deploy	30-60 seconds	0 seconds
Manual intervention	Required	Zero
Rollback time	30+ minutes	60 seconds
Deploy frequency	Multiple/week	Multiple/day (if needed)
Deployment success rate	90%	99.5%

What Went Wrong & How We Fixed It

Issue 1: Health Checks Too Strict

Problem: Health checks were failing on green because of database connection timeouts during peak traffic.

Solution: Added connection pooling to health check, increased timeout to 5 seconds.

Issue 2: Database Migrations Blocking Deployment

Problem: A large table migration locked the database, preventing both blue and green from operating.

Solution: Split large migrations into smaller batches, used expand-contract earlier in the process.

Issue 3: Cascade Rollbacks

Problem: One bad deployment caused metrics to spike, which auto-triggered rollback, which scared the team about blue-green safety.

Solution: Improved monitoring to reduce false positives (added warm-up time before metric checks).

Lessons Learned

1. Automation Beats Manual Process

Every time we automated a step, reliability increased. Manual deployments were the bottleneck.

2. Health Checks Prevent Most Problems

Good health checks catch bad deployments before they reach customers. We spent more time here than anywhere else.

3. Monitoring Enables Confidence

With automated rollback, we're not afraid to deploy. Metrics tell us if something goes wrong.

4. Database Migrations Are Deployment Constraints

Start with the database. Blue-green code changes are easy; schema changes are hard.

Conclusion

Zero-downtime deployments feel like a luxury, but they're actually a necessity once you're serving customers. Every deployment we make without downtime builds confidence in our system.

If you're still doing manual Elastic Beanstalk deployments, invest in automation. The setup takes time, but the payoff (faster iteration, fewer incidents, happier on-call engineers) is huge.