Portfolio Details

Problem Statement:

Our Client needed to optimize its deployment processes for managing over 30 productive multi-tenant environments. Each client had its own AWS account, creating challenges related to:

• Complex Deployment Processes: Manual deployments across multiple AWS accounts led to inconsistencies and high error rates.
• Scalability Issues: The infrastructure needed to scale effectively to support the growing number of tenants and environments.
• Security and Compliance: Ensuring secure deployment processes while maintaining strict isolation between client environments.
• Operational Overhead: A small SRE team managed the deployments, necessitating a more efficient and automated solution.

Proposed Solution & Architecture:

Unified Technologies designed and implemented a CI/CD pipeline to automate and standardize deployments across all client environments. The solution leveraged AWS services and AWS Cloud Development Kit (CDK) with TypeScript to achieve a “one-click deployment” approach, minimizing manual effort and reducing errors.

CI/CD Pipeline Flow Overview:

Architecture:

Continuous Integration (CI) Part:

1. Source Control and Code Commit (Bitbucket):
   • Tool: Bitbucket
   • Description: Developers commit code changes to a Bitbucket repository, which triggers the CI/CD pipeline. The pipeline integrates directly with Bitbucket to handle source code management and initiate automated testing and validation steps.
2. Code Quality Checks:
   • Unit Tests and Lint Tests:
     • Tool: Bitbucket Pipelines
     • Description: Bitbucket Pipelines executes unit tests and lint tests to validate code quality and ensure adherence to coding standards.
   • SonarQube Scanning:
     • Tool: SonarQube
     • Description: The pipeline integrates with SonarQube for static code analysis, scanning for vulnerabilities, code smells, and maintainability issues. The pipeline stops if critical issues are detected, ensuring only high-quality code proceeds.
3. Artifact Creation and Storage:
   • Create Artifact:
     • Tool: Bitbucket Pipelines
     • Description: After successful completion of tests and scans, Bitbucket Pipelines packages the application code and AWS CDK resources into a deployable artifact (CDK zip file).
   • Store Artifact:
     • Tool: AWS S3
     • Description: The artifact is uploaded to a central S3 bucket, serving as a version-controlled repository accessible to all client environments.

Continuous Deployment (CD) Part:

1. Deployment Orchestration:
   • One-Click Deployment:
     • Tool: AWS CodePipeline
     • Description: AWS CodePipeline is configured to provide a “one-click deployment” mechanism, automatically deploying the artifact to the appropriate client environments upon artifact storage in S3.
2. Environment Management Across Multiple AWS Accounts:
   • Cross-Account Deployment:
     • Tool: AWS CodePipeline and AWS CodeBuild
     • Description: AWS CodePipeline orchestrates deployments across over 30 client AWS accounts, leveraging AWS CodeBuild to execute environment-specific deployment commands. The pipeline uses AWS IAM roles and policies to securely access each AWS account, ensuring strict separation and isolation between environments.
   • Environment-Specific Builds:
     • Tool: AWS CodeBuild
     • Description: Each environment has its own build configuration, allowing customized deployments per client account. This setup ensures that each client’s unique requirements and configurations are respected during deployments.
3. Blast Radius Management:
   • Controlled Deployments:
     • Tool: AWS CodeBuild and IAM Roles
     • Description: Deployments are executed with environment-specific IAM roles, limiting permissions to only necessary resources within each client AWS account. This minimizes the blast radius of any deployment errors, containing them within the respective client environment.

CI/CD Flow Overview

Bitbucket Side:

1. Tests:
   • Tool: Bitbucket Pipelines
   • Description: Automated tests are run to ensure code quality and functionality.
2. Build:
   • Tool: Bitbucket Pipelines
   • Description: If tests pass, the code is built into an artifact (e.g., a .zip file).
3. Artifact Upload:
   • Tool: AWS S3
   • Description: The built artifact is uploaded to an S3 bucket in the central AWS account.

AWS Central Account (S3 and EventBridge):

1. S3 Event Notification:
   • Tool: AWS S3 and AWS EventBridge
   • Description: An event notification is triggered upon artifact upload and sent to AWS EventBridge.
2. EventBridge Rule Execution:
   • Tool: AWS EventBridge
   • Description: EventBridge rules filter the incoming events based on metadata and trigger the appropriate rule.
3. Cross-Account Event Forwarding:
   • Tool: AWS EventBridge
   • Description: The notification is forwarded to the target AWS account’s EventBridge service using EventBridge resource policies.

Target AWS Account:

1. Event Rule Trigger:
   • Tool: AWS EventBridge
   • Description: In the target AWS account, the EventBridge rule is triggered upon receiving the forwarded event.
2. Lambda Execution:
   • Tool: AWS Lambda (assets_tracker_lambda function)
   • Description: The triggered rule invokes the assets_tracker_lambda function, which parses the artifact version from the event metadata and triggers an AWS CodeBuild job with the artifact version as an environment variable.

AWS CodeBuild:

1. Artifact Retrieval:
   • Tool: AWS CodeBuild
   • Description: CodeBuild uses the artifact version to identify and retrieve the correct artifact from the central S3 bucket.
2. Artifact Extraction:
   • Tool: AWS CodeBuild
   • Description: The artifact is pulled, extracted, and prepared for deployment.
3. Deployment:
   • Tool: AWS CodeBuild
   • Description: CodeBuild deploys the artifact changes to the designated environment or service.

Metrics for Success:

• Deployment Time Reduction: Achieved a 75% reduction in deployment times through the “one-click deployment” approach.
• Error Rate: Reduced configuration and deployment errors by 90% using automated CI/CD processes.
• Infrastructure Consistency: Ensured 100% consistency across over 30 client environments using AWS CDK and centralized artifact storage.
• Operational Efficiency: Enabled a small SRE team to manage deployments across 30+ environments with minimal manual intervention.

Lessons Learned:

• Scalable Automation is Key: Automating deployments across multiple AWS accounts significantly reduced operational overhead and error rates.
• Environment-Specific Management Improves Security: Using distinct AWS accounts and environment-specific configurations enhanced security and compliance while maintaining flexibility for individual client needs.
• Centralized Artifact Management Ensures Consistency: A centralized S3 bucket for artifact storage provided a single source of truth, ensuring uniform deployments across all environments.
• “One-Click Deployment” Boosts Efficiency: Achieving a “one-click deployment” mechanism greatly simplified the deployment process, reducing manual errors and enhancing agility.