Problem / Statement Definition

Medmo Inc. migrated from their existing legacy infrastructure swiftly due to EOL security related issues. Medmo Inc. provides services in the Healthcare sector and needed to maintain HIPAA compliance on AWS and move on from its existing infrastructure.

Proposed Solution and Architecture

Triumph Tech used DevOps and DevSecOps to deploy a HIPAA compliant AWS Architecture and to migrate from the existing Legacy environment.

Triumph Tech chose AWS to provide the resources required to fix the problem.

We used CloudFormation to deploy all of our resources. This included:

• 1 x code commit repo – used to store the CFN
• 1 x VPC (2 public subnets route to internet via IGW, 2 private subnets route to internet via NAT Gateway, 2 VPC only subnets with only local VPC route)
• 1 x prod ALB has certs for and routes
• 1 x Staging ALB has certs for and routes
• 1 x prod ECS Cluster
• 1 x staging ECS Cluster
• 3 x ECR (frontend, admin, api) [Shared between staging and prod via tags [‘stable’, ‘master’] matching github
• 3 x prod ECS tasks (frontend, admin, api)
• 7 x code pipeline:
  • 1 x medmo-main-prd-CFN = Used for CloudFormation lint testing, approval and deployment – triggers on a commit to code commit repo ‘medmo-main-prd-CFN’
  • 6 x medmo${APPNAME}-{$Environment} = Used to build a docker image and place in ECR, tag image with github commit, manage release approval and tag with branch name, create ECS deployment from github repo medmoinc/${AppRepo} triggers on a commit to ${Environment} branch

We automated security scans using Trivy within CodeBuild. Scan results were sent to AWS Security Hub.

Problem / Statement Definition

• We researched the legacy environment in order to understand Medmo’s application stack.
• We discovered that the current architecture was at the end of life (EOL) and needed to be migrated soon to maintain HIPAA compliance.
• Project success was determined by migrating into an AWS-maintained environment.
• Deployments and security scans were automated, and most importantly, the environment was HIPAA compliant.

Describe TCO Analysis Performed

Medmo, Inc. has been a current AWS customer, so TCO analysis was performed by collecting data from the AWS Billing Console.

Lessons Learned

Automated solutions are the optimal ways to reduce time to market. Automated security scanning makes maintaining compliance more affordable. CloudFormation offers repeatable deployments of customer environments.

Summary Of Customer Environment

Cloud environment is native cloud. The entire stack is running on Amazon Web Services in the US-East-2 region.

AWS Account Configuration

• Root User is secured and MFA is required. IAM password policy is enforced.
• Operations, Billing, and Security contact email addresses are set and all account contact information, including the root user email address, is set to a corporate email address or phone number.
• AWS CloudTrail is enabled in all regions and logs stored in s3.
• Security HUB is integrated with CodePipeline Operational Excellence.

Metric Definitions

CodePipeline Health Metrics

If any step within the pipeline fails, notifications are sent to the DevOps Slack channel. SNS topics and Slack’s AWS Chatbot integration achieves this.

Container Metrics

Container health is handled at the orchestration level. We set up health checks to monitor specific ports and endpoints within our containers, and we check for a 200- or predetermined response code.

ELB Target Group Metrics

Unhealthy targets are identified as targets who are not passing ELB health checks.

Security Compliance Metrics

If a critical vulnerability is found during the scan within CodeBuild, a non-zero exit code is produced and no artifact is published to the ECR. Results are formatted and piped out to Security Hub, so the application team can patch the environment.
For warnings and information, results are published to Security Hub and the artifacts are published to ECR.

Metric Collection and Analytics

We consult with clients on best practices in terms of log / metric collection. Application logs are integrated with CloudWatch.
Security-related logs are integrated with Security Hub.

Operational Enablement

Enabling the client to manage and to maintain the DevOps pipeline after handover is of the utmost importance. Our goals are to minimize the maintenance of the automation and for all members of the Development team to simply push code, follow a development process, and know that their applications are being tested and rapidly deployed.

Enabling the client to manage and maintain the DevOps pipeline after handover is of the utmost importance. Our goals are to:

• Minimize the maintenance required with the level of automation.
• Allow the Development Team to push code, follow a development process, and know that their applications are tested and rapidly deployed.
• Training and handover with documentation specific to the customer’s workload, outlining the development lifecycle.
• Document how to version IAC modules / templates that were developed and push out updates to their infrastructure.
• Provide architecture diagrams that outline the branch strategy / git workflow.
• Schedule a video conference, and conduct a hands-on session with the client, going over how to push application updates throughout the development, staging, and production environments.
• Review the development workflow and branching strategy.
• Show clients how to troubleshoot a failed pipeline build within CodeBuild.
• Teach clients on where to find all relevant logs of their build and test stages within CodePipeline. (The majority of DevOps related troubleshooting tasks after the creation of a CI / CD automation pipeline will be found within the CodeBuild logs and fixed at the application layer.)
• Outline common troubleshooting scenarios via videoconferencing that the client will run into and show them how to effectively troubleshoot the workload.
• We go over each and every component of the infrastructure and CI / CD pipeline that was developed with the client and allow them time to ask any questions.

Security: Identity and Access Management

Access Requirements Defined

In order to discover access requirements, we look at the organizational units within the client’s business; this is required to access the DevOps infrastructure. We identify developers, systems engineers, security engineers, and stakeholders. We have previously defined best practices that we follow for each of these groups.

IAM groups are created for each of these Organizational Units and at least privilege access is granted to each. Each group is only granted access to what they actually need.

Developer Policy

Access Requirements Defined

Our developer policy looks like this:

{
    "Version": "2012-10-17",
    "Statement": [
    	{
            "Action": [
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupEgress"
        	],
            "Resource": "arn:aws:ec2:*:*:*",
            "Effect": "Allow"
    	},
    	{
            "Action": [
                "ec2:Describe*",
                "iam:ListInstanceProfiles",
                "mgh:CreateProgressUpdateStream",
                "mgh:ImportMigrationTask",
                "mgh:NotifyMigrationTaskState",
                "mgh:PutResourceAttributes",
          	  "mgh:AssociateDiscoveredResource",
                "mgh:ListDiscoveredResources",
                "mgh:AssociateCreatedArtifact",
                "discovery:ListConfigurations"
        	],
            "Resource": "*",
            "Effect": "Allow"
        },
    	{
            "Action": [
                "ec2:CreateSecurityGroup",
                "ec2:ModifyInstanceAttribute",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:AttachVolume",
                "ec2:DetachVolume",
                "ec2:DeleteVolume",
                "ec2:CreateImage"
        	],
            "Resource": "*",
            "Effect": "Allow"
    	},
    	{
            "Condition": {
                "ForAllValues:StringLike": {
                    "ec2.ResourceTag/appenv": [
                        "rmmigrate-dta"
                	]
            	}
        	},
            "Action": [
                "ec2:TerminateInstances",
                "ec2:StartInstances",
                "ec2:StopInstances",
                "ec2:RunInstances"
        	],
            "Resource": "*",
            "Effect": "Allow"
    	},
    	{
            "Action": "iam:PassRole",
   	     "Resource": "*",
            "Effect": "Allow"
    	}
	]
}

No processes deployed to AWS infrastructure will make use of static AWS credentials. All instances that call other AWS functions use roles. Static AWS credentials are only used to call AWS services when third party integrations can not make use of assumed roles.

Log into AWS for each APN partner and user of the platform, and make use of unique IAM users or federated logins. No root access is permitted. We have a CloudWatch alarm setup that triggers an SNS notification via email anytime the root user logs in.

Security: Networking

The following Security Group example meets the requirements of:

• Restricting traffic between the internet and VPC
• Restricting traffic within the VPC
• Only allows access from the Security Group

Security IT / Operations

Components which require encryption in order to maintain HIPAA compliance on AWS:

• ECS Task Definition Variables: These are encrypted at rest using KMS and stored in Secrets Manager.
• RDS Mysql Instance: Data Encrypted at Rest using KMS.
• DevSecOps – Integration of Trivy and Security Hub.

Trivy vulnerability scans run before an artifact is published to ECR. If a Critical Vulnerability is found, the container image artifact will not be published nor deployed until patched. This ensures HIPAA Compliance on AWS.

AWS API Integration
AWS CLI is used for all programmatic access.

Reliability

Deployment Automation

The deployment process is fully automated. When we merge a change into the master branch from development within GitHub, CodePipeline is triggered. CodePipeline first runs CodeBuild and builds an application artifact using Docker. A Docker image is pushed to the ECR and then deployment is approved for production.

Vulnerability scans are fully automated using Trivy which is integrated with Codebuild. Vulnerability scan results are published to Security Hub.

All AWS components are deployed using CloudFormation.

Availability Requirements

• RTO: Application needs to be available 24 / 7

Any application downtime will result in a critical impact to the business.
• RPO: 24 Hours

Data is backed up every 12 hours, so even in extreme conditions, only one day’s worth of data may be lost as opposed to the entirety.

Adapts to Changes in Demand

We use Elastic Container Service (ECS) Auto Scaling to respond to changes in demand.

Cost Optimization

Cost Modeling

We deployed the workload into a development environment to run a test. We ran a load test over a period of 30 minutes, recorded the number of tasks that were deployed by the service, and estimated the cost based on CPU / memory allocation in AWS Fargate. We took the outcome and reduced it by 50% to account for off-peak hours.

Cost for Fargate containers during peak hours is: $498.75. This accounts for a total of 35 containers.

• Total cost to run workload on Fargate each month is: $247.87
• The remainder of resources were calculated using AWS SMC.
• Total cost for remaining resources (RDS, ALB) each month is: $107.11
• Total cost to run annually: $4,259.76

Looking to implement HIPAA Compliance on AWS? Contact one of our DevOps Professionals today.