AWS CodePipeline CI/CD 구축Building CI/CD with AWS CodePipeline

CI/CD

CI/CD is a core methodology in modern software development. CI (Continuous Integration) means automating the process of developers regularly merging their code changes into the main branch, and CD stands for Continuous Delivery and Continuous Deployment.

CI (Continuous Integration)

• Regularly integrates code changes from multiple developers
• Runs automated builds and tests
• Keeps code quality up and catches bugs early

CD (Continuous Delivery/Deployment)

• Continuous Delivery: always keeps the code in a state that can be deployed to production
• Continuous Deployment: deploys automatically all the way to production

React App + Docker NGINX

Source code

This code builds a React app and runs it in a Docker NGINX container. We’ll deploy it to EC2 with AWS CodePipeline.

You can check it locally with the commands below.

$ docker build -t docker-react .

$ docker run -it -d -p 8080:80 --name web docker-react

AWS CodePipeline

AWS CodePipeline is a fully managed AWS service for modeling and automating CI/CD stages. Every time code changes, it automatically runs the build, test and deploy stages, enabling reliable application updates.

AWS CodePipeline architecture

Here we’ll use AWS CodePipeline to wire the following services into each stage:

• AWS CodeCommit
• AWS CodeBuild
• AWS CodeDeploy

We’ll hook up git with AWS CodeCommit, build and test with AWS CodeBuild, and deploy the code to EC2 with AWS CodeDeploy.

Artifacts

The output produced by each stage is called an artifact. Pay close attention to how these artifacts are created and used. Follow that flow and CI/CD becomes much easier to understand!

CI/CD Architecture

1. CI/CD built around AWS CodePipeline
2. AWS CodeBuild builds the Docker image and pushes it to Docker Hub
3. AWS CodeDeploy pulls the Docker image and deploys it to Amazon EC2

AWS CodeCommit

AWS CodeCommit is AWS’s git-based version control service. Think of it as something similar to GitHub. Through AWS CodePipeline, source code from AWS CodeCommit or GitHub can be pulled into S3 as an artifact.

Caveats

Existing accounts can also create more repositories only if they already have an AWS CodeCommit repository.

Still, some companies continue to use AWS CodeCommit, so here we’ll connect to AWS CodeCommit over SSH. If you’re using GitHub or GitLab, skip ahead to AWS CodeBuild!

Create a repository

In AWS CodeCommit, enter a repository name and create it.

> Creating a git repository with AWS CodeCommit

Upload an SSH public key

Let’s connect to the AWS CodeCommit repository over SSH. You’ll see it works exactly the same way as GitHub.

Go to IAM - Users - Security credentials - Upload SSH public key for AWS CodeCommit. Register the public key (id_rsa.pub) from the asymmetric key pair generated with the command below. After uploading the public key you’ll get an SSH key ID, which is the value you’ll use in your SSH config.

$ ssh-keygen -t rsa

SSH config

Write the following in your SSH config file:

• Host : git-codecommit.*.amazonaws.com
• User : SSH key ID
• IdentityFile : the private key of the asymmetric key pair

Host git-codecommit.*.amazonaws.com
    User APKAZN6Q
    IdentityFile C:\Users\tiaz0128\.ssh\id_rsa

Connect the git remote

Copy the Clone URL - SSH address of the repository created in AWS CodeCommit and register it as a git remote.

$ git remote add aws ssh://git-codecommit.ap-northeast-2.amazonaws.com/v1/repos/docker-react

AWS CodeBuild

AWS CodeBuild compiles source code and runs tests in Docker containers. You can also configure the build environment yourself with a custom Docker image. It uses the source artifact and runs according to the phases defined in the buildspec.yml file.

• Compile / build source code
• Run tests

Create a project

1. In AWS CodeBuild, go to Build projects - Create project.
2. For Source provider, choose AWS CodeCommit or GitHub.
3. Under Environment - Additional configuration, select Enable this flag if you want to build Docker images or want your builds to get elevated privileges.
4. Under Environment - Additional configuration, use Create parameter to create the sensitive environment variables.
   • DOCKER_USERNAME : your Docker Hub ID
   • DOCKER_PASSWORD : your Docker Hub password
5. Under Buildspec - Build specifications, select Use a buildspec file
6. Artifacts - Type - No artifacts

buildspec.yml

buildspec.yml is a YAML file that defines how the project is built, phase by phase. The main points are:

• Use environment variables stored in the AWS Systems Manager parameter store
• Build the Docker image and push it to Docker Hub
• Copy the files AWS CodeDeploy will use into the artifacts folder

For the details of each section, see the build specification reference for CodeBuild.

version: 0.2
env:
  parameter-store:
    DOCKER_USERNAME: "/CodeBuild/DOCKER_USERNAME"
    DOCKER_PASSWORD: "/CodeBuild/DOCKER_PASSWORD"
    
phases:
  pre_build:
    commands:
      - mkdir -p artifacts    # create the artifacts directory
      - cp -r scripts/ artifacts/
      - cp appspec.yml artifacts/
      - ls -al artifacts
  build:
    commands:
      - docker build -t $DOCKER_USERNAME/docker-react .
  post_build:
    commands:
      - docker login docker.io -u $DOCKER_USERNAME -p $DOCKER_PASSWORD
      - docker push $DOCKER_USERNAME/docker-react

artifacts:
  files:
    - "**/*"
  base-directory: artifacts

AWS CodeBuild architecture

As described above, it builds the Docker image and pushes it to Docker Hub. Then it passes the files needed by the next stage, AWS CodeDeploy, along as an artifact.

> Building a Docker image with AWS CodeBuild and pushing it to Docker Hub

AWS CodeDeploy

AWS CodeDeploy is a service that automates application deployment. Based on tags, it runs the appspec.yml deployment script through the CodeDeploy agent on the target EC2 instances.

Create a CodeDeploy role

Let’s create the role AWS CodeDeploy needs for deployments.

• AWSCodeDeployRole

Create an application

First, in AWS CodeDeploy, go to Application configuration - Compute platform, select EC2/On-premises, and create the application.

Deployment group

Next, create a group of target instances within the application.

1. Service role - select the CodeDeploy role you created
2. Deployment type - select In-place
3. Environment configuration - select Amazon EC2 instances
4. Environment configuration - Tag group - enter the key and value
   • ⚠️Note: type Name with exactly this capitalization: Name.
   • Name : fe-server
5. Load balancer - uncheck Enable load balancing

appspec.yml

appspec.yml is the core configuration file of AWS CodeDeploy; it defines the deployment process and steps. See the CodeDeploy AppSpec file reference.

• Runs a hooks script at each defined lifecycle event
• BeforeInstall: stops the existing Docker container with the cleanup.sh script
• AfterInstall: pulls the new Docker image and runs the container with the update_container.sh script

version: 0.0
os: linux
files:
  - source:  /
    destination: /home/ec2-user
    overwrite: true
file_exists_behavior: OVERWRITE

permissions:
  - object: /
    pattern: "**"
    owner: ec2-user
    group: ec2-user

hooks:
  BeforeInstall:
    - location: scripts/cleanup.sh
      runas: ec2-user
  AfterInstall:
    - location: scripts/update_container.sh
      runas: ec2-user

Target EC2

AWS CodeDeploy sends the artifact to the CodeDeploy agent on EC2 instances with the specified tag. The deployment happens when the scripts are run via the appspec.yml inside the artifact.

> The appspec.yml file runs on EC2 instances with the specified tag

EC2 name and tags

Create the target EC2 instance with Name and tags - fe-server. AWS CodeDeploy finds its deployment targets based on the EC2 tags specified in the tag group, so it’s important to enter the tag exactly when you create the instance.

> AWS CodeDeploy finds target EC2 instances by name and tags.

Attach a role to EC2

To use AWS CodeDeploy from EC2, create the role it needs and attach it.

• AmazonEC2RoleforAWSCodeDeploy

Install Docker

Install Docker on the EC2 instance.

$ sudo yum -y install docker

$ sudo systemctl restart docker

$ sudo usermod -a -G docker ec2-user

Install the CodeDeploy agent

Finally, install the agent that lets the EC2 instance talk to AWS CodeDeploy. Follow the CodeDeploy agent installation guide to install it.

$ wget https://aws-codedeploy-ap-northeast-2.s3.ap-northeast-2.amazonaws.com/latest/install

$ chmod +x ./install

$ sudo ./install auto

You can check the main agent log with the command below. Double-check that the EC2 instance’s role has the AmazonEC2RoleforAWSCodeDeploy permission.

$ sudo tail -f /var/log/aws/codedeploy-agent/codedeploy-agent.log

Wiring It Up with AWS CodePipeline

Now connect the services to each stage with AWS CodePipeline.

1. Create pipeline - Build custom pipeline
2. Add source stage - for Source provider, choose AWS CodeCommit or GitHub
3. Add build stage - Other build providers - select AWS CodeBuild
4. Add deploy stage - select AWS CodeDeploy, then choose the Application name / Deployment group

Check the deployment

Whenever git changes, you’ll see the Docker image get built and deployed, and the container replaced.

Finally, you can check the log with the command below to confirm the deployment to EC2 went through properly.

$ sudo tail -f /opt/codedeploy-agent/deployment-root/deployment-logs/codedeploy-agent-deployments.log

Wrapping Up

We’ve now built CI/CD with AWS CodePipeline for source code that uses Docker.

• Connected git to AWS CodeCommit
• Built and pushed the Docker image with AWS CodeBuild
• Deployed to each EC2 instance with AWS CodeDeploy

Each service has many more features, so give them a try. Going further, it would be great to level up your CI/CD by combining it with other AWS services and third-party tools! 😊