End to End CI/CD pipeline using AWS Cloud for my Portfolio Website🔥
Table of contents
📍 Introduction
The concept of Continuous Integration (CI) and Continuous Deployment (CD) has become increasingly popular in the software development industry, as organizations strive to streamline their software delivery processes and reduce errors. CI/CD are practices that aim to automate the process of building, testing, and deploying software, enabling developers to deliver new features and bug fixes to production faster and with greater confidence. Implementing CI/CD pipelines has been made easier by the usage of cloud computing platforms like Amazon Web Services (AWS).
✔ Features of Project:
1. Comprehensive Study: The Report provides a comprehensive analysis of the steps involved in implementing a complete CI/CD pipeline using AWS. The study covers all the key components of a CI/CD pipeline and explains how they can be automated using various AWS services. This allows organizations to understand the end-to-end process of creating a CI/CD pipeline, from code management to deployment.
2. The Benefits of using AWS: The Report highlights the key benefits of using AWS to create a CI/CD pipeline. These benefits include increased speed, as automation reduces the time taken for manual processes, reduced errors, as automation eliminates human error, improved collaboration, as developers can work together on the same source code,
3. scalability: Organizations can scale their infrastructure as required, and enhanced security, as AWS implements various security measures to ensure the safety of customer data.
4. Case Study: The Report includes a case study that demonstrates how a complete CI/CD pipeline can be created using AWS. This provides organizations with a practical example of how the different components of a CI/CD pipeline can be automated using AWS services. The case study also provides recommendations for organizations looking to adopt a similar approach, allowing them to learn from the experience of others.
5. AWS Services: The Report explores the various AWS services that can be used to automate the different stages of a CI/CD pipeline, such as GitHub for code management, AWS CodeBuild for building and testing code, and AWS CodeDeploy for deployment. The Report explains how each of these services can be used and how they integrate to create a complete CI/CD pipeline.
6. Recommendations: The Report ends with a list of suggestions for businesses wishing to use Amazon to construct a full CI/CD pipeline. These suggestions include advice on best practices, such as how to choose the proper Amazon services, how to assure security, and how to grow the infrastructure to meet the requirements of the software delivery process.
7. Security: The Report also addresses the security aspect of using AWS for a CI/CD pipeline, highlighting the various measures implemented by AWS to ensure the safety of customer data. These measures include network security, data encryption, and access control.
8. Scalability: While discussing the scalability of utilising Amazon for a CI/CD pipeline, the Report emphasises how simple it is for businesses to extend their infrastructure to match the needs of their software delivery process. With AWS services, businesses can scale their infrastructure up or down as needed without having to make substantial initial hardware expenditures.
✔ Tools involve in the project:
AWS CodeBuild: This completely managed-to-build service compiles source code, performs tests, and generates software packages that are prepared for deployment. You won't need to set up, maintain, or grow your build servers anymore, freeing you your time to focus on creating code rather than taking care of infrastructure.
AWS CodeDeploy: It is a fully-managed deployment service that automates software deployments to a variety of computing services, including Amazon EC2 instances, AWS Lambda functions, and on-premises servers. It eliminates the need for you to manually deploy your application code and ensures that your deployments are consistent, error-free, and repeatable.
AWS CodePipeline: It is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates. It orchestrates and automates the different stages of your release process, including building, testing, and deploying your code changes.
AWS S3: It is a cloud-based object storage service offered by Amazon Web Services (AWS). It provides scalable and secure storage for a wide range of use cases, such as data backup, content distribution, and application data storage. In the context of a CI/CD pipeline, S3 is often used for storing artifacts and deploying applications
AWS EC2: It is a web service that provides resizable computing capacity in the cloud. For developers, it is intended to make web-scale cloud computing simpler. To meet the unique requirements of the project, EC2 instances can be deployed with a number of operating systems and programmes.
AWS IAM: It is a web service that helps you securely control access to AWS resources for your users. IAM allows you to create and manage AWS users and groups, and control their access to AWS services and resources using permissions. You can use IAM to manage who can sign in to your AWS account, create and manage AWS users, groups, and roles, and assign granular permissions to users and groups to access specific resources.
AWS VPC: It is a service that lets you launch AWS resources into a virtual network that you define. This virtual network is logically isolated from other virtual networks in the AWS cloud, giving you complete control over your virtual networking environment. With VPC, you can create a virtual network topology that closely resembles a traditional network that you might operate in your own data centre, with multiple subnets, routing tables, and access control policies.
AWS CloudWatch: It is a monitoring and management service provided by Amazon Web Services (AWS) for applications, infrastructure, and services running on the cloud. CloudWatch collects and tracks metrics, collects and monitors log files, and sets alarms. It allows users to gain system-wide visibility into resource utilization, application performance, and operational health.
GitHub: It is a popular web-based platform that allows developers to collaborate on and manage software projects using the Git version control system. It provides a range of features for source code management, including version control, issue tracking, project management, code review, and collaboration tools.
📍 Project Creation step by step:
✔ Step 1: Creating Pre-requisites:
IAM Role Creations:
i. A role for CodeBuild: AWSCodePipelineServiceRole-us-east-1-website_pipeline
Policy Names:
AmazonS3FullAccess
CloudWatchFullAccess
AWSCodeBuildAdminAccess
ii. A role for CodeDeploy: devopsCodeDeployRole [5]
Policy Names:
- AmazonS3FullAccess
iii. A Role for EC2 Instance: CodeDeploy-Role-for-EC2 [7]
Policy Names:
AmazonEC2FullAccess
AmazonEC2RoleforAWSCodeDeploy
AmazonS3FullAccess
AWSCodeDeployFullAccess
AWSCodeDeployRole
AmazonEC2RoleforAWSCodeDeployLimitation
✔ Step 2: Now, it’s time to create An EC2 Instance & configure it:
i. Installing CodeDeploy on EC2
Steps to follow along:
Connect to the Instance using:
Putty & Putty-gen Software
Command Prompt or PowerShell by following the below:
Connect to the SSH client and copy the Example text.
Open the File Manager and locate the folder where you have downloaded your Key-pair. In the search bar, type
cmdand hit the Enter button. Now, paste the Example text to connect with the EC2 instance.
We have successfully logged in to our console.
Now first, update the instance, and then we will create a shell script to install the CodeDeploy agent. It's even possible by entering single commands one by one, but it's always a good practice to execute using the shell script.
Save and exit from the vi (editor) by pressing
esc,:,wq!.Now, we will execute the shell script to install the CodeDeploy Agent.
- Run ./install.sh command and press enter
Validate CodeDeploy agent is running or not using the following command:
sudo service codedeploy-agent status
✔ Step 3: Creating the Build Stage
a. Source Stage Configuration:
b. Build Stage Configuration:
c. Deploy Stage Configuration:
d. Reviewing the Pipeline:
e. Created Pipeline with all the stages defined:
f. Encountered Unknown error due to lack of CodeDeploy agent on EC2 Server:
This problem was caused by us failure to install the Code Deploy agent on our EC2 server:
Let us Install and Configure CodeDeploy Agent:
g. Below are commands to install it:
✔ Step 6: Source Code Modification to Re-run the Pipeline:
a. Source Code After Modification:
b. Committed & pushed the changes to the repository:
c. Our website is live and running using the Public Ip of our EC2 Instance:
d. Server Monitoring:
📍 Results and Discussion
With the proposed solution, the development team can easily track changes in the code repository and collaborate on the same source code. This has resulted in faster development and delivery of new features and bug fixes to production, reducing the time to market. Automated testing using AWS CodeBuild has significantly reduced the occurrence of errors, as it eliminates human error in the testing process. This has led to increased confidence in the software delivery process and improved customer satisfaction. The scalability of the proposed solution has been a significant advantage, as organizations can easily scale their infrastructure as required without having to make substantial initial hardware expenditures. This has allowed small startups to compete with large enterprises, providing a level playing field in the software development industry. The security measures implemented by AWS have ensured the safety of customer data, protecting it from unauthorized access and cyber-attacks. This has increased customer trust and confidence in the software delivery process. Overall, the proposed solution of implementing a CI/CD pipeline using AWS services has shown significant improvements in the software delivery process, reducing errors, increasing speed, and improving collaboration, scalability, and security.
✔ Applications:
📍 CONCLUSION
In conclusion, implementing a complete CI/CD pipeline using AWS can provide significant benefits to organizations looking to streamline their software delivery processes. By automating the different stages of the pipeline using services such as GitHub, AWS CodeBuild, and AWS CodeDeploy, organizations can speed up cooperation, decrease the likelihood of errors, and cut down on the time and effort needed to deploy software.
The proposed solution for the implementation of the CI/CD pipeline using AWS is a comprehensive approach that includes several stages, from source code management to testing and deployment. The architecture of the proposed solution includes various AWS services, such as GitHub, CodeBuild, CodeDeploy, and CloudFormation, which are combined to create a complete CI/CD pipeline.
🔍 Checkout GitHub Repository for projects:
