HOW TO CREATE CUSTOM EC2 VPCS IN AWS USING TERRAFORM
One of the fundamentalsof cloud administrationin AWSisknowing how tocreateacustomVirtualPrivateCloud (VPC)thatenablesthelaunchofAWS resourcessuch as EC2 instancesinto a virtual network. What gets deployed within a VPC varies across use cases,butVPCsare generally usedas a foundationforthe majority ofAWSinfrastructure componentsand services.
Automate forScalability
In thecurrentworld of cloud computing,gone are the days of hand-building infrastructure componentsmanually.Without automation, provisioning and configuration tasks will not only take days (or weeks…or months…) but are prone to errors and inconsistencies. All of the plumbing that makes up and supports an application stack can,and should be,provisioned and configured using automation toolsandinfrastructure-as-code (IaC), especiallywhendeploying applications in the public cloud. Automation allows for the ability to scale and to track deployed resources. Automation tools also make it possible to build a self-service catalogthatintegrateswithplatformslike ServiceNow. A self-service catalog can be designedwithinthese platformsto orchestrate multiple automation tools, like Terraformand Ansible, for provisioning and configuration management of cloud resources.Aself-service catalogautomates workflows and approvals to enable organizations to improve the customer experience, accelerate service delivery,and reduce operational costs. For this guide, we will be creating acustomVPC and deploying two EC2 VMsusing Terraform.Terraformis an open-sourceIaCsoftware toolthatallows cloud architects to define components and their dependencies using relatively simpledeclarative configuration files. Terraform allowsforprovisioning, modification, and decommissionofall cloud resources usingasimpleCLI workflow(write, plan, apply).Terraform has anopen-sourceversion which isfreetoinstall and use.At a larger, scalesubscription versions – Terraform Cloud and Terraform Enterprise –canbe used tomanage deployments for different projects and teams, and integrate withother platforms. Let’s get to the code! All code for thisexamplecan be found on my GitHub repo at: The code is broken into three different modules:- Networking(define the VPC and all of its components)
- SSH-Key(dynamically create an SSH-key pair for connecting to VMs)
- EC2(deploy a VM in the public subnet, and deploy another VM in a private subnet)
Module 1 –
What this code will do:- Create a custom VPC
- Define VPC name
- Create anInternetGatewayand aNAT gateway
- Define CIDR blocks
- Deploy two public subnets,acrosstwo different AZs
- Deploy two private subnets,acrosstwo different AZs
- Create two security groups (one for public, and one for private access)
Module 2 –
What this code will do:- Dynamically create an SSH Key pair that will beassociatedwiththeEC2 instances
- This SSH Key will be created dynamically, and be deleted along with all the other resourcesprovisionedwith Terraform.
Module 3 –
What this code will do:- Create at2.micro AWS Linux VM in thePUBLICܲԱ for use as a bastion/gatewayhost.
- Terraform will copy the SSH Key from your local systemto the VMand apply appropriate file permissionsto it.
- This key will be usedfor connections to instances in the private subnet
- Create at2.micro AWS Linux VM in thePRIVATEܲԱ
Note: In order to followthis tutorialyou will need to haveTerraform and AWS CLI installed and configured. To get started, clone thisto your local system and run the following commands: “tڴǰԾ”
- This will initialize the working directory that contains a Terraform configuration code with modules and plugins fromHashiCorp.
“terraform apply”
- This will first showan execution plan and reporttheresources to be deployed in AWS (23 resources in thisexample).
- Once you confirm by typing “yes,” Terraform will beginprovisioningtheVPC,EC2 instances, and theSSH-keypairin AWS.
OnceTerraform has completed provisioning resources,itwill output a string you can copy and paste to the command line in order connect to your EC2 instances.Idefinedthis tobetheoutputcode as a convenience.
Outputs:
private_connection_string=ssh-i<namespace>-key.pemec2-user@<private IP address>
public_connection_string=ssh-i<namespace>-key.pemec2-user@<public IP address>
Now you can connect to the public EC2 instance using the public connection string, and once you are logged in to that VM, you can connect to the private EC2 instance with the private connection string.
(connecting to EC2 in public subnet from local host)
(connecting to EC2 in private subnet from bastion host)
To see all the components provisioned withTerraform, log into the AWS web console, and click the VPC and EC2dashboards(make sure you are in the correct AWS region).
DeleteComponents of VPC
Imagine buildingall 23of these AWS resourcesmanually, andthenlaterneeding to make modifications to a resourcein theVPCandtrying todeterminethe dependencies between these resources to do so. Also, an environment may only need to be available briefly for dev and test, so how do you go about deleting all of these resources when they are no longer needed? How many times have you received an unexpected bill from AWS charging you for resources that you forgot to delete? To avoid that, let’sautomate this process todelete these EC2 instances and all the components that make up the newly created VPC with one command. “terraform destroy”.- The“terraform destroy”command is used to destroy the Terraform-managed infrastructure.This will ask for confirmation before destroying.
- Once you confirm by typing “yes,” Terraform will delete all of the 23 AWS resources it created earlier. (Note: This will only destroy resources provisioned from the current project,nothing else.)
Of course, you can use Terraform to deploy more complex things, like load balancers, autoscalinggroups, and Kubernetes clusters (among other things), but the intentionin this articlewas to demonstrate how a simple tool like Terraform can be used to deploy and keep track of the components of a VPC, including EC2 instances, in a public cloud.
Automation at Scale
At scale, Terraform is part of a larger automation workflow and provides additional functionality that isn’t covered here, such as keeping track of the state of deployed resources. Terraform code can be managed and deployed the same way application code is deployed, through DevOps practices and automated CI/CD pipelines using tools like ServiceNow.
(Example automation workflow with Terraform and ServiceNow using 91ý’s Automation Hub solution.)
With the right tools in place, infrastructure can be 100% defined in codeandchange managementcan beperformed through version control tools, such as GitHub. In public clouds,VMs can be handled as ephemeral resourcesthe same wayascontainers, and these VMs can beversioned,deployed,and released like application code in an agile framework,suchas part of a CI/CD pipeline.Using these practices, security should also be integrated and automated throughout the stack as well. Adopting these practices will eventually help you and your team achieve theoperational modelneeded for deploying cloud-native applicationsand achieving cloud scalability.
For more resources and best practices on scaling cloud adoption, download our complimentaryCloud @Scale Playbook.