AWS CloudFormation

AWS CloudFormation gives developers and systems administrators an easy way to create, manage, provision, and update a collection of related AWS resources in an orderly and predictable way. AWS CloudFormation uses templates written in JSON or YAML format to describe the collection of AWS resources (known as a stack), their associated dependencies, and any required runtime parameters. Users can use a template repeatedly to create identical copies of the same stack consistently across AWS Regions. After deploying the resources, users can modify and update them in a controlled and predictable way. In effect, users are applying version control to the AWS infrastructure the same way as an application code.

  • The templates require a specific syntax and structure that depends on the types of resources being created and managed. Users author the resources in JSON or YAML with any code editor such as AWS Cloud9, check it into a version control system, and then CloudFormation builds the specified services in safe, repeatable manner.
  • A CloudFormation template is deployed into the AWS environment as a stack. Users can manage stacks through the AWS Management Console, AWS Command Line Interface, or AWS CloudFormation APIs.
  • AWS CloudFormation makes it easy to organize and deploy a collection of AWS resources and enables users describe any dependencies or pass in special parameters when the stack is configured.
  • With CloudFormation templates, you can work with a broad set of AWS services, such as Amazon S3, Auto Scaling, Amazon CloudFront, Amazon DynamoDB, Amazon EC2, Amazon ElastiCache, AWS Elastic Beanstalk, Elastic Load Balancing, IAM, AWS OpsWorks, and Amazon VPC
AWS CloudFormation

AWS CloudFormation Benefits

Manage resource scaling by sharing CloudFormation templates to be used across users organization, to meet safety, compliance, and configuration standards across all AWS accounts and regions. Templates and parameters enable easy scaling, so that the best practices and company policies would be shared. Additionally, CloudFormation StackSets enables users to create, update, or delete stacks across multiple AWS accounts and Regions, with a single operation.

To further automate resource management across the organization, users can integrate CloudFormation with other AWS services, including AWS Identity and Access Management (IAM) for access control, AWS Config for compliance, and AWS Service Catalog for turnkey application distribution and additional governance controls. Integrations with CodePipeline and other builder tools enable users to implement the latest DevOps best practices and improve automation, testing, and controls.

Users can create or modify an existing AWS CloudFormation template to describes all resources and their properties. When using template to create an AWS CloudFormation stack, AWS CloudFormation provisions the Auto Scaling group, load balancer, and database for users. After the stack has been successfully created, the AWS resources will be up and running. Users can delete the stack just as easily, which deletes all the resources in the stack. AWS CloudFormation easily manage a collection of resources as a single unit.

The AWS CloudFormation GitHub organization offers open source projects that extend CloudFormation’s capabilities. The CloudFormation Registry and CloudFormation CLI let users define and create resource providers to automate the creation of resources safely and systematically. Using CloudFormation GitHub projects, users can do things like check CloudFormation templates for policy compliance (using cfn-guard), or validate use of best practices (using cfn-lint).

AWS CloudFormation Features

AWS CloudFormation gives an easy way to model a collection of related AWS and third-party resources, provision them quickly and consistently, and manage them throughout their lifecycles, by treating infrastructure as code. A CloudFormation template describes the desired resources and their dependencies so users can launch and configure them together as a stack. Users can use a template to create, update, and delete an entire stack as a single unit, as often as necessary, instead of managing resources individually. Users can manage and provision stacks across multiple AWS accounts and AWS Regions.

  • Using the AWS CloudFormation Registry, users  can model and provision third-party such as monitoring, team productivity, incident management, CI/CD, and version control applications alongside AWS resources. Using the open source CloudFormation CLI, users can build CloudFormation resource providers – native AWS types published as open source.
  • Users can build their own resource providers using the AWS CloudFormation CLI, an open-source tool that streamlines the development process, including local testing and code generation capabilities.

AWS CloudFormation Designer (Designer) is a graphic tool for creating, viewing, and modifying AWS CloudFormation templates. With Designer, users can diagram the template resources using a drag-and-drop interface, and then edit their details using the integrated JSON and YAML editor. For both new and experienced AWS CloudFormation user, AWS CloudFormation Designer can help quickly see the interrelationship between a template’s resources and easily modify templates.

  • CloudFormation allows users to model the entire cloud environment in text files. Using the open-source declarative languages such as JSON and YAML, users can create and configure AWS resources. If users prefer to design visually, they can use AWS CloudFormation Designer to get started with AWS CloudFormation templates.
  • AWS CloudFormation Designer (Designer) allows users to see graphic representations of the resources in the template, simplifies template authoring, and simplifies template editing.

The AWS Cloud Development Kit (AWS CDK) is an open source software development framework to model and provision cloud application resources using familiar programming languages. AWS CDK enables users to model application infrastructure using TypeScript, Python, Java, and .NET. Developers can leverage their existing Integrated Development Environment (IDE), leveraging tools like autocomplete and inline documentation to accelerate development of infrastructure.

  • AWS CDK provides high-level components that preconfigure cloud resources with proven defaults, so that users can build cloud applications without needing to be an expert. 
  • AWS CDK utilizes AWS CloudFormation in the background to provision resources in a safe, repeatable manner. Constructs are the basic building blocks of CDK code. A construct represents a cloud component and encapsulates everything AWS CloudFormation needs to create the component.
  • The AWS CDK includes the AWS Construct Library containing constructs representing many AWS services. By combining constructs together, users can quickly and easily create complex architectures for deployment in AWS.

The AWS Serverless Application Model (SAM) is an open-source framework for building serverless applications. It provides shorthand syntax to express functions, APIs, databases, and event source mappings. With just a few lines per resource, users can define the application using YAML. During deployment, SAM transforms and expands the SAM syntax into AWS CloudFormation syntax, enabling users to build serverless applications faster.

  • During deployment, SAM transforms and expands the SAM syntax into AWS CloudFormation syntax. Then, CloudFormation provisions your resources with reliable deployment capabilities.
  • To get started with building SAM-based applications, use the AWS SAM CLI. SAM CLI provides a Lambda-like execution environment that lets users locally build, test, and debug applications defined by SAM templates. Users can also use the SAM CLI to deploy your applications to AWS.
  • A SAM template file is a YAML configuration that represents the architecture of a serverless application. AWS SAM templates are an extension of AWS CloudFormation templates, so any resource that can declared in an AWS CloudFormation template can also be declared in an AWS SAM template. Learn more

CloudFormation automates provisioning and updating infrastructure in a safe and controlled manner. There are no manual steps or controls that can lead to errors. Users can use Rollback Triggers to specify the CloudWatch alarms that CloudFormation should monitor during the stack creation and update process. If any of the alarms are triggered, CloudFormation rolls back the entire stack operation to a previously deployed state.

  • Using ChangeSets, users can preview the proposed changes that CloudFormation intends to make to infrastructure and application resources prior to execution, so that the deployments go exactly as planned.
  • CloudFormation determines the right operations to perform, provisions resources in the most efficient way possible, and rolls back automatically if errors are encountered. Which returns the state of the infrastructure and application resources to the last known good state.
  • Using Drift Detection, users can keep track of changes to resources outside CloudFormation, making sure users always have the most up-to-date picture of their infrastructure.

CloudFormation StackSets enables users to provision a common set of AWS resources across multiple accounts and regions, with a single CloudFormation template. StackSets takes care of automatically and safely provisioning, updating, or deleting stacks, no matter where they are.

  • AWS CloudFormation Change Sets allow users to preview how proposed changes to a stack might affect the running resources. CloudFormation makes the changes to the stack only after users decide to execute the Change Set. 
  • AWS CloudFormation  reuse the template to set up the resources consistently and repeatedly. Just describe the resources once and then provision the same resources over and over in multiple regions.
  • AWS CloudFormation automatically manages dependencies between resources during stack management actions. Users don’t need to worry about specifying the order in which resources are created, updated, or deleted; CloudFormation determines the correct sequence of actions to take for each resource when performing stack operations.

 

CloudFormation helper scripts

AWS CloudFormation API

AWS CloudFormation provides a simple set of APIs that are easy to use and highly flexible. Some of the most commonly used APIs and their functionality are listed below:

CreateStack: Starts the creation of a new stack. The input parameters to the call include the stack name and a file name (or Amazon S3 URL) for the source template. 

ListStacks: Lists all stacks in customers account. Users are able to use ListStacks to view the set of stacks and their current status, such as whether the stack is being created or updated.

ListStackResources: Lists all the AWS resource names and identifiers that were created as part of creating a stack. In addition to providing users information, this call can be used by an AWS CloudFormation-aware application to understand its environment.

DescribeStackEvents: Lists all AWS CloudFormation generated operations and events for a stack so that users can see how creation or deletion is progressing.

UpdateStack: Starts the update process for an existing stack. The input parameters to the call include the stack name and a file name (or Amazon S3 URL) for the updated template. 

AWS CloudFormation is integrated with the Amazon Simple Notification Service (Amazon SNS), that enables users to receive notifications as the creation, update and deletion of the stack progresses. 

Bootstrapping Applications and Handling

AWS CloudFormation provides a number of helper scripts that can be deployed to your EC2 instances. These scripts provide a simple way to read resource metadata from users stack and use it to configure their application, deploy packages and files to the instance that are listed in the template, and react to stack updates such as changes to the configuration or updates to the application. Here are some of the scripts that are available:

  • cfn-get-metadata: Retrieve metadata attached to users resources in the template.
  • cfn-init: Download and install packages and files described in users template.
  • cfn-signal: Signal to the stack creation workflow that your application is up and running and ready to take traffic.
  • cfn-hup: A daemon to listen for stack updates that were initiated through the AWS console, command line tools or API directly and execute application-specific hooks to react to those changes.

Users can use CloudFormation scripts on their own or in conjunction with CloudInit, a feature available on the Amazon Linux AMI and some other Linux AMIs. For more details of bootstrapping applications and updating configuration, see the AWS CloudFormation Developer Resources 

AWS  CloudFormation Concepts

Template

 
 

A CloudFormation template is simply a JSON (JavaScript Object Notation)- formatted text file that describes the AWS infrastructure needed to run an application or service along with any interconnection between them. These template packages deploy a base stack of resources, such as a three-tiered Virtual Private Network (VPC) structure, Identity Access Management (IAM) configuration, and S3 buckets and policies. EC2 and RDS instances can then be deployed into this base architecture while making use of standardized security groups, IAM policies, and other resources. The templates are designed to deploy architecture that is in alignment with AWS leading practices, as well as the identified security framework.

  • Users can save JSON or YAML files with any extension, such as .json.yaml.template, or .txt. AWS CloudFormation uses these templates as blueprints for building AWS resources. For example, in a template, users can describe an Amazon EC2 instance, such as the instance type, the AMI ID, block device mappings, and its Amazon EC2 key pair name.
  • To provision and configure the stack resources, users need to understand AWS CloudFormation templates. Users can use AWS CloudFormation Designer or any text editor to create and save templates. These templates describe the resources that is provisioned in AWS CloudFormation stacks. 

Users can author AWS CloudFormation templates in JSON or YAML formats. AWS support all AWS CloudFormation features and functions for both formats, including in AWS CloudFormation Designer. AWS CloudFormation Designer (Designer) is a graphic tool for creating, viewing, and modifying AWS CloudFormation templates. With Designer, users can diagram template resources using a drag-and-drop interface, and then edit their details using the integrated JSON and YAML editor. Designer has four panes.

  • The canvas pane shows a diagram of the template resources so that users can see them and their relationships at a glance.
  • To add resources to the template, users drag them from the Resources types pane onto the canvas pane.
  • Use the Integrated JSON and YAML editor pane to specify template details, such as resource properties or template parameters. After modifying the template, users can save it to a local file or to an Amazon S3 bucket.
  • When converting a valid template from JSON to YAML or vice-versa, the Messages pane displays a success or failure message. When opening or validating an invalid template, the Messages pane displays validation errors.

A stack is a collection of AWS resources that you can manage as a single unit. In other words, you can create, update, or delete a collection of resources by creating, updating, or deleting stacks. All the resources in a stack are defined by the stack’s AWS CloudFormation template. A stack, for instance, can include all the resources required to run a web application, such as a web server, a database, and networking rules. If you no longer require that web application, you can simply delete the stack, and all of its related resources are deleted.

AWS CloudFormation ensures all stack resources are created or deleted as appropriate. Because AWS CloudFormation treats the stack resources as a single unit, they must all be created or deleted successfully for the stack to be created or deleted. If a resource cannot be created, AWS CloudFormation rolls the stack back and automatically deletes any resources that were created. If a resource cannot be deleted, any remaining resources are retained until the stack can be successfully deleted.

Stack Deployment

The template package uses nested CloudFormation templates. Each stack can be deployed independently, if needed, by specifying the required parameters for each upon deployment. To deploy the full package, the IAM user must have permissions to deploy each of the resources it creates, including IAM configuration for groups and roles.

Stacks included in the solution

There are four (4) CloudFormation template files included, as well as a master template, which can be used to deploy the full stack version deploying each individual stack in this solution:

  • Stack 1 – Access Enables CloudTrail, S3 buckets and IAM settings for S3 bucket access. Creates IAM Roles and Groups.
  • Stack 2 – Network Three tier VPCs (Management, Development and Production), Subnets, Gateways, Route Tables, NACLs
  • Stack 3 – Security Elastic Load Balancers, S3 Buckets Policies, Security Groups, SNS, SQS, CloudWatch
  • Stack 4 – Server/Data Instances – Proxy, WebApp and DB, or RDS DB, ELB, SNS Alarms for Cloudwatch, Auto Scaling Groups. This Stack is depend on Stack 2 “Output” values
  • Main Stack – Primary template file which is used to deploy the full template package (multi-tier linux-based web application); passes parameters to nested templates. The Main Stack is dependent on the values of Stack 2 and Stack 3 “Output” 
Nested Stacks

Nested stacks are stacks created as part of other stacks. Users can create a nested stack within another stack by using the AWS::CloudFormation::Stack resource. This template relies on the use of nested CloudFormation stacks. Stacks 1-4 JSON templates deploy the actual resources, with the main*.json templates specifying each of these template files as stack resources to launch. The main* templates are basically the entry points to launch the entire architecture. Main.json is used to launch the additional stacks and also allow parameters to be passed into each of the nested stacks. To deploy the full package, the IAM user must have permissions to deploy each of the resources it creates, which includes IAM configuration for groups and roles.

Stack

 
 

 

StackSet

 
 

CloudFormation StackSets allow users to roll out CloudFormation stacks over multiple AWS accounts and in multiple Regions with just a couple of clicks. When launching StackSets, grouping accounts was primarily for billing purposes. Since the launch of AWS Organizations, users can centrally manage multiple AWS accounts across diverse business needs including billing, access control, compliance, security and resource sharing.

AWS CloudFormation StackSets extends the functionality of stacks by enabling users to create, update, or delete stacks across multiple accounts and regions with a single operation known as A target account. Using an administrator account, users define and manage an AWS CloudFormation template, and use the template as the basis for provisioning stacks into selected target accounts across specified regions. An administrator account is the AWS account in which you create stack sets. A stack set is managed by signing in to the AWS administrator account in which it was created. 

  • stack set lets users to create stacks in AWS accounts across regions by using a single AWS CloudFormation template. All the resources included in each stack are defined by the stack set’s AWS CloudFormation template. When creating the stack set, users specify the template to use, as well as any parameters and capabilities that template requires.

Stack sets can be created using either self-managed permissions or service-managed permissions.

  • With self-managed permissions, users create the IAM roles required by StackSets to deploy across accounts and Regions. These roles are necessary to establish a trusted relationship between the account administering the stack set from and the account deploying stack instances to. Using this permissions model, StackSets can deploy to any AWS account in which user have permissions to create an IAM role.
  • With service-managed permissions, users can deploy stack instances to accounts managed by AWS Organizations. Using this permissions model, users don’t have to create the necessary IAM roles; StackSets creates the IAM roles on their behalf. With this model, users can also enable automatic deployments to accounts that are added to the organization in the future.
Stack instances

stack instance is a reference to a stack in a target account within a Region. A stack instance can exist without a stack; for example, if the stack could not be created for some reason, the stack instance shows the reason for stack creation failure. A stack instance is associated with only one stack set.

Stack set operations

Users can perform the four different operations on stack sets such as Create stack set, Update stack set, Delete stacks, and Delete stack set.

The following figure shows the logical relationships between stack sets, stack operations, and stacks. When updating a stack set, all associated stack instances are updated throughout all accounts and Regions

 
AWS CloudFormation Stack instances
AWS CloudFormation Stack instances

AWS CloudFormation Best Practices

 

Best practices are recommendations that can help users use AWS CloudFormation more effectively and securely throughout its entire workflow. Learn how to plan and organize the stacks, create templates that describe the resources and the software applications that run on them, and manage the stacks and their resources. The following best practices are based on real-world experience from current AWS CloudFormation customers.

Planning and organizing
  • Use the lifecycle and ownership of AWS resources to help decide what resources should go in each stack.
  • When organizing AWS resources based on lifecycle and ownership, users might want to build a stack that uses resources that are in another stack.
  • Before launching a stack, make sure to create all the resources without hitting the AWS account limits.
  • Once the stacks and resources are set up, users can reuse the templates to replicate the infrastructure in multiple environments.
  • As the infrastructure grows, common patterns can emerge in which users declare the same components in each of the templates.
  • If the template requires inputs for existing AWS-specific values, such as existing Amazon Virtual Private Cloud IDs or an Amazon EC2 key pair name, use AWS-specific parameter types. 
  • With constraints, users can describe allowed input values so that AWS CloudFormation catches any invalid values before creating a stack. Users can set constraints such as a minimum length, maximum length, and allowed patterns.
Creating templates
  • When launching stacks, users can install and configure software applications on Amazon EC2 instances by using the cfn-init helper script and the AWS::CloudFormation::Init resource.
  • The helper scripts are updated periodically. Be sure to include the following command in the UserData property of the template before calling the helper scripts to ensure that launched instances get the latest helper scripts.
  • Before using a template create or update a stack, users can use AWS CloudFormation to validate it. Validating a template can help users catch syntax and some semantic errors, such as circular dependencies, before AWS CloudFormation creates any resources. 
  • Users can validate the template for compliance to organization policy guidelines. AWS CloudFormation Guard (cfn-guard) is an open-source command-line-interface (CLI) tool that provides a policy-as-code language to define rules that can check for both required and prohibited resource configurations.
Managing stacks
  • After launching a stack, use the AWS CloudFormation consoleAPI, or AWS CLI to update resources in the stack. Do not make changes to stack resources outside of AWS CloudFormation. 
  • Change sets allow users to see how proposed changes to a stack might impact the running resources before implementing them. 
  • Stack policies help protect critical stack resources from unintentional updates that could cause resources to be interrupted or even replaced. A stack policy is a JSON document that describes what update actions can be performed on designated resources. 
  • Users stack templates describe the configuration of the AWS resources, such as their property values. To review changes and to keep an accurate history of the resources, use code reviews and revision controls. 
  • On all Amazon EC2 Linux instances and Amazon EC2 Linux instances created with AWS CloudFormation, regularly run the yum update command to update the RPM package. This ensures that the latest fixes and security updates.

AWS CloudFormation Security  

 
AWS CloudFormation

AWS CloudFormation provides a number of security features to consider as users develop and implement their own security policies. The following best practices are general guidelines and don’t represent a complete security solution. Because these best practices might not be appropriate or sufficient for the environment, treat them as helpful considerations rather than prescriptions.

Use IAM to control access

IAM is an AWS service manage users and their permissions in AWS. Users can use IAM with AWS CloudFormation to specify what AWS CloudFormation actions users can perform, such as viewing stack templates, creating stacks, or deleting stacks. Furthermore, anyone managing AWS CloudFormation stacks will require permissions to resources within those stacks. For example, if users want to use AWS CloudFormation to launch, update, or terminate Amazon EC2 instances, they must have permission to call the relevant Amazon EC2 actions.

  • In most cases, users require full access to manage all of the resources in a template. AWS CloudFormation makes calls to create, modify, and delete those resources on their behalf. To separate permissions between a user and the AWS CloudFormation service, use a service role. AWS CloudFormation uses the service role’s policy to make calls instead of the user’s policy. 
Do not embed credentials in the templates

Rather than embedding sensitive information in the AWS CloudFormation templates, AWS recommend users to use dynamic references in the stack template. Dynamic references provide a compact, powerful way to reference external values that are stored and managed in other services, such as the AWS Systems Manager Parameter Store or AWS Secrets Manager. When using a dynamic reference, CloudFormation retrieves the value of the specified reference when necessary during stack and change set operations, and passes the value to the appropriate resource. However, CloudFormation never stores the actual reference value. 

Use AWS CloudTrail to log AWS CloudFormation calls

AWS CloudTrail tracks anyone making AWS CloudFormation API calls in users AWS account. API calls are logged whenever anyone uses the AWS CloudFormation API, the AWS CloudFormation console, a back-end console, or AWS CloudFormation AWS CLI commands.

  • Enable logging and specify an Amazon S3 bucket to store the logs. That way, users can audit who made what AWS CloudFormation call in your account. 

AWS CloudFormation gives developers and systems administrators an easy way to create, manage, provision, and update a collection of related AWS resources in an orderly and predictable way. AWS CloudFormation uses templates written in JSON or YAML format to describe the collection of AWS resources (known as a stack), their associated dependencies, and any required runtime parameters. Users can use a template repeatedly to create identical copies of the same stack consistently across AWS Regions. After deploying the resources, users can modify and update them in a controlled and predictable way. In effect, users are applying version control to the AWS infrastructure the same way as an application code.

  • The templates require a specific syntax and structure that depends on the types of resources being created and managed. Users author the resources in JSON or YAML with any code editor such as AWS Cloud9, check it into a version control system, and then CloudFormation builds the specified services in safe, repeatable manner.
  • A CloudFormation template is deployed into the AWS environment as a stack. Users can manage stacks through the AWS Management Console, AWS Command Line Interface, or AWS CloudFormation APIs.
  • AWS CloudFormation makes it easy to organize and deploy a collection of AWS resources and enables users describe any dependencies or pass in special parameters when the stack is configured.
  • With CloudFormation templates, you can work with a broad set of AWS services, such as Amazon S3, Auto Scaling, Amazon CloudFront, Amazon DynamoDB, Amazon EC2, Amazon ElastiCache, AWS Elastic Beanstalk, Elastic Load Balancing, IAM, AWS OpsWorks, and Amazon VPC