[001.2] Introduction to ECS and Fargate

Running our dockerized app on ECS.

Subscribe now

Introduction to ECS and Fargate [01.09.2018]

Today's video is the next in the series of setting up our produciton app for production use in AWS ECS. In our last video, we Dockerized the app to prepare it to run on ECS / Fargate; in this video, we will work on doing just that.

Introduction to AWS ECS and Fargate

So, what are Amazon's Elastic Container Service and Fargate?

Here are some quick snippets from Amazon's website:

  • ECS: A highly scalable, high-performance container orchestration service that supports Docker containers and allows you to easily run and scale containerized applications on AWS
  • Fargate: a technology for Amazon ECS and EKS that allows you to run containers without having to manage servers or clusters

What does this mean?

The Elastic Container Service is a container orchestration service much like Kubernetes or Swarm. Where the magic happens is when ECS is connected to Fargate. Fargate abstracts the idea of managing the instances, so your only concern is setting up containers, auto scaling policies, and allowing Fargate to manage the underlying hardware.


Getting Started

Setting up our Repository

The first thing we're going to do is set up a Repository so that we can push our Docker image up to Amazon. In order to do that, we need to have the the AWS CLI tool configured correctly. We'll need to visit Amazon's users management page to create a user with the following security groups:

  • AmazonEC2ContainerRegistryFullAccess
  • AmazonECS_FullAccess

You should now have a user that looks similar to this:

iam user

Note: You don't necessarily need to add the FullAccess permissions, but we've done that to simplify this tutorial.

Next, we need to go to the Security Credentials tab and create an Access key. Clicking this button should pop up a modal and give us an Access Key ID and Secret Access Key. We'll need to add these to the AWS CLI.

Armed with these keys, let's switch over to the console and type aws configure. That command should ask you for both of your keys, default region, and default output format. We input all these keys and that should be enough. You should now have something that looks similar to this (you can re-run the aws configure command to verify).

➜  produciton.com git:(master)  aws configure
AWS Access Key ID [****************R6SA]:
AWS Secret Access Key [****************m7z5]:
Default region name [us-east-1]:
Default output format [None]:

Now, we can use the AWS CLI to create a repository:

➜  produciton.com git:(master) ✗ aws ecr create-repository --repository-name dailydrip/produciton
    "repository": {
        "registryId": "154477107666",
        "repositoryName": "dailydrip/produciton",
        "repositoryArn": "arn:aws:ecr:us-east-1:154477107666:repository/dailydrip/produciton",
        "createdAt": 1513463983.0,
        "repositoryUri": "154477107666.dkr.ecr.us-east-1.amazonaws.com/dailydrip/produciton"

Note: Take note of the repositoryUri since we will be using this to tell Docker where to push the image.

Our next step is to get an authentication token to allow Docker to authenticate to our repository.

➜  produciton.com git:(master)$(aws ecr get-login --no-include-email)
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Login Succeeded

Note: The aws ecr get-login command returns a Docker login command; you can run it without the $() eval wrapper to see the output.

Now that we have Docker authenticated, we can tag our image and push it to our repository.

➜  produciton.com git:(master)SECRET_KEY_BASE=$SECRET_KEY_BASE DATABASE_URL=postgres://user:password@produciton.c3ssrcrj9xkk.us-east-1.rds.amazonaws.com/produciton_production RAILS_ENV=production bundle exec rake assets:precompile
➜  produciton.com git:(master) ✗ docker build -t production .
...(concatenated output)
Successfully tagged production:latest
➜  produciton.com git:(master) ✗ docker tag production:latest 154477107666.dkr.ecr.us-east-1.amazonaws.com/dailydrip/produciton
➜  produciton.com git:(master) ✗ docker push 154477107666.dkr.ecr.us-east-1.amazonaws.com/dailydrip/produciton
...(concatenated output)
latest: digest: sha256:b66848ca073884975e1c7eab9ebf52fc39738de8675d85e6825a4ecb16fcb1ab size: 2841

If we go back to the AWS Console for Repositories, we should see our newly created dailydrip/produciton repository and, if we click on it, we should see our image that was pushed up.

Setting up RDS

The next step we are going to take is setting up our Postgres db using RDS. To do this, we need to navigate to the RDS section of the AWS Console. Once there, we can select Instances from the left navigation pane. On the right pane, we should see our instances listing page. On the top right of the page, click on the button Launch DB Instance. On this page, we're going to select Postgres and click Next.

Now, we should be on the Choose Use Case page. We're going to make sure the Production use case is selected and click Next.

We should now be on the Specify DB Details page where we can select the engine version, instance size, multi availability zones, and database name, user, and password. For this app, I'm going to choose the db.t2.medium instance, set the instance identifier to produciton, and set my username and password.

On the last page, Configure advanced settings, we're going to use the default VPC and choose to have the DB publicly accessible (more about this later). The last change we're going to make is setting the database name to produciton_production. After that, we can click Launch DB Instance.

Now, if we go back to the Instances page, we should see that our database is being provisioned.


Once our database is provisioned, we are going to run our migrations on our database from our local box. This is why we chose the publicly accessible option earlier. First thing we'll need to do is go to our RDS instance for produciton, scroll down, and click on the security group in the details section.

We should now be on the Security Groups page. Toward the bottom, there should be a set of tabs, the second one being Inbound. Let's select that tab and click Edit. Now, we're going to click Add Rule and for the Source dropdown, we will select My IP. This will create a rule to allow only our IP to connect to the database. Our inbound rules should look like this:

security group

Now, we should be able to connect to our database locally and run our migrations via a command similar to this:

SECRET_KEY_BASE=<key> DATABASE_URL=<url> RAILS_ENV=production bundle exec rake db:migrate


There are a few ways to handle running migrations for our rails app. This is the easiest method, but not the best solution. For a more involved solution, we could write a script that would be our entrypoint in the dockerfile that could run migrations before it started the server. However, if the ECS container doesn't start within a certain amount of time, it could kill the container and you might end up in a cycle of failed migrations or, even worse, a busted database.

Another option that seems to be more appropriate in this case would be to create a separate task definition that is used for solely running migrations. Then, once you pushed up your Docker image changes, you could run the migrations task before restarting your application. This could be done from the AWS console or from the AWS command line via aws ecs run-task command, which would spin up the newest image, run the migrations, and then exit (one-off task). Then, you'd just restart your application.

Setting up ECS and Fargate

Let's start by going over some terminology: We will be introducing a few different resources over the next few minutes.

  • Clusters
    • Clusters are a logical way to group resources (services and tasks).
  • Services
    • Services are used to run a load balance in front of a group of tasks.
    • This is also where you will specify how many instances of a task should be running. The service scheduler is in charge of starting new instances in the case of an instance failing.
  • Tasks
    • Tasks are the running instances of a task definition.
  • Task Definitions
    • Task Definitions are where you specify the resources for a Docker container or group of containers.
    • It is also where you specify the Docker image, any volumes, environment variables, and more.

Creating a Task Definition

We're going to start by creating the base resource, a task definition.

From the navigation pane on the left, let's click on Task Definitions. From here we can create and manage the Task Definitions we’ll be using. Let's click on Create new Task Definition, which should take us to the first step of creating a Task Definition.

At this step, we need to choose the launch type for our Task Definition. Let's choose Fargate and click Next Step.

Now, we're on step 2, which is where we configure the name, roles, memory and cpu size, container definitions, and more.

Let's start by setting the Task Definition Name to Web, Task Role to ecsTaskExecutionRole, Task Memory to .5GB, and the Task CPU to .25 vCPU.

Next, we're going to click Add Container to setup our container. When we click Add Container we should see a slide out that allows us to configure our container.

In this modal, we're going to set the container name to produciton and our image to the URI that you used to push your image to your repository. In our case it's 154477107666.dkr.ecr.us-east-1.amazonaws.com/dailydrip/produciton:latest (include the latest tag).

Next, in the Advanced Container Configuration section, we're going to make sure Essential is selected. Now, we're going to add a few environment variables. We're going to add these:

  • RACK_ENV: production
  • RAILS_ENV: production
  • PORT: 80
  • DATABASE_URL: (it might be easier to open a second window to look at your RDS instance settings)
    • ours will look something like: postgres://user:password@endpoint:5432/produciton_production

You should now have something that looks similar to this:

container config

Now, we can Add at the bottom of the modal and Create at the bottom of the page. Clicking “Add” should take us to a page that gives the status of our task definition. We can click the View Task Definition to go back to the page and review all of our changes.

Creating a Cluster

Now, let's go to the Clusters console and click Create a Cluster. This should take us to a page to choose a few different cluster templates. Let's choose Networking Only and click Next Step.

On the next page, we can name the cluster and choose to create a VPC. Let's name it Produciton and leave the Create VPC unchecked. Click Create.

Now, we can click View Cluster, which should take us back to the overview of our cluster.

Creating a Service

The next step is to create a Service that will house our tasks. If we're not there, we can navigate to the Clusters section, and choose our cluster, which should take us to the overview of our cluster. From here, we can choose the Services tab and click Create.

On the first page, we will choose our configuration for the service; we want to select FARGATE for our Launch Type. Next, we want to select the Task Definition that we just created. In our case, we named ours Web, so if we look in the dropdown, we should see a Web:1, which we will choose.

NOTE: The 1 in the name is the revision. As you make edits to your task-definition, you will need to make sure you update the service to point to the new version, so Web:2 and so on.

While on this page, we're also going to set the Service Name and Number of Tasks to produciton and 1, respectively. Then, we can click Next Step.

Now, we should be on the VPC and Security Groups page. Here, we're going to set the Cluster VPC to the VPC you have available, which will probably be the default one that was created. For subnets, you can choose the first one in the list, since you might not have more than one. The rest of the settings we will leave to their default settings for the time being. We're not going to worry about setting up a Load Balancer for now. Now, you can click Next Step.

We should now be on Step 3. This is where we would configure auto scaling. We're not going to configure auto scaling for our app, but if you were to click on the Configure Service Auto Scaling option, you'd see a list of available configurations to set the minimum and maximum number of tasks, along with the scale in and scale out policies. For this video, we're just going to choose Do not adjust the service's desired count and click Next Step.

Now, we should be on the 4th and final step. This is just to review the service settings before we create the service. Look over the configuration settings and, if you are okay with them, click Create Service.

Now, we can click on the View Service button and we should see a page similar to this:


NOTE: If you don't see the task right away, you might need to click the refresh button.

Now, we've set up everything and our app should be running in a moment. Once the Last Status of the Task says RUNNING, we should be able to hit the site.

Viewing the site

If we click on the task id, we should be looking at the details of the running Web task. If we scroll down a bit, we see the Network section; there, we should see an ENI ID that we can click on. Once we click on that, we should see a page similar to this:

Elastic Network Interface

Once here, we can see the IPv4 Public IP column and an IP address for our ENI. We can use this IP address to hit our service. Now, let's take a look.

rails broken

Hmm… This is embarrassing…What happened?

Let's take a look at our logs for the running task. To do this, we’ll switch back over to our AWS Console and go back to the Produciton Cluster. Click on the Tasks tab and click on our Task. We should now be on the details page for our running task. Click on the Logs tab and let's see what's going on.

task logs

Aha! Now, we can see that it looks like our rails app can't connect to the database. First, we want to make sure that we used the correct database URL in our config. In our case, we have. So, let's make sure our service has access to our RDS instance.

Let's go back to our services page and click on our produciton service. We should be at the details page and see a security group. In our case it's sg-f7d0d882. Let's switch over to our RDS instance. Now, we're going to navigate to RDS > Instances and choose our postgres instance. Go to the Details section and find the Security Groups. We should see one group, probably starting with rds-launch-wizard. Let's click on that security group, which should take us to a page like this:

security groups

Let's move over to the Inbound tab at the bottom of the page and click Edit. Now, we want to click Add Rule and add a Custom Rule with a Custom Source which we can search for our security group from the service. Now, our rules should look similar to this:

security group rules

Now, we've updated the rules which should allow our service to communicate with our database. So, let's check our application again to see if it’s working.

rails app

Voila… Now we have our app successfully running on AWS using the ECS and Fargate stack.


We've actually done quite a few things in this video. We've set up an elastic container repository (ECR) to allow us to push our Docker images up. We set up a postgres database with RDS and we've setup an ECS cluster using Fargate with a service and task definition.

Even though we have our production application running on ECS now, there are still many things left to handle. We are not hosting our assets correctly, and we have not set up logging or monitoring. Over the next few videos, we will be working toward a more "production-ready” application.