Before we start digging into creating your own Heroku clone for your application a quick introduction. I’m a self taught developer. Everything I have ever written has put functionality first. Any recommendations are greatly appreciated as I, like all of us, continue on the learning journey. I will work on future articles based on the feedback I receive from this one.
You are responsible for understanding the costs associated with working on AWS. Assume that everything you do will cost real money. And keep assuming that until you read AWS’s pricing tiers for each thing you do.
This application and all the information surrounding it were built for a production level Rails application. We know that most have moved on to Node or other frameworks but Rails is mature enough to be considered stable. You can adapt the information in this how-to for your application/framework.
We will be launching the application in an application load balanced ECS cluster (Fargate launch type), with a worker container for Delayed::Job. The application will be backed by an AWS Aurora RDS instance. Sendgrid will be used for emails. Each element will be discussed in random order so feel free to search for what you need.
The first part (this article) will be focused on getting your application launched containerized and deployed. It may feel like drinking out of a fire hydrant but having an app deployed is the first step before we start cloning heroku.
Here is what the final result looks like (2018/06/12):
You need your app to be containerized. There are a thousand tutorials on how to do this. To put it as simply as possible, get a docker image of your app. To begin to understand how your containers will work together run them locally. I recommend using `docker-compose` for this. Our application container will handle the usual Rails MVC stuff. The Worker container will deal with the background jobs, things like rendering reports and batch database changes.
Below is a quick sample of a docker-compose file that may help you get started. (You will need to ensure correct tab spacing)
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs && apt-get install -y ghostscript
RUN mkdir /app
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
# Prevent bundler warnings; ensure that the bundler version executed is >= that which created Gemfile.lock
RUN gem install bundler
RUN bundle install
# Copy the Rails application into place
COPY . .
CMD rails s -b 0.0.0.0 -p 3000
app: # Building app container
command: rails s -b 0.0.0.0 -p 3000
# these allow you to debug with byebug by attaching to the container
command: rake jobs:work
# these allow you to debug with byebug by attaching to the container
It can look overwhelming at first but don’t worry. You are building a handful of containers here. The application (where your app runs), the database in a container, the worker, and mailcatcher (a great utility to view emails your app is trying to send). To run you containers together:
docker-compose up --build -d
You can attach to containers to view logs or run commands inside them with:
docker ps # to view containers
docker-compose exec [service name] [command]
docker attach [container]
If you can stumble your way through the above with a basic understanding of what is happening. If you would like to learn more about the above docker-compose check out the attached article which discusses this docker-compose in more depth.
ECR: Elastic Container Registry
Login to your AWS console. Head over to
ECS and click on the
Repositories link in the left hand pane. Once through the setup there you will have AWS Command Line Tools installed. If not:
brew install awscli
You should have a repository. You don’t need to keep note of your push commands as they will always be available in the AWS console. You can use your commands from your terminal to build and push the image of your app to ECR. Your image will be built based on your Dockerfile from above. You can use these commands in a pipeline in a repository host. Below is a BitBucket example of building the image from your repo when the master branch is touched.
- bundle install
- apt-get update && apt-get install -y python-dev
- curl -O https://bootstrap.pypa.io/get-pip.py
- python get-pip.py
- pip install awscli --upgrade
- aws_ecr_login=$(aws ecr get-login --no-include-email --region us-east-1)
- docker build -t [your build name here].
- docker tag [your tag command here]
- docker [your push command here]
There will be two task definitions. One for the application and one for the worker. This is because we will have two services in the cluster (each service needs a task definition). One of those services, the application, will be auto scaling and load balanced. The other, the worker, does not need to scale with the application. Make your own decisions on how to build your application. Here we don’t want one worker for each application.
Set a task definition to meet the needs of your application. Here is a sample for the rails application:
Click the add container button for you container ‘Application’
Now create a second definition for your worker. It will probably look very similar. But will likely have less
Task CPU. The command will be different
bundle exec rake jobs:work for Delayed::Job. Using the command field overwrites the command that your Dockerfile specifies.
Next up is the elastic load balancer which will spread the traffic over our tasks once they are running. This part may feel like putting the cart before the horse but you need a way to send traffic to your cluster. Head to the EC2 section of your AWS console. On the left side you will find
LOAD BALANCING with a sub header of
Load Balancers. Start by creating a new load balancer, and then select
HTTP/HTTPS for Application Load Balancer. Select internet facing and add your availability zones. Click
Next: Configure Security Settings Configure your ssl if using https/443. Then next again to add your security groups. You need a security group that has a port range including the app. This may be port
80. Give the new target group a name and add it to your load balancer. You don’t need to register any targets as this will be handled for you at the cluster/service level. Review and create your load balancer.
Back to the ECS console as it is finally time to create our first cluster. Click to create cluster. Chose networking only (Fargate Launch type), give it a name and launch it!
Right now you have images in your repository (ECR), a load balancer for the application (EC2) and task definitions that describe what will happen in your cluster (ECS). Now it is time to bring it all together.
Create Service. Choose
Fargate , give your service a name and a minimum number of tasks you want it to be running at all times.
On the next page select the VPC and subnets that match your load balancer from earlier. Click to edit security groups. Add groups to match the load balancer and other service you need. You will probably want to include your AWS RDS security group.
Once you have your VPC selected you can select the
Application Load Balancer option, which will then prompt you to select your LB.
# Make sure that you app has a working healthcheck path
On the next page set your auto scaling information. This is your minimum, maximum and when to scale information.
Review and start your service. Your service will start the tasks and ensure they are running for you. You can use the application load balancer to access your application once it comes online. Logs are viewable in cloudwatch and the tasks panel of the cluster service if you have problems.