Running TeamCity on EC2

One of Headspring’s core mantras is to outsource everything that is not part of our core business. We are true believers on running the business in the cloud. We are even giving a talk about it. We use many cloud service providers on a daily basis. We use Google Apps for Email, Calendar, Voice and Intranet. We use Salesforce as our CRM. We use BitBucket to host our source code.

We use all this services to do things that are necessary in order to run the company but that we don’t want to manage or build ourselves. So, we decided to do the same thing with our build system. As we were using TeamCity as our CI server and it has integration with EC2 out of the box we decided to go with EC2.

First, TeamCity is composed of two pieces. The build server, that runs TeamCity’s dashboard and the build agents, that actually run the builds. As TeamCity Server is a Java application, a small Linux instance on EC2 is enough to run it. The server itself won’t leverage the Elastic features on EC2, but having the server running on the same region and availability zone as the agents is important in order to avoid Data Transfer charges.

Server

To install TeamCity server on Linux, follow this steps:

1) Get TeamCity from JetBrains.

2) Unpack the file Teamcity.tar.gz as documented in the docs.

3) Elevate your privileges to root:

> sudo su

This is needed because we want the server to listen to port 80 and ports below 1024 can only be listened by the root user.

4) Install Sun/Oracle JDK.

EC2 instance is a RedHat distribution, that uses OpenJDK by default. To the cloud integration on TeamCity to work properly, it needs Sun/Oracle’s JDK.

5) With Sun/Oracle’s JDK installed, set the JAVA_HOME environment variable to use it, instead of OpenJDK:

> JAVA_HOME=/usr/java/jdk

6) Change TeamCity server port to 80 (if you need to)

7) Start the Server. On TeamCity’s bin directory, run:

> ./teamcity-server.sh start

That’s it. The server should be up and running. If you hit instance’s public URL on your browser, after the server finishes starting up you should get a screen to accept TeamCity’s EULA.

After accepting the EULA, you will be prompted to create an Administrator Account. With the Administrator Account created, TeamCity server is installed and running properly.

Installing the Build Agent

With the Server up and running it’s time to create the agents that will run the builds.

To do so, first launch a instance ( I’m going to use a Windows one as we are doing .NET here) on EC2 and configure it to work as a TeamCity agent. In our case, we chose to use a Large Windows x64 instance.

As it’s necessary to remote into the instance to configure it to run as a TeamCity agent, wait approximately 15 minutes after the instance is launched to Remote into it, otherwise you won’t be able to get the password from AWS.

With the instance credentials, follow this steps:

1) Remote into the instance

2) Open the browser and go to TeamCity Server URL. After authenticating, go to the Agents tab and download MS Windows Installer.

3) Run the installer. After the agent is installed, a form to “Configure Build Agent Properties” will be prompted. Change the serverUrl property to point to the correct URI.

4) TeamCity agent-server communication uses port 9090 by default, unless you’ve changed it when configuring Build Agent properties on the previous step. So, in order to enable the communication between agent and server, create a inbound rule on Windows firewall to allow communication trough port 9090.

5) If everything is configured properly, you should have a Unauthorized agent showing up on TeamCity Server.

6) Authorize the agent and check that it’s capable of running properly your builds.

Making the Build Agent Elastic

Right now we have a EC2 based TeamCity server and a TeamCity Agent running on a EC2 instance but it’s not leveraging the Elastic features of TeamCity yet. To enable Elastic, Cloud Based, build agents one has to create a AMI image of the instance  running the Agent.

1) On AWS, select the running instances and right-click on the instance running the Build Agent. On the context menu, select “Create Image (EBS AMI)”

2) When the AMI becomes available on aws, go to Cloud Tab under TeamCity Agents page:

3) On the configuration page, create a new Cloud Profile. As of now, the only option of Cloud Type is Amazon EC2. Choose your location and availability zone to match the ones the server instance is running in order to avoid extra Data Transfer costs.

That’s it. TeamCity is configured to use Elastic Build Agents.

When a build is triggered, if there are no agents available to run it, the build will be placed on the Build Queue. A instance will be requested to EC2. It will take approximately 10 minutes for the instance to be running and authorized as a Build Agent on TeamCity.

My goal with this post was to document the process of migrating Headspring’s build system to leverage EC2 features. I hope this helps not only me.

One last thing, can you do me a favor and vote for this TeamCity feature request?