Using a CircleCI build server as an ad-hoc QA environment

Picture of Jean-Philippe Boily
Jean-Philippe Boily, Thursday September 11, 2014

If you like many others don't have an optimal environment setup, you are likely lacking an isolated QA environment for your integration tests. Ever wondered why can't you use your CI build server as an ad-hoc QA environment? It's already setup to be able to run your code after all...

We were asked this question recently; one of our clients hasn't had the time to make an optimal QA environment, but they are using the awesome CircleCI as we do. They wanted to use Rainforest more, to test each build. We reached out to CircleCI and asked them if they've done this before; It turns out it's a new use case, so I dived right in and started to hack on a proof of concept to help our client.

The requirements are pretty short - we need to run a webserver inside CircleCI, make that webserver accessible to Rainforest's testers and trigger a Rainforest test run against this webserver.

We need to be able to route our testers to the Cirlce CI machine somehow, right? I used ngrok to develop Fourchette and it was a pleasure to use, so why not use it again? This is the core of this feature as ngrok makes it super easy to expose a local machine to any machine connected to internet, even if it's not routable.

First, you will need to sign up for a ngrok account. The free version will be enough for custom subdomains. If you want to use your own domain, you can, but it is paid. As an alternative to ngrok, you could also use

To make ngrok work inside CircleCI, you will need your circle.yml file configured so that the build server will download and run the ngrok binary in the background. In this example, I am using the SHA1 of the build as a subdomain, as it should be good enough in terms of uniqueness for most needs. It gives you a URL that will look like this: (as an added bonus, https also works!).

Here is how it looks in the circle.yml:

yml machine: post: - wget - unzip - ./ngrok -authtoken $NGROK_TOKEN -log ngrok.log -subdomain=$CIRCLE_SHA1 9292: background: true

If you copy it as-is you will need to configure NGROK_TOKEN in CircleCI, they have the process well documented. The "9292" is the port we are exposing to the world, so you should configure this to be whatever your webserver starts on.

Now, we have to start our web app. I made a hello world for this using Sinatra, so in my case, it was just adding cd circle-ci-says-hello-world && bundle install && bundle exec rackup -D -p 9292. We're done - we can see this "Hello World, from CircleCI!" from anywhere, all that, served from inside a CircleCI build server!

Running your Rainforest tests against this is covered in the follow up post Running against an Ad-hoc Environment with Rainforest.

Want to build a better QA environment? Find out how top companies are scaling QA for faster development cycles. Happy testing!