Sync Hacks: How Angie’s List Reduced Their Web Deployment Time to Seconds

Angie’s List, one of the world’s largest websites, reduced web deployment time from hours to seconds thanks to a tool built on BitTorrent Sync. Read on learn more.

Angie’s List helps you pick the right roofer, plumber, dentist and more by bringing crowd-sourced reviews for local businesses to your fingertips. With 2 and a half million paid members, it’s one of the largest websites in the world.

But, the company had a problem – excruciatingly slow web deployments. That is, they had a problem, until Ken Erwin, one of their DevOps engineer developed a solution called Overmind. Overmind, built on the BitTorrent Sync API, reduced Angie’s List’s web deployment time from hours to seconds. Without down time. Read on to find out how Ken solved Angie List’s slow-deployment-problem, and get the tool to start setting up Sync on all your servers.

From Ken:

The Problem: Hundreds of servers. Hours of time to deploy.
At Angie’s List, we have hundreds of servers. Our web deployment process prior to switching to Sync took hours. We’d have to just leave it going overnight. Each environment would take about an hour to transfer data. When we went into production, it was even worse. We were deploying to one server at a time, no more than once a week.

030414-angies-list-problem-graphic

The Solution: Building Twitter’s “Murder” for Windows
I started looking for a solution that would reduce that time. It turned out, Twitter experienced a similar problem. Deploying to thousands of servers in a centralized system was a long and ugly process. So, the guys over at Twitter turned to the BitTorrent protocol, and were able to speed up a 40 minute deploy process into one that lasted 12 seconds. It wasn’t exactly right for Angie’s List because we had Windows servers. So, enter BitTorrent Sync.

Introducing Overmind: Built on BitTorrent Sync For Faster Deploys that Scale
I have spent quite a bit of time developing a web deployment method that utilizes BitTorrent Sync and Powershell; reducing deployment time from hours to seconds for hundreds of servers (or more), with no downtime. It’s changed the way we work. We can now deploy daily. That’s something that wouldn’t have been possible before switching to Sync.

Overmind is a powershell library with a simple GUI. I named it “Overmind” because of its ability to keep all of our servers constantly in sync. Our first major production release with Overmind ended up being the first deployment we’ve had –- at least since I began working at Angie’s List — that required zero downtime. We’re currently refining the code to make it even smaller. It’s already substantially simpler than our previous tools.

2014_02_13_13_36_55_BTSync_Tool
BTSync Install/Add Sync folder GUI courtesy of Ken Erwin.

Want to get started? Here’s a tool that will allow you to install BitTorrent Sync on all your servers automatically (using the Sync API & Powershell). It’s completely customizable so that it can work for any environment. It even allows you to specify what ports to use. You can download the tool here. It may require Powershell 3.0 (as that is the only thing that I code in).

For anyone using the API, the BitTorrent Sync Install/Add Sync folder should be self explanatory. If you don’t have a Sync API key, request one here. Then, decide what ports you would like BitTorrent Sync to run on (and make sure that the firewall is open on those ports). Type in the information that you want to set up, and that’s it!

GettheAPI

Keep a look out. I’ll share the deployment script in a followup post, after I finish polishing it up.

Ken

You can keep up with Ken (kenerwin88) in our developers forum.