GTAC 2014: I Don't Test Often ... But When I Do, I Test in Production

Gareth Bowles (Netflix) Every day, Netflix has more customers consuming more content on an increasing number of client devices. We're also constantly innovating to improve our customers' experience. Testing in such a rapidly changing environment is a huge challenge, and we've concluded that running tests in our production environment can often be the most efficient way to validate those changes. This talk will cover three test methods that we use in production: simulating all kinds of outages with the Simian Army, looking for regressions using canaries, and measuring test effectiveness with code coverage analysis from production.