GTAC 2011: Automating Hadoop Stack Deployment and Testing

6th Annual Google Test Automation Conference 2011 (GTAC 2011) "Cloudy With A Chance Of Tests" Computer History Museum Mountain View, CA USA October 26-27, 2011 Presented by Andre Arcilla. ABSTRACT Hadoop is the distributed technology that powers Yahoo! grids. Due to the complex nature of the Hadoop ecosystem, Hadoop stack deployment requires significant investments of time and engineering expertise. Hadoop engineering team came up with a solution for automatic deployment and validation of the software stacks on a small computational footprint, enabling wider use of the technology for running small-scale trials, and using and testing Hadoop-based products in realistic environments. This talk presents HIT - Hadoop deployment and integration test environment that enables rapid automated Hadoop deployment. Andre Arcilla is the Hadoop Integration Architect at Yahoo!, focused on assembly and deployment of Hadoop ecosystem. He previous experience includes architecting and building scalable utility and distributed enterprise solutions.