NIPS 2011 Big Learning - Algorithms, Systems, & Tools Workshop: Vowpal Wabbit Tutorial

Big Learning Workshop: Algorithms, Systems, and Tools for Learning at Scale at NIPS 2011 Tutorial: Vowpal Wabbit by John Langford Abstract: We present a system and a set of techniques for learning linear predictors with convex losses on terascale datasets, with trillions of features,\footnote{The number of features here refers to the number of non-zero entries in the data matrix.} billions of training examples and millions of parameters in an hour using a cluster of 1000 machines. One of the core techniques used is a new communication infrastructure--often referred to as AllReduce--implemented for compatibility with MapReduce clusters. The communication infrastructure appears broadly reusable for many other tasks.