Statistical Virtualization: Scale as a Tool for Implementing Service Overlays

Google Tech Talks January, 10 2008 In this talk, we will explore the use of on-line non-parametric time series analysis and prediction to build virtualized services in distributed computing environments. In particular, by analyzing and predicting the future behavior of one set of distributed services we explore how they can be amalgamated dynamically to implement a "virtual" service overlay with properties that are not supported by any of the constituent service components (i.e. derive strictly from aggregation). We illustrate the approach by detailing distributed batch-scheduling mechanisms that provide both reservation and co-allocation services in environments that explicitly do not support them. While our work focuses on national-scale scientific computing infrastructure, we believe its alternative approach to virtualizing distributed systems abstractions is important in a larger scalable systems context for two reasons. First, because the methodology is inherently statistical, it improves with scale making scale a tool (rather than an impediment) in terms of implementation. Secondly, it shares many common features, both conceptually and implementationally with scalable search services making it possible, we believe, to explore the use of commercial search infrastructure in future work. Speaker: Rich Wolski My origins, like those of most people born in North America during this century, are ambiguous and questionable. I am currently an Associate Professor in the Computer Science Department at the University of California, Santa Barbara which is, of course, located in Goleta California for all intents and purposes that do not involve the U.S. Mail. My past is checkered (it used to be plaid, but I've been politely informed that a past can only be so retro). Formerly, I enjoyed the hospitable climes offered by the Computer Science Department at the University of Tennessee. I've also done time as research faculty member in the U.C. San Diego CS&E Department where I researched CS and a little E (every now and then) in a decidedly pedagogical manner. My research interests include, but are not limited to, Computational Grid computing for performance, parallel and distributed systems, and the endless metaphysical search for the perfect coffee cup.