Remote Profiling of Web Server Resources Using Mini-Flash Crowds

Google Tech Talks December, 14 2007 Unexpected surges in Web request traffic can exercise server-side resources such as access bandwidth, processing and storage in unanticipated ways. Server administrators today do not have requisite tools to understand the impact of such "flash-crowd" like situations on the various server resources. Currently, most Web servers either rely on over-provisioning and admission control, or use expensive solutions like CDNs, to ensure high availability in the face of flash crowds. A more fine-grained understanding of the performance of individual server resources under emulated but realistic and controlled flash crowd-like conditions can aid administrators to make more efficient provisioning and resource management decisions. In this talk, I will present the design, implementation and evaluation of mini-flash crowds (MFC) -- a light-weight wide-area profiling service that reveals resource bottlenecks in a Web server infrastructure, including access bandwidth, processing resources, and back-end data management. MFC uses a set of controlled probes where an increasing number of distributed clients make synchronized requests that exercise specific resources of a remote Web server. We have carried out controlled lab-based and wide-area experiments using the MFC approach, as well as experiments in collaboration with operators of large production servers. Our experiments have shown that MFC can faithfully track the impact of request loads on different server resources, and provide useful, and sometimes surprising, insights to server operators on the constraints of their infrastructure. The MFC approach is non-intrusive and can be used to actively probe Web servers in the wild. I will conclude the talk with a discussion of the key results from an MFC-based measurement study of the provisioning of several hundred commercial servers of popular Web content providers and a few hundred Web servers of startup companies. Speaker: Aditya Akella Assistant Professor, Department of Computer Sciences, University of Wisconsin-Madison His research spans various areas of Computer Networking including Internet Routing, Network Security, Wireless Networking, Internet Measurements and Network Management.