GTAC 2010: Measuring and Monitoring Experience in Interactive Streaming Applications

Google Test Automation Conference 2010 October 28-29, 2010 Measuring and Monitoring Experience in Interactive Streaming Applications Presented by Shreeshankar Chatterjee, Adobe Systems India Abstract: Interactive multimedia based applications are becoming increasingly ubiquitous over the internet; Applications like RIA web conferencing, multi-player gaming, video telephony, and unified communications based offerings, etc. There can be hundreds of variables that affect the Quality of Experience (QoE) for such applications. Tolerance for good end‐user experience is low and feedback is mostly subjective and opinionated enough to impact business. Under such random conditions, can we measure experience objectively? Can the process of identifying bottlenecks be made more deterministic? Can we give more objective debugging input to developers and architects? Can we benchmark the system for acceptable experience? The speaker proposes an organized test methodology for the problem. It starts with building testability, developing Agents, design of QoE metrics, and then modeling the metrics to isolate areas of high risk. The talk then discusses some examples of using such techniques for debugging user‐experience bottlenecks (risk correction), for benchmarking and in continuous integration test systems (risk detection) and building 'expert system' monitoring tools (risk prevention). Speaker Bio: Shreeshankar Chatterjee has over 8 years of test engineering experience, including 5 years at Adobe Systems. Shreeshankar is currently leading the performance, user-experience and automation engineering labs at Adobe Systems, India for collaboration products based on the Adobe® Flash® eco-system. He has a publication in IEEE Computer Society on the topic of debugging user-experience bottlenecks for web applications involving live multimedia streams. Prior to Adobe, Shreeshankar has worked in Oracle, Server Technologies Group. Shreeshankar has done his BS in Information Technology from University of Delhi. He is currently pursuing his MS in Software Systems from Birla Institute of Technology and Science (BITS), Pilani. Shreeshankar is also a Certified Information Systems Auditor (CISA). Talk Slides: https://docs.google.com/leaf?id=0B4fT-BFGDnQkY2FkNzdjYWUtZDFlNS00YjIxLWE4YzMtZDQzMDFhY2QwOTQ0