Cloud-Based Automated Software Reliability Services

Google Tech Talk July 22, 2010 ABSTRACT Presented by Professor George Candea http://people.epfl.ch/george.candea This talk proposes cloud-based automated software reliability services (SRS), a step toward making testing and debugging of code as easy as using webmail. SRS is automatic, without human involvement from the service user's or provider's side; this is unlike today's "testing as a service" businesses, which employ humans to write tests. First, I will outline four of the SRS components we envision: a "home edition" on-demand testing service for consumers to verify the software they are about to install on their PC or mobile device; a "programmer's sidekick" enabling developers to thoroughly and promptly test their code with minimal upfront resource investment; a public "certification service," akin to Underwriters Labs, that independently assesses the reliability, safety, and security of software; and an "automated debugging" service that helps developers fix code based on bug reports from the field. Then I will present in detail execution synthesis, the technique that makes automated debugging (the latter SRS component) a reality. Given a program and a bug report, execution synthesis combines static analysis and symbolic execution to "synthesize" a thread schedule and various required program inputs that cause the reported bug to manifest. The synthesized execution can then be played back deterministically in a regular debugger, like gdb. We have found this determinism to be particularly useful in debugging concurrency bugs. Our technique requires no runtime tracing or program modifications, thus incurring no runtime overhead and being practical for use in production systems. We evaluate it on popular software (e.g., the SQLite database, ghttpd Web server, HawkNL network library, UNIX utilities) and find that, starting from mere bug reports, it can reproduce on its own several real concurrency and memory safety bugs in less than three minutes.