Internet Systems Consortium's SIE & Google Protobufs

Google Tech Talk December 3, 2009 ABSTRACT Presented by Robert Edmonds, Eric Ziegast, and Paul Vixie. ISC SIE (Security Information Exchange) is a trusted, private framework for information sharing in the Internet Security field. Participants can operate real time sensors that upload and/or inject live data to SIE, and other participants can subscribe to this data either in real time, or by query access, or by limited and anonymized download. While SIE began in 2007 with a method for collecting and sharing raw packet captures for Passive DNS in near real time, correlation with other data types and data sources was required. SIE needed a way to efficiently pass structured data between participating nodes in the loosely-coupled broadcast ethernet message bus. We would like to present why SIE selected Google Protocol Buffers, how we utilize the technology within SIE, and how security researchers can use the libraries (libnmsg), APIs and tools for real-time analysis of disparate data sources.