Learning from Oracle or Crowd: Budget fidelity trade-offs and query design strategies

ABSTRACT: Design of many machine learning systems rely on well-curated datasets. Developing such datasets is an expensive and time-consuming process. Digital crowdsourcing (CS) is a modern approach to infer this from small contributions of a large and potentially non-expert crowd. In this talk, the CS problem, as a human-in-the-loop computation problem, is modeled and analyzed in an information theoretic rate-distortion framework. The purpose is to identify the ultimate fidelity that one can achieve by any form of query from the crowd and any inference algorithm with a given budget. This in turn motivates the design of coded query schemes. Strategies are presented for efficient and reliable query design in presence of a crowd or an oracle. The query rate performance and speed of learning are analyzed and the role of pricing is investigated. Joint work with Victoria Kostina and Babak Hassibi.

BIO: Farshad Lahouti received the Ph.D. degree in electrical engineering from the University of Waterloo, Canada, in 2002. In 2005, he joined the faculty of the School of Electrical and Computer Engineering, University of Tehran, where he founded the Center for Wireless Multimedia Communications. Dr Lahouti received the distinguished scientist award from Iran National Academy of Sciences in 2014. He joined the electrical engineering department at Caltech as a visiting faculty in 2013, where he initiated the digital ventures design program. He is also the co-founder and the scientific lead of cntxts Inc inventing the next generation ultra energy efficient IoT technology and its digital AI contexts. His current research interests are coding and information theory, and statistical signal processing with applications to machine learning, wireless networks and biological and neuronal networks. Information on his recent works can be found here: http://www.its.caltech.edu/lahouti

Hosted by: Salman Avestimeh