Information-theoretic Interpretations of Compressive Sampling

Compressive sampling or compressed sensing(CS)is a new paradigm for data acquisitionand signal recovery. There are two-way relationships between CS and information theory:the formershould and can be analyzed from the perspective of the latter，while the latter’s content and extent areenriched and broadened by the former. Specifically，some basic concepts and theorems in informationtheory，such as source，channel，source coding，channel coding，rate distortion，Fano inequality，andthe data processing theorem，provide theoretical foundation for research on CS，in particular，thatconcerning performance limits(e. g.，sampling rates).CS provides a highly efficient strategy for col-lecting，storing，transmitting，and reconstructing sparse signals through its unique concepts and algo-rithms，such as the sparsity of real signals (enabling CS sampling at a rate lower than Nyquist rate)，the information sensing capacity of random sampling matrices(which preserve information);and in-formation reconstruction based on convex optimization(different from signal reconstruction by Sinckernels in the Shannon-Nyquist sampling theorem).Thus，CS is a mechanism for direct informationsampling and processing，extending the domain of classic information theory. This paper seeks to clar-ify and explain the relationships between CS and information theory，revealing some of the fundamen-tal issues in CS，in particular，those concerning CS sampling，and providing guidance for CS researchdirections.