Perception, Action and the Information Knot that Ties Them

I will describe a notion of Information for the purpose of decision and control tasks, as opposed to data transmission and storage tasks implicit in Communication Theory. It is rooted in ideas of J. J. Gibson, and is specific to classes of tasks and nuisance factors affecting the data formation process. When such nuisances involve scaling and occlusion phenomena, as in most imaging modalities, the ``Information Gap'' between the maximal invariants and the minimal sufficient statistics can only be closed by exercising control on the sensing process. Thus, senging, control and information are inextricably tied. This has consequences in understanding the so-called ``signal-to-symbol barrier'' problem, as well as in the analysis and design of active sensing systems. I will show applications in vision-based control, navigation, 3-D reconstruction and rendering, as well as detection, localization, recognition and categorization of objects and scenes in live video.