On-Focal-Plane Embedded Processing

We are actively pursuing research to implement computational functionality on the focal plane to enable very high-bandwidth applications.

Conventional imagers often are limited with regard to the applications that can be supported because of limited ability to quickly read data off the focal plane array. Digital-pixel focal plane arrays described elsewhere on this site have the ability to perform simple data processing steps at the pixel level, and have demonstrated significant improvement over conventional infrared imagers.

The objective of this program is to add further processing to the focal plane to enable, first, the ability to address high-bandwidth tasks that are impractical with current technology and, second, sensors that are capable of analyzing data, making a decision, and/or triggering an action directly.

Imager data collectionFigure 1. Imager data collection and various levels of on-chip processing are depicted. The red arrow in the Typical Commercial column depicts that the high data rate in this case is a limitation that is eased in the next three columns, which incorporate increasingly more on-focal-plane processing to reduce the readout rate problem.


Imagers with embedded processing generate an output that is a useful function of the observed scene, rather than a representation of the scene itself, which must be further processed and analyzed. Embedded processing techniques can be beneficial in numerous situations, including
  • When one needs to observe fast motion or changes in the scene
  • When low-latency readout and decision making is required
  • When full-frame imaging would result in too large a volume of data
  • When platform size, weight, and power constraints are a driver for ultracompact electronics

Lincoln Laboratory is currently pursuing numerous applications using imagers with embedded processing, ranging from compressive sensing to optical communications. By taking advantage of massively parallel analog-to-digital conversion and digital signal processing on the focal plane array, smarter, more efficient and higher performance sensor systems may be implemented. The applicability of this potentially disruptive technology is just beginning to be explored. The Laboratory’s prototyping efforts will help allow for realization of the potential of this promising technology.

top of page