Surveillance system helps officials identify security threats
A suspicious object is left on the side of a busy city street and is not reported to police until several hours later. When authorities try to determine who left the object, they turn to footage from multiple commercial surveillance systems in the area. To find a person of interest, they must first sort through the hours of footage these systems have recorded and deal with gaps where one system’s coverage ends and another system’s begins. Finding a suspect could take them hours — if they are even able to find one at all.
With Lincoln Laboratory’s surveillance system, the Immersive Imaging System (IIS), identifying a suspect takes only minutes. A team of researchers from the Integrated Systems and Concepts Group at Lincoln Laboratory began work on IIS in 2007 at the request of the Department of Homeland Security (DHS) Science and Technology Directorate. The goal of IIS was to provide DHS and other government organizations with wide-area situational awareness and forensic capabilities.
“IIS allows for 360-degree surveillance of an area with full forensic recall for investigative purposes,” said Jonathan Blanchard, who worked on the IIS project for 10 years as a mechanical designer.
IIS can sufficiently identify a person’s face at distances of up to 100 meters in any given direction. In terms of total surveillance, the system’s range can monitor an area equivalent to that of seven football fields.
To achieve wide-area situational awareness, IIS uses four core components: a 240-megapixel camera, an advanced processing server, video analytics algorithms, and a graphical user interface. The IIS camera consists of 48 inexpensive commercial off-the-shelf imagers that overlap to achieve 360-degree surveillance that has no drops in coverage. The team also chose lenses with different focal lengths that together maximize the different fields of view. These imagers and lenses, when paired together, provide users with resolutions 30 times greater than what commercial surveillance camera systems are able to achieve.
The raw image data are pulled from the camera, fed into a processing center within the system, and stitched into a video for a user to view just three seconds after the images are captured. Users can view this video feed in real time through a versatile graphical user interface. With this interface, users can choose to view the entire 360-degree area or a single defined section. Within these views, users can also pan, tilt, and zoom in on a selected area. They can also speed up a feed, view it backwards, or jump it forward in time. This video feed can also be simultaneously viewed by multiple users in different locations.
To automatically detect threats, the IIS system applies algorithms to the video data that can track a specific moving target and analyze behavior, such as loitering or moving against the normal flow of crowd traffic. These algorithms alert officials before a threat has the chance to escalate into a potentially dangerous situation. IIS can also detect threats in real time by using organization-specific algorithm parameters. For instance, if an area of a building is closed for renovation, algorithms can be set to monitor the entrance to that area and alert users when the door is opened.
To test their system and this capability, Laboratory researchers first demonstrated IIS at Logan International Airport’s Terminal A in 2010. They used the system to monitor the concourse, which includes the security lines and people entering from the parking garage and passenger drop-off curb. In addition, people exiting from the secure side of the terminal or “sterile areas” were also observed for wrong-way pedestrian traffic, as it indicates suspicious activity, which can cause officials to shut down a terminal for hours. With IIS, algorithm parameters were set up to monitor these exits and successfully alerted officials when an individual walked in the wrong direction.
“Seeing our prototype in the real world and receiving positive feedback was very rewarding,” said Cindy Fang, the project manager for IIS. The positive feedback that the team received from the Boston Police Department and the Massachusetts Port Authority after those 2010 tests led to a mobile configuration of IIS being deployed in highly trafficked Boston events, such as the 2013 Fourth of July celebration and 2014 Boston Marathon. The system has now been transferred to industry and is being tested for installation in a sports venue in Seattle. The system was also awarded a 2018 R&D 100 Award.
Researchers from the Laboratory are currently working on DragonFly, the next iteration of IIS. The most significant improvement made to DragonFly is its smaller size. While IIS weighs about 40 pounds, DragonFly will weigh only four pounds. In addition to its reduced weight, researchers will weatherize the system and increase the camera’s pixel count. The team’s ultimate goal for the DragonFly project is to produce a surveillance system that can work on the move.
This research was funded by the Department of Homeland Security Science and Technology Directorate (HSHQPM-14-X-00181).