Closed Loop Perception for Resource Efficient Autonomous Systems

Autonomous Systems such as Autonomous Vehicles (AV), robots and drones are being developed for large scale deployments in real world applications such as transportation, agriculture, defense, urban planning etc. To operate safely in such diverse and dynamic scenarios, the perception engine within these systems must be capable of adapting to the dynamic real-time constraints such as latency and energy consumption. This adaptability is not present in the modern perception systems as they are open-loop by design and therefore neither aware nor capable of reacting to the dynamics of a real-world scenario. My research presents the Closed Loop Perception that interprets the perception process in modern autonomous systems as a control system. It creates the notion of 'perception risk' which represents the state of the process by estimating perception failures and then proposes a risk-resource controller that generates feedback signals to dynamically control the resource consumption within the system by using biologically inspired focus-of-attention mechanisms. The proposed Closed Loop Perception System can introspect and adapt to the real-time requirements of an Autonomous System operating in the wild.