Coopetitive Multimedia Surveillance
Basic idea
In our previous work [1] we have introduced ‘coopetitive’ interaction approach with focus on camera-interaction only. On the other hand, effective surveillance often requires multi-modal information which is obtained from more than one type of sensors. Hence, in this paper we extend our work by:
The practical implementation has been done using 2 active cameras and a 5 by 3 motion-sensor grid. The top-view of the proposed setup is shown here.
Experiments
Two sets of experiment were conducted to verify the suitability of our proposed approach.
Experiment 1
Setup- An enclosed premise similar to that found in museum sub-section, ATM lobby etc.
Aim- Capture at least 3 high resolution images of each intruder and then continue to focus on his face if there are no other intruders.
|
Approach |
Comments |
Video |
|
Only competition |
The sensors are competing to undertake the task on their own and they do not pass any intruder localization info to each other. Hence, no localization info is passed by the motion-sensor grid to the cameras or between the cameras, when the intruder changes his facial direction. Hence if the face is ‘lost’ or not yet found, the camera must pan on its own to obtain intruder location. |
Link (Size:1.2 MB) |
|
Only cooperation |
The sensors have full faith in each others abilities and are willing to help each other (even if they are not worthy of such help). There is no competition to decide which camera is better suited for focusing on face and hence both share this role on a time sharing basis (50 frames each). The camera allocated the face focusing task has been annotated in the video. The other camera continuously pans to find ROI location while waiting for its turn to focus on face. |
Link (Size:1.5 MB) |
|
Coopetition without MPC |
With ‘coopetitive’ approach, the localization info is passed to appropriate sensors and the roles are also allocated correctly. The performance improves significantly compared to earlier two approaches. |
Link (Size:1 MB) |
|
Coopetition with MPC |
We hope to do even better than ‘coopetition without MPC’. We use a predictive method to determine intruder’s trajectory which can help to better focus on his face and capture his 3 high resolution images faster. |
Link (Size:1.4 MB) |
Experiment 2
Setup- A walkway leading to an important room e.g. Nuclear plant control room etc.
Aim- Capture at least 3 high resolution images of each intruder passing through the walkway.
We want to see whether our proposed predictive approach can help in better ordering the intruder-focusing task when multiple intruders are present. Two different scenarios are possible wherein the 2 intruders may be looking towards same camera or different cameras. Sample video for each scenario are shown here.
Scenario1- Video Link (Size: 0.6MB)
Scenario1 shows how the system handles two intruders simultaneously walking towards the important room (direction from camera2 towards camera1). The camera2 notices the intruder1 first but then realizes that intruder2 is going to exit much faster and it has high probability of capturing intruder1 even after completing intruder2 focusing task. Thus it focuses on intruder2. After obtaining three images of intruder2 it goes back to focus on intruder1 and captures its three high resolution images.
Scenario2- Video Link (Size: 0.4MB)
Scenario2 shows a similar case where intruder is detected first but intruder2 if focused as it is exiting faster and its 3 facial images are obtained. However, this time around intruder1 changes his direction and camera2 is required to focus on it and obtain three high resolution images.
Reference
1. Vivek K. Singh and Pradeep K. Atrey Coopetitive Visual Surveillance using Model Predictive Control, 3rd ACM International Workshop on Video Surveillance and Sensor Networks in conjunction with ACM Multimedia 2005 (ACM VSSN'05), pp 149-158, November 11, 2005, Singapore.
