Bias-aware Gaze Uniformity Detection and Correction in Group Images
Associate Professor and Director of Undergraduate Studies
Co-Director, Albany Lab for Privacy and Security (ALPS)
Department of Computer Science
College of Engineering and Applied Sciences
University at Albany, State University of New York (SUNY)
UAB 421, 1215 Western Avenue, Albany, NY 12222, USA, The Earth, The Universe
Phone: 518-437-4943, Fax: 518-442-5638
Email: first letter of first name plus last name at albany dot edu
Affiliated Faculty,
College of Emergency Preparedness, Homeland Security and Cybersecurity,
University at Albany
Faculty Fellow,
Center for Technology in Government (CTG UAlbany)
|
Motivation
Do you capture photos in group settings?
Are there multiple individuals taking pictures of the group?
Do all members of the group face the same camera or gaze in the same direction?
Research Objective
Developing a method to ensure consistent gazes among subjects in group photos is crucial for determining overall aesthetic appeal.
Contribution
Our proposed method, GARGI (Gaze-Aware Representative Group Image), uses AI algorithms to select group images with optimal gaze uniformity. Unlike existing mechanisms, such as those in Apple iPhones, which overlook gaze considerations, especially in group settings, GARGI prioritizes minimizing gaze deviation for each subject relative to their expected gaze directions.
More about this research
Since the advent of the smartphone, the number of group images taken every day is rising exponentially. A group image consists of more than one person in it. While taking the group picture, photographers usually struggle to make sure that every person looks in the same direction, typically toward the camera. This occurs more often when multiple photographers take pictures of the same group. The direction in which a person looks is called the gaze. Gaze uniformity in group images is an important criterion to determine their aesthetic quality. To address the problem of gaze nonuniformity in group images, this dissertation proposes to invent novel techniques to detect and correct, if needed, the gaze uniformity in group images in a bias-aware manner. The images considered in the work include: i) multiple group images taken by a single photographer in instant mode, and in live mode (the mode typically available in Apple iPhone), and ii) one or more group images taken by more than one photographer in instant as well as live mode. In addition, the proposed gaze uniformity detection framework and underlying algorithms are also audited for gender bias. In this talk, the work performed in this direction so far and the plan to complete the remaining work along with a timeline will be presented.
Demo
Related Publications
-
O. Kulkarni, S. Arora, A. Mishra, V. K. Singh, and P. K. Atrey. A multi-stage bias reduction framework for eye gaze detection. MIPR'23: IEEE International Conference on Multimedia Information Processing ad Retrieval, Singapore, August 2023.
-
O. Kulkarni, S. Arora and P. K. Atrey. GARGI: Selecting gaze-aware representative group image from a live photo. The 5th IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR'2022), San Jose, USA, August 2022.
-
O. Kulkarni, V. Patil, V. K. Singh and P. K. Atrey. Accuracy and fairness in pupil detection algorithm. IEEE International Conference on Big Multimedia (BigMM'2021), Taichung, Taiwan, November 2021.
-
O. Kulkarni, V. Patil, S. Parikh, S. Arora and P. K. Atrey. Can you all look here? Towards determining gaze uniformity in group images. IEEE International Symposium on Multimedia (ISM'2020), Tapei, Taiwan, December 2020.
* Coming soom. *