Follow the Eyes: Head-Mounted Cameras Could Help Robots Understand Social Interactions

What is everyone looking at? It’s a common question in social settings because the answer identifies something of interest, or helps delineate social groupings. Those insights someday will be essential for robots designed to interact with humans, so researchers at Carnegie Mellon University’s Robotics Institute have developed a method for detecting where people’s gazes intersect.

The researchers tested the method using groups of people with head-mounted video cameras. By noting where their gazes converged in three-dimensional space, the researchers could determine if they were listening to a single speaker, interacting as a group, or even following the bouncing ball in a ping-pong game.

The system thus uses crowdsourcing to provide subjective information about social groups that would otherwise be difficult or impossible for a robot to ascertain.

The researchers’ algorithm for determining “social saliency” could ultimately be used to evaluate a variety of social cues, such as the expressions on people’s faces or body movements, or data from other types of visual or audio sensors.

"This really is just a first step toward analyzing the social signals of people," said Hyun Soo Park, a Ph.D. student in mechanical engineering, who worked on the project with Yaser Sheikh, assistant research professor of robotics, and Eakta Jain of Texas Instruments, who was awarded a Ph.D. in robotics last spring. "In the future, robots will need to interact organically with people and to do so they must understand their social environment, not just their physical environment."

Social Synchronicity

Humans have a tendency to spontaneously synchronize their movements. For example, the footsteps of two friends walking together may synchronize, although neither individual is consciously aware that it is happening. Similarly, the clapping hands of an audience will naturally fall into synch. Although this type of synchronous body movement has been observed widely, its neurological mechanism and its role in social interactions remain obscure. In a new study, led by cognitive neuroscientists at the California Institute of Technology (Caltech), researchers found that body-movement synchronization between two participants increases following a short session of cooperative training, suggesting that our ability to synchronize body movements is a measurable indicator of social interaction.

"Our findings may provide a powerful tool for identifying the neural underpinnings of both normal social interactions and impaired social interactions, such as the deficits that are often associated with autism," says Shinsuke Shimojo, Gertrude Baltimore Professor of Experimental Psychology at Caltech and senior author of the study.

Shimojo, along with former postdoctoral scholar Kyongsik Yun, and Katsumi Watanabe, an associate professor at the University of Tokyo, presented their work in a paper published December 11 in Scientific Reports, an online and open-access journal from the Nature Publishing Group.

"The most striking outcome of our study is that not only the body-body synchrony but also the brain-brain synchrony between the two participants increased after a short period of social interaction," says Yun. "This may open new vistas to study the brain-brain interface. It appears that when a cooperative relationship exists, two brains form a loose dynamic system."

The team says this information may be potentially useful for romantic or business partner selection.

"Because we can quantify implicit social bonding between two people using our experimental paradigm, we may be able to suggest a more socially compatible partnership in order to maximize matchmaking success rates, by preexamining body synchrony and its increase during a short cooperative session" explains Yun.

Science Reveals the Power of a Handshake

New neuroscience research is confirming an old adage about the power of a handshake: strangers do form a better impression of those who proffer their hand in greeting.         

A firm, friendly handshake has long been recommended in the business world as a way to make a good first impression, and the greeting is thought to date to ancient times as a way of showing a stranger you had no weapons. Now, a paper published online and for the December print issue of the Journal of Cognitive Neuroscience on a study of the neural correlates of a handshake is giving insight into just how important the practice is to the evaluations we make of subsequent social interactions.

The study was led by Beckman Institute researcher Florin Dolcos and Department of Psychology postdoctoral research associate Sanda Dolcos. They found, as they wrote, that “a handshake preceding social interaction enhanced the positive impact of approach and diminished the negative impact of avoidance behavior on the evaluation of social interaction.”

Their results, for the first time, give a scientific underpinning to long-held beliefs about the important role a handshake plays in social or business interactions. Sanda Dolcos said their findings have obvious implications for those who want to make a good impression.

“I would tell them to be aware of the power of a handshake,” she said. “We found that it not only increases the positive effect toward a favorable interaction, but it also diminishes the impact of a negative impression. Many of our social interactions may go wrong for a reason or another, and a simple handshake preceding them can give us a boost and attenuate the negative impact of possible misunderstandings.”