• Home
  • News
  • 2018
  • July
  • Communicating with drones using facial expressions and gestures could help save lives


Communicating with drones using facial expressions and gestures could help save lives

July 24, 2018

With first responders beginning to use robots and drones to assist in search and rescue operations, having simple and easy to use ways of communicating with them can save precious moments and could help save lives.

In Simon Fraser University’s Autonomy Lab, researchers are using artificial intelligence to develop drone technologies that can make the human-robot interaction more intuitive, without requiring the standard controller.

“Most commercial drones today come with controllers which work really well, but sometimes you may find yourself in a situation where your hands are busy,” notes computing science professor Richard Vaughan, who leads this research.

Computing science professor Richard Vaughan (right) and PhD student Sepehr MohaimenianPour (left) are developing technologies in SFU's Autonomy Lab to help users interact with drones in a more intuitive manner.

“Or maybe you weren’t expecting to interact with a drone today so you don’t have special equipment with you. We’d like to be able to command drones in these situations and make the interaction natural and intuitive.”

Vaughan’s team is building a drone whose flight direction can be controlled by arm gestures—you can even command the drone to do a flip in mid-air.

Another drone the team has developed is one that executes commands based on the user’s facial expressions. By recognizing a “trigger-face” the drone could be used to carry out actions such as taking a photo or video of an inaccessible area to assess a situation.

SFU computing science PhD student Sepehr MohaimenianPour demonstrates one of the Autonomy Lab's technologies that allows users to communicate with drones using arm gestures. The computer screen in the foreground displays what the drone is "seeing" and how it can identify a person's face and hands.

“We would like to get to the point where interacting with a robot is as easy as working with a co-worker or a trained animal,” says Vaughan.

As robot technology continues to advance, it is likely that one day we will be interacting with robots in our everyday life. Being able to communicate and provide commands to them without a controller is not only convenient but can help navigate emergency situations.

Fast Facts:

  • Fire Fighters in the U.S. are already using drones for emergency scenarios like fires and search and rescue missions.
  • Drones can be deployed very quick. What used to take 10 or 15 minutes with just humans will only take 2-3 minutes with drones.
  • Currently, search and rescue teams send out 100 people out to walk on a foot in a grid path to find someone, but now you can use a drone thermal energy camera to pick that person out quickly.
  • According to BC Search and Rescue Association, there are 1600 incidents a year in B.C.