Amania: AI-based interaction techniques for Human-Swarm Interaction

by | Mar 18, 2024 | Projects

The primary objective of this project is to create a Human-Centred Intelligent System that facilitates collaboration between human operators and swarms of drones. In contrast to the conventional idea that increased automation reduces human control, this project aligns with a two-dimensional framework proposed by Shneiderman (2022) that demonstrates that innovative designers can envision highly automated systems while retaining human control. The project focuses on the context of controlling multiple UAVs by a single operator, considering the challenges of managing and monitoring a fleet of drones effectively, with a focus on using AI planning and navigation tools and scene understanding in restricted visual conditions.

Innovative Aspects:

  • The project will leverage environment aware path planning and VR technologies to make swarms control effective for a single human operator. We will also address the challenge of transitioning between swarm control and single drone control without causing disorientation or motion sickness.
  • It introduces algorithms for removing rotor noise from drone audio recordings, detecting and classifying acoustic events, and creating an acoustic map of the area, for scene understanding in situations with limited visibility. The extension of StickEar technology to drones for acoustic scene monitoring is a novel approach.

Expected Outcomes:

  • Development of a VR-based interface for supervising and controlling a swarm of drones.
  • Insights into the impact of swarm behavior and control on human operators’ workload, situation awareness, and decision-making.
  • New interaction techniques for controlling drone swarms, combining AI and manual control, and supporting Search and Rescue (S&R) operations.
  • Models to predict and reduce motion sickness in VR interfaces.
  • Algorithm to remove rotor noise from audio recordings.
  • Algorithm for detecting, classifying, and localising acoustic events from drone microphones.
  • A curated dataset of microphone recordings with labelled events.
  • Neural network models for noise suppression and acoustic event detection.

PIs:

  • C. Jouffrais, DR CNRS, FR
  • S. Nanayakkara, Ass. Prof, NUS, SG

Co-Is:

  • J. Garcia, ENAC, FR
  • M. Bronz, ENAC, FR
  • C. Gupta, Senor RF,  NUS, SG

PhD students:

  • TBD