For robots to efficiently and effectively perform material recovery in complex environments, they need to see clearly and locate objects accurately. In the recent years, advancements in robotics have also accompanied developments in the fusion of tactile and visual sensing to enhance robotic comprehension of their surroundings.
But to really support this fusion and move towards a multimodal perception system, machine learning algorithms play a key role. It is precisely this combination of tactile and visual sensing that will truly empower robots with multimodal perception capabilities needed to perform highly complex tasks in challenging environments, such as detection and localisation of Critical Raw Materials (CRMs) present in hazardous waste streams, as in the iBot4CRMs project. This process will improve the recovery of materials that are of strategic importance to the European Union.
In principle, multimodal perception technologies are already being used in lab settings and with good results. But bring them onto the factory floor, and cracks in practical implementation begin to appear. The industry keeps asking this question: how can lab results be translated into real-world, industrial settings?
To respond with evidence, some EU-funded projects are collaborating to focus on this gap in our interactive workshop on “Multimodal perception: from mature technologies to industrial deployment”.
Our interactive workshop brings four EU-funded projects together on multimodal perception for robotics: COGNIMAN, iBot4CRMs, MUPSIS and AeroNex.
Here are the details:
Date and time: 13 October at 17:00 CEST
Location: Arrábida room, ADRF, Porto, Portugal
Register here to attend
Workshop description
Multimodal perception is maturing from laboratory technology into industrial systems, yet scaling remains difficult. This workshop brings together EU-funded projects addressing this frontier: COGNIMAN (manufacturing, presented by Lakshmy Arvind, Deep Blue), iBot4CRMs (raw materials recovery, presented by Ines Baeta, EGF and Marcelo Petri, INESC TEC), MUPSIS, and AeroNex (infrastructure monitoring, presented by Diāna Gauče, Sadales tīkls and Carlos Bernad, NORCE). Crucially, the session addresses the Data strand by examining sensor data pipelines, annotation scarcity, and sim-to-real transfer gaps.
End-user organisations: EGF (waste sector) and Sadales tīkls (energy infrastructure), are active participants, presenting operational challenges and validating practical impact. Following brief project overviews, participants will engage in structured flipchart sessions.
Moderator: Assia Belbachir, NORCE. Rapporteur: Antonio M. Ortiz, NORCE.
Predefined discussion questions:
- What technical/organisational conditions must be met before a multimodal perception
system can be deployed at production scale? - How can industry overcome data scarcity and high annotation costs for robust sim-to-real transfer?
- What regulatory and certification pathways are required to validate safety without
slowing down industrial deployment? - What other challenges industries are facing to integrate multimodal perception into
their daily activities?
Main workshop objectives:
- Share validated deployment results across manufacturing, waste, and energy sectors.
- Map persistent barriers regarding data pipelines, integration, and certification.
- Produce a strategic roadmap for dissemination via ADRA channels