WS#41 Towards next generation robotics: Generative AI use cases, challenges and opportunities

Andrea Orlandini (CNR-ISTC), Aksel A. Transeth (SINTEF), Magí Dalmau-Moreno (EURECAT)Silchersaal

iCal

Questions to be answered

What are the promising, near-term use cases for Generative AI, possibly in combination with other AI models, in robotics across sectors such as inspection, maintenance, assistive robotics, and manufacturing?
What are the latest research findings, opportunities, risks, and challenges with Generative AI in robotics?
In which areas should Europe lead in the field of Generative AI for robotics, and why?
Given the hype around LLMs and Generative AI, what is the realistic potential for impact, and should the focus on these technologies be balanced with other AI advancements?

Description

This workshop aims to provide a foundational understanding of the transformative potential of Generative AI (GenAI) in robotics by exploring current and emerging opportunities, practical applications, and challenges. With both academic and industrial perspectives, we’ll examine high-impact use cases across sectors such as inspection and maintenance, assistive robotics, and manufacturing, where GenAI is already proving valuable. Attendees will gain insight into the latest advancements in robotic capabilities powered by GenAI and complementary AI models, including foundation models for grasping, LLM-based code generation, and mission planning that combines LLMs with classical AI techniques. Discussions will also address the specific risks and challenges posed by GenAI in robotics, from data privacy to hardware constraints, to better understand the limitations of these technologies.

Through practical insights from key research projects, including TRIFFID, JARVIS, and PEER, this workshop will guide stakeholders on how GenAI is shaping the European robotics landscape. The session outcomes will be particularly valuable to end-users, suppliers, and researchers, providing them with insights into near- and medium-term high-value industry use cases and actionable strategies for cost-effective, safe, and efficient GenAI implementations in robotics. Attendees will leave with a clear understanding of both the potential and the obstacles associated with GenAI in robotics, including open software and hardware development challenges. A summary report will capture the key discussions and insights, contributing to broader strategic efforts in ADRA, euRobotics, and related networks.

Organisation of the WS

08:30 - 09:20 Presentations/Statements (50 minutes): A series of 7-8 minute talks on the workshop's key questions and insights from real-world GenAI use cases in robotics, presented by speakers from industry and academia
Talk about Challenges in GenAI for Robotics from euRobin project (Bruno Siciliano — Uni Naples, Italy)
From Surface to Depth: Generating Synthetic Underwater Images with Stable diffusion for perception for robotic underwater intervention (Ahmed K. Mohammed — SINTEF, Norway)
Formal methods combined with GenAI for task planning (Marco Roveri — Uni Trento, Italy)
Talk about Generative AI in Manufacturing (Sotiris Makris — LMS Patras University, Greece)
LLM and Autonomous robots: Potential use-cases in the energy industry (Svein Ivar Sagatun — Equinor, Norway)
(M)LLM reasoning for robots in open worlds (Silvia Izquierdo — Eurecat, Spain)
LLMs to control large robots (Nima Enayati — ABB, Germany)
09:20 - 09:50 Plenary Discussion with active participation from all attendees (with live polling and audience input on key questions)

Main outcomes and reasons to attend for stakeholders categorized as end-users (E), suppliers (S) and researchers (R):

Insights into near- and medium term high-value industry use cases to steer research, development and innovation efforts (S, R).
Clear understanding of what opportunities GenAI & robotics can create in terms of innovations to achieve cost reduction, HSE and efficiency in use cases and new products (E,S), and what are the challenges and risks.
Issues limiting a faster uptake and advance of GenAI in Robotics, identifying what are the open challenges, both in software and hardware development.

We will produce a summary of the workshop and share presentations in the cases possible. The inputs from the workshop could be used as inputs to other documents, e.g., strategy work in ADRA and euRobotics, and efforts in other organizations and networks.