WS#41 Towards next generation robotics: Generative AI use cases, challenges and opportunities

WS#41 Towards next generation robotics: Generative AI use cases, challenges and opportunities

Andrea Orlandini (CNR-ISTC), Aksel A. Transeth (SINTEF), Magí Dalmau-Moreno (EURECAT)Silchersaal

Questions to be answered

  1. What are the promising, near-term use cases for Generative AI, possibly in combination with other AI models, in robotics across sectors such as inspection, maintenance, assistive robotics, and manufacturing?
  2. What are the latest research findings, opportunities, risks, and challenges with Generative AI in robotics?
  3. In which areas should Europe lead in the field of Generative AI for robotics, and why?
  4. Given the hype around LLMs and Generative AI, what is the realistic potential for impact, and should the focus on these technologies be balanced with other AI advancements?

Description

This workshop aims to provide a foundational understanding of the transformative potential of Generative AI (GenAI) in robotics by exploring current and emerging opportunities, practical applications, and challenges. With both academic and industrial perspectives, we’ll examine high-impact use cases across sectors such as inspection and maintenance, assistive robotics, and manufacturing, where GenAI is already proving valuable. Attendees will gain insight into the latest advancements in robotic capabilities powered by GenAI and complementary AI models, including foundation models for grasping, LLM-based code generation, and mission planning that combines LLMs with classical AI techniques. Discussions will also address the specific risks and challenges posed by GenAI in robotics, from data privacy to hardware constraints, to better understand the limitations of these technologies.

Through practical insights from key research projects, including TRIFFID, JARVIS, and PEER, this workshop will guide stakeholders on how GenAI is shaping the European robotics landscape. The session outcomes will be particularly valuable to end-users, suppliers, and researchers, providing them with insights into near- and medium-term high-value industry use cases and actionable strategies for cost-effective, safe, and efficient GenAI implementations in robotics. Attendees will leave with a clear understanding of both the potential and the obstacles associated with GenAI in robotics, including open software and hardware development challenges. A summary report will capture the key discussions and insights, contributing to broader strategic efforts in ADRA, euRobotics, and related networks.

Organisation of the WS

Presentations/Statements (50 minutes)

  • Series of 7-8 minute talks by speakers addressing the main questions of the workshop and presenting insights from real-world GenAI use cases in robotics.
  • Additional speakers to address key themes in GenAI for robotics, including representation from industry and academia.

Plenary Discussion (30 minutes)

  • Moderated plenary session, with active participation from all workshop attendees.
  • Use of online tools like Slido for live polling and audience input on key questions.

Main outcomes and reasons to attend for stakeholders categorized as end-users (E), suppliers (S) and researchers (R):

  • Insights into near- and medium term high-value industry use cases to steer research, development and innovation efforts (S, R).
  • Clear understanding of what opportunities GenAI & robotics can create in terms of innovations to achieve cost reduction, HSE and efficiency in use cases and new products (E,S), and what are the challenges and risks.
  • Issues limiting a faster uptake and advance of GenAI in Robotics, identifying what are the open challenges, both in software and hardware development.

We will produce a summary of the workshop and share presentations in the cases possible. The inputs from the workshop could be used as inputs to other documents, e.g., strategy work in ADRA and euRobotics, and efforts in other organizations and networks.

Intended outcome

Speakers, Panelists, etc

  1. Bruno Siciliano, Uni Naples, Italy Talk about Challenges in GenAI for Robotics from euRobin project (along the interview reported here https://www.scuoladirobotica.it/en/bruno-siciliano-generative-ai-and-robotics/)
  2. Ahmed K. Mohammed, SINTEF, From Surface to Depth: Generating Synthetic Underwater Images with Stable diffusion for perception for robotic underwater intervention. From the SAFESub project: https://www.sintef.no/en/projects/2023/safesub_en/
  3. Marco Roveri, Uni Trento, Italy. Formal methods combined with GenAI for task planning.
  4. Sotiris Makris, LMS Patras University, Greece. Talk about Generative AI in Manufacturing.
  5. Svein Ivar Sagatun, Equinor, LLM and Autonomous robots: Potential use-cases in the energy industry.
  6. Silvia Izquierdo, Eurecat, (M)LLM reasoning for robots in open worlds.
  7. Nima Enayati, ABB, LLMs to control large robots.

Topic Groups and/or Innovation networks involved

Inspection and maintenance euRobotics topic group,

RIMA Alliance (euRobotics Innovation Network)

TG on Artificial Intelligence and Cognition in Robotics (euRobotics Topic Group)

Also collaboration with the ADRA topic group on I&M.

Projects involved

Further information

Organisers

  • Andrea Orlandini (CNR-ISTC), andrea.orlandini@istc.cnr.it
  • Aksel A. Transeth (SINTEF), aksel.a.transeth@sintef.no
  • Magí Dalmau-Moreno (EURECAT) magi.dalmau@eurecat.org
Thu 08:30 - 09:50
Challenges, Data, Generative AI, Industry, Planning, Use-cases