Categoría: Robótica y Automatización

Autonomous mobility already exists… to some extent. Building an autonomous vehicle that can safely navigate an empty highway is one thing. The real challenge lies in adapting to the dynamic and messy reality of urban environments. Unlike the grid-like streets of many American cities, European roads are often narrow, winding and irregular. Urban environments have countless intersections without clear markings, pedestrian-only zones, roundabouts and areas where bicycles and scooters share the road with cars. Designing an autonomous mobility system that can safely operate in these conditions requires more than just sophisticated sensors and cameras. It’s mostly about tackling a tremendous challenge: predicting the dynamics of the world, in other words, understanding how humans navigate within given urban environments. Pedestrians, for example, often make spontaneous decisions such as darting across a street, suddenly changing direction, or weaving through crowds. A kid might run after a dog. Cyclists and scooters further complicate the equation, with their agile and often unpredictable maneuvers. “Autonomous mobility, whether in the form of self-driving cars or delivery robots, must evolve beyond merely reacting to the present moment. To navigate our complex, dynamic world, these AI-driven systems need the ability to imagine, anticipate, and simulate possible futures—just as humans do when we wonder what might happen next. In essence, AI must learn to wonder”, says Alexandre Alahi, head of EPFL’s Visual Intelligence for Transportation Laboratory (VITA). Pushing the boundaries of prediction: GEM At VITA laboratory, the goal of making AI “wonder” is becoming a reality. This year, the team has had seven papers accepted to the prestigious Conference on Computer Vision and Pattern Recognition (CVPR’25). Each contribution introduces a novel method to help AI systems imagine, predict, and simulate possible futures—from forecasting human motion to generating entire video sequences. In the spirit of open science, all models and datasets are being released as open source, empowering the global research community and industry to build upon and extend this work. Together, these contributions represent a unified effort to give autonomous mobility the ability not just to react, but to truly anticipate the world around them. One of the most innovative models is designed to predict video sequences from a single image captured by a camera mounted on a vehicle (or any egocentric view). Called GEM (Generalizable Ego-Vision Multimodal World Model), it helps autonomous systems anticipate future events by learning how scenes evolve over time. As part ot the Swiss AI Initiative, and in collaboration with four other institutions (University of Bern, SDSC, University of Zurich and ETH Zurich), they trained their model using 4000 hours of videos spanning autonomous driving, egocentric human activities (meaning, activities from the first person point of view) and drone footage. GEM learns how people and objects move in different environments. It uses this knowledge to generate entirely new video sequences that imagine what might happen next in a given scene, whether it’s a pedestrian crossing the street or a car turning at an intersection. These imagined scenarios can even be controlled by adding cars and pedestrians, making GEM a powerful tool for safely training and testing autonomous systems in a wide range of realistic situations. To make these predictions, the model looks simultaneously at several types of information, also called modalities. It analyzes RGB images—which are standard color video frames—to understand the visual context of a scene, and depth maps to grasp its 3D structure. These two data types together allow the model to interpret both what is happening and where things are in space. GEM also takes into account the movement of the camera (ego-motion), human poses, and object dynamics over time. By learning how all of these signals evolve together across thousands of real-world situations, It can generate coherent, realistic sequences that reflect how a scene might change in the next few seconds. “The tool can function as a realistic simulator for vehicles, drones and other robots, enabling the safe testing of control policies in virtual environments before deploying them in real-world conditions. It can also assist in planning by helping these robots anticipate changes in their surroundings, making decision-making more robust and context-aware,” says Mariam Hassan, Ph.D student at VITA lab. The road to predictions Predicting human behavior is a complex and multi-faceted challenge, and GEM represents just one piece of the VITA Lab’s broader effort to tackle it. While GEM focuses on generating the videos of the future and exposing autonomous systems to diverse virtual scenarios, other research projects from Professor Alahi’s team are tackling lower levels of abstractions to enhance prediction with robustness, generalizability, and social awareness. For example, one of them aims to certify where people will move, even when the data is incomplete or slightly off. Meanwhile, MotionMap tackles the inherent unpredictability of human motion through a probabilistic approach. This probabilistic approach helps systems prepare for unexpected movements in dynamic environments. These efforts form a comprehensive framework that maps out the complex interactions at play in crowded urban settings. There are still challenges: long-term consistency, high-fidelity spatial accuracy, and computational efficiency are still evolving. At the heart of it all lies the toughest question: how well shall we predict people who don’t always follow patterns? Human decisions are shaped by intent, emotion, and context—factors that aren’t always visible to machines. About the Swiss AI InitiativeLaunched in December 2023 by EPFL and ETH Zurich, the Swiss AI Initiative is supported by more than 10 academic institutions across Switzerland. With over 800 researchers involved and access to 10 million GPU hours, it stands as the world’s largest open science and open source effort dedicated to AI foundation models. The model developed by VITA lab, in collaboration with four other institutions (University of Bern, SDSC, University of Zurich and ETH Zurich) is among the first major models to emerge from this ambitious collaboration. It was trained on the Alps supercomputer at the Swiss National Supercomputing Centre (CSCS), which provided the massive computational power needed to process vast amounts of multimodal data. Autonomous mobility in SwitzerlandIn Switzerland, fully autonomous mobility

La movilidad autónoma ya existe… hasta cierto punto. Construir un vehículo autónomo que pueda circular con seguridad por una autopista vacía es una cosa. El verdadero reto reside en adaptarse a la dinámica y caótica realidad de los entornos urbanos. A diferencia de las calles cuadriculadas de muchas ciudades estadounidenses, las carreteras europeas suelen ser estrechas, sinuosas e irregulares. Los entornos urbanos presentan innumerables intersecciones sin señalización clara, zonas peatonales exclusivas, rotondas y zonas donde bicicletas y patinetes comparten la vía con coches. Diseñar un sistema de movilidad autónoma que pueda operar con seguridad en estas condiciones requiere algo más que sensores y cámaras sofisticados. Se trata principalmente de afrontar un enorme reto: predecir la dinámica del mundo; en otras palabras, comprender cómo se desenvuelven los humanos en entornos urbanos determinados. Los peatones, por ejemplo, suelen tomar decisiones espontáneas, como cruzar la calle a toda velocidad, cambiar de dirección repentinamente o sortear multitudes. Un niño podría correr tras un perro. Los ciclistas y los patinetes complican aún más la ecuación, con sus maniobras ágiles y a menudo impredecibles. La movilidad autónoma, ya sea en forma de coches autónomos o robots de reparto, debe evolucionar más allá de la simple reacción al momento presente. Para desenvolverse en nuestro mundo complejo y dinámico, estos sistemas impulsados ​​por IA necesitan la capacidad de imaginar, anticipar y simular futuros posibles, tal como lo hacemos los humanos cuando nos preguntamos qué podría suceder a continuación. En esencia, la IA debe aprender a preguntarse, afirma Alexandre Alahi, director del Laboratorio de Inteligencia Visual para el Transporte (VITA) de la EPFL. Ampliando los límites de la predicción: GEM En el laboratorio VITA, el objetivo de hacer que la IA «maraville» se está haciendo realidad. Este año, el equipo ha aceptado siete artículos en la prestigiosa Conferencia sobre Visión Artificial y Reconocimiento de Patrones (CVPR’25). Cada contribución introduce un método novedoso para ayudar a los sistemas de IA a imaginar, predecir y simular futuros posibles, desde la predicción del movimiento humano hasta la generación de secuencias de vídeo completas. En el espíritu de la ciencia abierta, todos los modelos y conjuntos de datos se publican como código abierto, lo que permite a la comunidad investigadora y a la industria global desarrollar y ampliar este trabajo. En conjunto, estas contribuciones representan un esfuerzo conjunto para dotar a la movilidad autónoma de la capacidad no solo de reaccionar, sino de anticiparse al mundo que la rodea. Uno de los modelos más innovadores está diseñado para predecir secuencias de vídeo a partir de una sola imagen captada por una cámara instalada en un vehículo (o cualquier vista egocéntrica). Denominado GEM (Modelo Mundial Multimodal Generalizable de Visión Egocéntrica), ayuda a los sistemas autónomos a anticipar eventos futuros al aprender cómo evolucionan las escenas con el tiempo. Como parte de la Iniciativa Suiza de IA, y en colaboración con otras cuatro instituciones (Universidad de Berna, SDSC, Universidad de Zúrich y ETH Zúrich), entrenaron su modelo utilizando 4000 horas de vídeos que abarcan conducción autónoma, actividades humanas egocéntricas (es decir, actividades desde una perspectiva en primera persona) y grabaciones de drones. GEM aprende cómo se mueven las personas y los objetos en diferentes entornos. Utiliza este conocimiento para generar secuencias de vídeo completamente nuevas que imaginan lo que podría suceder a continuación en una escena determinada, ya sea un peatón cruzando la calle o un coche girando en una intersección. Estos escenarios imaginarios pueden incluso controlarse añadiendo coches y peatones, lo que convierte a GEM en una potente herramienta para entrenar y probar sistemas autónomos de forma segura en una amplia gama de situaciones realistas. Para realizar estas predicciones, el modelo analiza simultáneamente varios tipos de información, también llamados modalidades. Analiza imágenes RGB (fotogramas de vídeo a color estándar) para comprender el contexto visual de una escena, y mapas de profundidad para comprender su estructura 3D. Estos dos tipos de datos, en conjunto, permiten al modelo interpretar tanto lo que sucede como la ubicación de los objetos en el espacio. GEM también tiene en cuenta el movimiento de la cámara (ego-motion), las poses humanas y la dinámica de los objetos a lo largo del tiempo. Al comprender cómo evolucionan todas estas señales en conjunto en miles de situaciones del mundo real, puede generar secuencias coherentes y realistas que reflejan cómo podría cambiar una escena en los próximos segundos. La herramienta puede funcionar como un simulador realista para vehículos, drones y otros robots, lo que permite probar de forma segura las políticas de control en entornos virtuales antes de implementarlas en condiciones reales. También puede ayudar en la planificación, ayudando a estos robots a anticipar los cambios en su entorno, lo que hace que la toma de decisiones sea más robusta y contextualizada, afirma Mariam Hassan, estudiante de doctorado en el laboratorio VITA. El camino hacia las predicciones Predecir el comportamiento humano es un desafío complejo y multifacético, y GEM representa solo una parte del esfuerzo más amplio del Laboratorio VITA para abordarlo. Mientras que GEM se centra en generar los videos del futuro y exponer sistemas autónomos a diversos escenarios virtuales, otros proyectos de investigación del equipo del profesor Alahi abordan niveles de abstracción más bajos para mejorar la predicción con robustez, generalización y conciencia social. Por ejemplo, uno de ellos busca certificar el movimiento de las personas, incluso cuando los datos están incompletos o son ligeramente erróneos. Por otro lado, MotionMap aborda la imprevisibilidad inherente del movimiento humano mediante un enfoque probabilístico. Este enfoque probabilístico ayuda a los sistemas a prepararse para movimientos inesperados en entornos dinámicos. Estos esfuerzos conforman un marco integral que mapea las complejas interacciones que ocurren en entornos urbanos abarrotados. Aún existen desafíos: la consistencia a largo plazo, la precisión espacial de alta fidelidad y la eficiencia computacional siguen evolucionando. En el centro de todo esto se encuentra la pregunta más difícil: ¿qué tan bien podemos predecir a las personas que no siempre siguen patrones? Las decisiones humanas están condicionadas por la intención, la emoción y el contexto,

A robotic hand developed at EPFL can pick up 24 different objects with human-like movements that emerge spontaneously, thanks to compliant materials and structures rather than programming. When you reach out your hand to grasp an object like a bottle, you generally don’t need to know the bottle’s exact position in space to pick it up successfully. But as EPFL researcher Kai Junge explains, if you want to make a robot that can pick up a bottle, you must know everything about the surrounding environment very precisely. “As humans, we don’t really need too much external information to grasp an object, and we believe that’s because of the compliant – or soft – interactions that happen at the interface between an object and a human hand,” says Junge, a PhD student in the School of Engineering’s Computational Robot Design & Fabrication (CREATE) Lab, led by Josie Hughes. “This compliance is what we are interested in exploring for robots.” In robotics, compliant materials are those that deform, bend, and squish. In the case of the CREATE Lab’s robotic ADAPT hand (Adaptive Dexterous Anthropomorphic Programmable sTiffness), the compliant materials are relatively simple: strips of silicone wrapped around a mechanical wrist and fingers, plus spring-loaded joints, combined with a bendable robotic arm. But this strategically distributed compliance is what allows the device to pick up a wide variety of objects using “self-organized” grasps that emerge automatically, rather than being programmed. In a series of experiments, the ADAPT hand, which can be controlled remotely, was able to pick up 24 objects with a success rate of 93%, using self-organized grasps that mimicked a natural human grasp with a direct similarity of 68%. The research has been published in Nature Communications Engineering. ‘Bottom-up’ robotic intelligence While a traditional robotic hand would need a motor to actuate each joint, the ADAPT hand has only 12 motors, housed in the wrist, for its 20 joints. The rest of the mechanical control comes from springs, which can be made stiffer or looser to tune the hand’s compliance, and from the silicone ‘skin’, which can also be added or removed. As for software, the ADAPT hand is programmed to move through just four general waypoints, or positions, to lift an object. Any further adaptations required to complete the task occur without additional programming or feedback; in robotics, this is called ‘open loop’ control. For example, when the team programmed the robot to use a certain motion, it was able to adapt its grasp pose to various objects ranging from a single bolt to a banana. The researchers analyzed this extreme robustness — thanks to the robot’s spatially distributed compliance — with over 300 grasps and compared them against a rigid version of the hand. We are interested in exploiting the distributed mechanical intelligence of different body parts like skin, muscles, and joints, as opposed to the top-down intelligence of the brain.Kai Junge, CREATE Lab “Developing robots that can perform interactions or tasks that humans do automatically is a lot harder than most people expect,” Junge says. “That’s why we are interested in exploiting this distributed mechanical intelligence of different body parts like skin, muscles, and joints, as opposed to the top-down intelligence of the brain.” Balancing compliance and control Junge emphasizes that the goal of the ADAPT study was not necessarily to create a robotic hand that can grasp like a human, but to show for the first time how much a robot can achieve through compliance alone. Now that this has been demonstrated systematically, the EPFL team is building on the potential of compliance by re-integrating elements of closed-loop control into the ADAPT hand, including sensory feedback – via the addition of pressure sensors to the silicone skin – and artificial intelligence. This synergistic approach could lead to robots that combine compliance’s robustness to uncertainty, and the precision of closed-loop control. “A better understanding of the advantages of compliant robots could greatly improve the integration of robotic systems into highly unpredictable environments, or into environments designed for humans,” Junge summarizes. References Junge, K., Hughes, J. Spatially distributed biomimetic compliance enables robust anthropomorphic robotic manipulation. Commun Eng 4, 76 (2025). https://doi.org/10.1038/s44172-025-00407-4 EPFL News. C. L.