Advancing Robotics Technology

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data.  2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro  -> RoboCasa produces N (varying visuals)  -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are creating tools to enable everyone in the ecosystem to scale up with us: - RoboCasa: our generative simulation framework (Yuke Zhu). It's fully open-source! Here you go: http://robocasa.ai - MimicGen: our generative action framework (Ajay Mandlekar). The code is open-source for robot arms, but we will have another version for humanoid and 5-finger hands: https://lnkd.in/gsRArQXy - We are building a state-of-the-art Apple Vision Pro -> humanoid robot "Avatar" stack. Xiaolong Wang group’s open-source libraries laid the foundation: https://lnkd.in/gUYye7yt - Watch Jensen's keynote yesterday. He cannot hide his excitement about Project GR00T and robot foundation models! https://lnkd.in/g3hZteCG Finally, GEAR lab is hiring! We want the best roboticists in the world to join us on this moon-landing mission to solve physical AGI: https://lnkd.in/gTancpNK

🦾 Great milestone for open-source robotics: pi0 & pi0.5 by Physical Intelligence are now on Hugging Face, fully ported to PyTorch in LeRobot and validated side-by-side with OpenPI for everyone to experiment with, fine-tune & deploy in their robots! π₀.₅ is a Vision-Language-Action model which represents a significant evolution from π₀ to address a big challenge in robotics: open-world generalization. While robots can perform impressive tasks in controlled environments, π₀.₅ is designed to generalize to entirely new environments and situations that were never seen during training. Generalization must occur at multiple levels: - Physical Level: Understanding how to pick up a spoon (by the handle) or plate (by the edge), even with unseen objects in cluttered environments - Semantic Level: Understanding task semantics, where to put clothes and shoes (laundry hamper, not on the bed), and what tools are appropriate for cleaning spills - Environmental Level: Adapting to "messy" real-world environments like homes, grocery stores, offices, and hospitals The breakthrough innovation in π₀.₅ is co-training on heterogeneous data sources. The model learns from: - Multimodal Web Data: Image captioning, visual question answering, object detection - Verbal Instructions: Humans coaching robots through complex tasks step-by-step - Subtask Commands: High-level semantic behavior labels (e.g., "pick up the pillow" for an unmade bed) - Cross-Embodiment Robot Data: Data from various robot platforms with different capabilities - Multi-Environment Data: Static robots deployed across many different homes - Mobile Manipulation Data: ~400 hours of mobile robot demonstrations This diverse training mixture creates a "curriculum" that enables generalization across physical, visual, and semantic levels simultaneously. Huge thanks to the Physical Intelligence team & contributors Model: https://lnkd.in/eAEr7Yk6 LeRobot: https://lnkd.in/ehzQ3Mqy

🚀𝐖𝐞 𝐚𝐫𝐞 𝐢𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐢𝐧𝐠 𝐒𝐦𝐨𝐥𝐕𝐋𝐀-𝟒𝟓𝟎𝐌, 𝐚𝐧 𝐨𝐩𝐞𝐧-𝐬𝐨𝐮𝐫𝐜𝐞 𝐕𝐢𝐬𝐢𝐨𝐧-𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞-𝐀𝐜𝐭𝐢𝐨𝐧 𝐦𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐫𝐨𝐛𝐨𝐭𝐢𝐜𝐬! SmolVLA achieves best-in-class performance and inference speed, and the best part? It’s trained entirely on open-source datasets from the 🤖 LeRobot project hosted on the Hugging Face Hub. 🔍 Why is SmolVLA so good? Turns out that pretraining on a large, diverse and noisy collection of real-world community robotics data leads to better generalization and control. We saw a 26% boost in task success rate simply from adding community dataset pretraining! ⚡ Why is SmolVLA so fast? 1. We halved the size of SmolVLM and extract intermediate representations 2. Introduced interleaved cross- and self-attention layers in the action expert 3. Enabled asynchronous inference so the robot acts and reacts simultaneously 💡 Unlike most academic datasets, these community-contributed datasets are naturally diverse: ✅ Multiple robots, camera angles, and manipulation tasks ✅ Real-world messiness and complexity ✅ Crowd-sourced and community-cleaned using Qwen2.5-VL for high-quality task descriptions 🌍 SmolVLA is a step toward making robotics research more affordable, reproducible, and collaborative. 📖 Want to dive deeper? Check out our blog post & start using it today: https://lnkd.in/e3Gmy8gT Huge thanks to the team who made this possible: @Mustafa Shukor Francesco Capuano Remi Cadene, and the entire Lerobot team, amazing HF team Andrés Marafioti Merve Noyan Aritra Roy Gosthipaty Pedro Cuenca Loubna Ben Allal, Thomas Wolf  and to the amazing contributors to the LeRobot community: Ville Kuosmanen, Alexandre Chapin, Marina Barannikov, and more!

Terminator just went open source. This humanoid robot you’re seeing isn’t being manually programmed. It’s being controlled by an AI agent. Over the past few years we’ve watched AI slowly expand its reach. First it was text. Then images. Then video and voice. Then AI started operating software and doing tasks on computers... Now it’s starting to control physical machines. A developer called Stash Pomichter and the team behind Dimensional / DimOS just released an open-source system that connects AI agents like OpenClaw directly to robots. Humanoids, drones, quadrupeds, robotic arms. The idea is simple but powerful. Instead of writing thousands of lines of robotics code, you can prompt the system in natural language and the agent handles perception, navigation, memory, and control. Cameras, lidar, spatial mapping, movement, planning. Basically the robotics version of vibe coding. You tell the robot what you want and the agent figures out how to do it. In their demo a Unitree humanoid robot runs through this stack and can be instructed with a simple command while the agent interprets the environment and decides how to act. They also built something called Spatial Agent Memory, which lets the AI store and search real-world information over time. Things like where objects were seen, who entered a room, how spaces are structured, and what happened earlier. So the agent isn’t just reacting. It’s building a model of the physical world. And the entire stack was just released open source. If social media suddenly filled with AI-generated text, images, and videos once the tools became accessible… imagine what happens when physical robots become programmable the same way. What a time to be alive. Follow Endrit Restelica for more tech stuff.

China is turning fire trucks into drone launch systems. And that is a much bigger shift than it sounds. What interests me here is not just the hardware. It is the new logic of emergency response. Instead of relying only on ladders and human entry, these systems pair fire trucks with drones that can reach high-rise fire zones quickly, fly into smoke, and send live intelligence back to crews. That is what is new. The truck is no longer just transport. It becomes a mobile aerial response base. And that matters because in dense high-rise environments, access is often the real bottleneck. To me, this is where the story gets interesting. This is not just about fighting fires better. It is about changing who gets exposed to danger first. → drones go where ladders cannot → commanders get visibility earlier → crews make faster decisions → fewer firefighters enter blind conditions That is a serious innovation. And it opens up important use cases: → faster high-rise reconnaissance → targeted suppression from outside upper floors → better coordination in smoke-heavy environments → safer response where humans cannot reach quickly That is why I would not dismiss this as just another drone demo. It is a glimpse of what emergency response looks like when robotics, data, and frontline operations finally converge. What do you think matters more here: faster firefighting, or the fact that robots may now take the first risk instead of humans? #AI #Robotics #Drones #Firefighting #Innovation #EmergencyResponse #SmartCities #FutureOfWork #Technology

The future of elder care hinges on innovation. I know this first hand, and I lost my mother over a year ago. Through my experience caring for my mom, I saw how AI can transform how we support our aging population. Here’s how AI can revolutionize care for the elderly: 🤖 Personalized Care at Scale: AI analyzes health data to create customized care plans. This means better health outcomes tailored to each individual’s unique needs. 🏡 Promoting Independence: Smart home technologies powered by AI help seniors live independently longer. From fall detection to medication reminders, AI supports seniors in their desire to live independently longer and facilitates daily living. 👥 Reducing Caregiver Burden: AI tools can take over routine tasks, freeing up caregivers to focus on what matters most—human connection and emotional support. 🩺 Proactive Health Monitoring: AI tracks vital signs in real-time, predicting potential health issues before they become serious. Early intervention keeps seniors safer and healthier. 🚶♀️ Empowering Aging in Place: AI-enabled devices assist with mobility, home safety, and social engagement, helping seniors remain in their homes, surrounded by familiarity and comfort. Here’s how you can leverage AI to transform elder care: 🔍 Adopt AI-Powered Tools: Explore AI solutions that offer real-time health monitoring, personalized care plans, and smart home integrations. 🤝 Collaborate with Tech Providers: Work closely with AI developers to ensure that the tools meet the specific needs of the elderly population. 🌐 Educate and Empower: Provide training and resources for caregivers and seniors to integrate AI into their daily routines seamlessly. . 💡 Focus on human-AI collaboration: For the best outcomes, combine AI's strengths with human caregivers' empathy. . Did you know that by 2050, the global population aged 60 and over is projected to double? AI isn’t just an option—it’s essential for future care. Empower independence. Transform care. Embrace AI.

Engineers at Queen Mary University of London have developed a colour-changing tactile sensor that allows robots to see and feel touch in real time using a standard camera. Instead of relying on thousands of electronic pressure sensors, the material changes colour when touched, instantly revealing pressure, strain, and contact points. The new design eliminates complex reconstruction algorithms, making robot touch sensing simpler, faster, and potentially much cheaper. Researchers say the technology could improve precision manufacturing, robotic prosthetics, and surgical robots by helping them handle delicate objects with greater accuracy. The team believes turning touch into visible colour patterns could overcome one of robotics’ biggest challenges while bringing robots closer to a human-like sense of touch.

"The field of embodied AI (EAI) is rapidly advancing. Unlike virtual AI, EAI systems can exist in, learn from, reason about, and act in the physical world. With recent advances in AI models and hardware, EAI systems are becoming increasingly capable across wider operational domains. While EAI systems can offer many benefits, they also pose significant risks, including physical harm from malicious use, mass surveillance, as well as economic and societal disruption. These risks require urgent attention from policymakers, as existing policies governing industrial robots and autonomous vehicles are insufficient to address the full range of concerns EAI systems present. To help address this issue, this paper makes three contributions. First, we provide a taxonomy of the physical, informational, economic, and social risks EAI systems pose. Second, we analyze policies in the US, EU, and UK to assess how existing frameworks address these risks and to identify critical gaps. We conclude by offering policy recommendations for the safe and beneficial deployment of EAI systems, such as mandatory testing and certification schemes, clarified liability frameworks, and strategies to manage EAI’s potentially transformative economic and societal impacts" Jared Perlo Centre for the Governance of AI (GovAI) Centre pour la Sécurité de l'IA - CeSIA) Alex Robey Fazl Barez Luciano Floridi Jakob Mökander Tony Blair Institute for Global Change Digital Ethics Center (DEC), Yale University

It is always great to see something published that you have worked on for a considerable amount of time – but in this case, it feels really special. AI Act, GDPR, DSA, finance, medical devices, automotive regulation: so many things close to my (academic) heart, and I could combine them all in one study on the frictions, interdependencies, and ways forward through this regulatory jungle. Here are the key policy recommendations, structured by addressees, many more in the study, someone counted 25 :). Important: Almost all of them can be achieved without any diminished protection of fundamental rights.   European Legislators   1. Designate a "Lead Act": Assign a leading regulatory framework for each sector, such as the AI Act or sector-specific laws, to reduce conflicts and enhance coherence. If that Lead Act is complied with, compliance of the other designated acts should be presumed, unless some specific provisions are exempted from that rule. Example: Art. 17(4) AI Act, one of my favorite norms in the Act, a hidden gem ;)   2. Clarify AI Act-GDPR Alignment: Address contradictions, such as differing responsibilities for AI providers under the AI Act and data controllers under the GDPR, and rules for training AI on personal data.   3. Develop Safe Harbor Standards: Create technical standards that provide compliance with the AI Act AND related regulations.   4. Conduct Regular External Reviews: Periodically and EXTERNALLY evaluate the AI Act's implementation to address contradictions, regulatory gaps and new technological challenges.    European Commission (AI Office and Sectoral Authorities)   5. Enhance Risk Analysis for Hybrid Platforms: Develop integrated guidelines for platforms that incorporate generative AI, addressing systemic risks under both the AI Act and the DSA, and the mutual reinforcement of the specific platform and GenAI risks.   6. Expand Data Access for Research: Establish mechanisms for vetted researchers to access both platform AND AI system data, inspired by the DSA’s Article 40.   National Legislators and Authorities   7. Support SMEs: Introduce grant programs to help small and medium-sized enterprises comply with AI Act and sector-specific regulations. This could, for example, fund access to training programs.   8. Foster Oversight Synergies: Clearly institutionalize the necessary collaboration between national data protection, sectoral and AI Act oversight authorities for cohesive enforcement. Be agile and project-based in solving cases involving multiple Acts.   Standardization Bodies   9. Develop Unified Standards: Provide technical standards for the AI Act AND sectoral regulations.   Industry and Civil Society   10. Encourage Cross-Disciplinary Collaboration: Establish advisory groups combining industry, academic, and civil society expertise and liaising with the national AI authorities to address sector-specific challenges.   Many thanks to Bertelsmann Stiftung, Julia Gundlach and Asena Soydas for enabling this!        

A neural network cannot execute a complex task if the end-effector lacks the mechanical capability to express it. Paris-based GenesisAI just highlighted exactly what it takes to close this hardware-software gap. In their latest rollout, they showcased a dual-arm manipulator running a single foundation model—GENE-26.5—to autonomously execute complex, multi-step physical tasks. To achieve this, they bypassed traditional parallel grippers and integrated a tailored set of 22-DoF Wuji Hands. The engineering architecture relies on a unified system rather than fragmented, task-specific code. The AI is trained on a massive triad of multimodal data: - First-person human demonstration videos. - High-fidelity telemetry from motion, force, and tactile feedback gloves. - Large-scale, closed-loop simulation. By standardizing around highly dexterous, human-scale hands, Genesis is ensuring their foundation models can physically execute the workflows, tool operations, and tactile nuances originally designed for people. #PhysicalAI #Robotics #GenesisAI #WujiTech #Engineering #Automation #FoundationModels