Principles from Neuroscience Give Robots Behaviors They Were Never Trained For
At Voaige, our mission is to understand the computational principles that make intelligence possible, and then build systems that exhibit intelligent behavior grounded in those principles. In our previous post, we showed what this looks like in practice: a single observation about the brain's locally random wiring led to architectures that achieved 10x efficiency gains in deep learning. That work focused on perception, specifically how a network can learn to see robustly with far less compute.
This post is about what comes next. Perception alone is not intelligence. An agent that can recognize objects but cannot reason about how to act on them, adapt when things go wrong, or compose familiar skills into novel strategies is just pattern matching. The next chapter for us moves from perception into action, and the central question becomes: how do you build robotic systems that can handle the open-ended complexity of the physical world without needing to be shown every possible situation in advance?
Robotic manipulation spans a combinatorially large space of embodiments, tools, workspaces, and constraints. The dominant approach today, collecting teleoperation demonstrations and training a policy to imitate them, works well when the training distribution densely covers the situations the robot will encounter, but degrades sharply at the edges, precisely where robotics gets interesting.
The problem is compositional. A dual-arm robot might need to push an object clear of an obstacle with one tool, reposition it into the reachable zone of another arm, and then grasp it, a sequence that was never demonstrated end-to-end. A teleoperation pipeline would need to have captured not just each individual skill, but the specific multi-skill transitions that resolve the long-tail failure modes or mistakes of each skill's control policy, across many environment configurations. The number of required demonstrations scales roughly with the product of skills, their possible handoffs, and their error-recovery strategies and trajectories, making long-tail coverage prohibitively expensive.
This is not just an engineering inconvenience. It reflects a fundamental limitation: imitation learning encodes solutions rather than understanding. The robot learns what to do in situations it has seen, but has no mechanism to reason about what would happen under alternative actions in situations it has not seen.
Biological agents do not solve this problem by accumulating demonstrations. A child learning to use tools does not need to observe every possible obstacle configuration paired with the correct recovery strategy. Instead, they build internal models of how the world works, how objects move when pushed, what happens when something is blocked, which actions are available given the current state, and use those models to plan, compose, and recover on the fly.
This is the principle we wanted to translate. Not the neural implementation details, but the computational strategy: learn how the world responds to your actions, then reason through that model at test time to solve new problems. Rather than expanding the demonstration dataset to cover every composite trajectory, corner case, and error-correction behavior, learn reusable skill-level models that can be recombined and reasoned through to handle novel situations, satisfy new constraints, and resolve unexpected failures.
We adopted a world model-first methodology evaluated in a deliberately constrained, small-scale robotic setting designed to stress-test compositional generalization.
The intended outcome is a system where generalization is achieved by test-time reasoning and composition over learned predictive models, rather than by expanding the demonstration dataset to cover all composite trajectories, corner cases, and error-correction behaviors.
We explicitly targeted solutions that are non-obvious: success often requires intermediate interactions (e.g., freeing an object from obstacles, moving it into a reachable region, then grasping) that are not trivially implied by the goal specification.
Key challenges:
Overall, our emphasis was not merely on the complexity of the task, but on demonstrating that skill-specific world models plus planning enable one-shot, constraint-aware composition and self-correction at test time, without needing the system to learn the full behavior from end-to-end demonstrations.
We consistently find that our world model-based system exhibits diverse behaviors that were not trained for, smoothly stitching skills together to achieve goals that are not directly achievable or that require intelligent, constraint-aware subgoal discovery. We also observed dynamic error recovery where the system hands responsibility between skills, sometimes multiple times within an episode.
Furthermore, we find it robust across many environments, a wide range of obstacle complexities, and across different robot embodiments.
To successfully perform this task, our system was able to construct composite skill trajectories dynamically at test time: first use the stick to move the cube away from obstacles, then lift the cube using the gripper.
It is important to remember that the stick arm and gripper arm (each with their own skill policies and skill world models) were trained fully separately: at no point in training did the system see this full, compositional trajectory. Based on the state of the environment, the system successfully inferred that directly lifting the cube is not an option, and planned a composite trajectory using the complementary strengths of the two arms.
This approach contrasts sharply with teleoperation-based systems, which would have to cover such composite trajectories within the training data. As the set of individual skills becomes larger, the harder it is to augment the training set with useful composite trajectories, an issue that world model-based systems are much less susceptible to.
The system was able to effectively infer subgoals that must be achieved before the final objective (lifting the cube) became feasible. Our planner infers which skills should activate at each step, and the corresponding world model predicts the resulting state. By visualizing those predictions, we can see the inferred subgoals explicitly.
Finally, we observed that the system dynamically hands responsibility back and forth between the two tools/skills, based on the current state of the environment.
The clearest example is when the system first attempts to pick the cube immediately, then realizes it cannot (because of an obstacle), then switches to the stick tool until the cube is free and within range of the gripper. In some cases we saw multiple handoffs if the gripper slipped and pushed the cube away, handing the task back to the stick arm until the system was confident the gripper could complete the lift.
This work encapsulates Voaige's approach to AI: rather than trying to solve robotics with ever-larger demonstration datasets, we show that principles from neuroscience can lead to systems that reason through novel situations they were never explicitly trained for.
The core insight is simple: teach robots how the world works, not just what to do. By training skill-conditioned world models independently and composing them at test time through planning, we built a system that exhibits three critical capabilities that emerge naturally from this architecture:
These behaviors emerged from the interaction between learned predictive models and test-time planning, without the full behavior being demonstrated end-to-end. This is what separates understanding from imitation: a robot that has learned how its actions affect the world can generalize to situations it has never encountered, compose skills it has never chained together, and recover from failures it has never experienced.
This continues our broader mission at Voaige: identify the computational principles that make intelligence possible, and build systems grounded in those principles. We showed in our previous work how neuroscience-inspired architectures could achieve 10x efficiency gains in perception. Here, we've demonstrated that the same neuroscience-first philosophy enables robots to exhibit sophisticated, adaptive behavior in action and manipulation.