A ball at 150 km/h, a robot, and less than a millisecond to decide
Imagine an opponent who never blinks, never feels the pressure, and who learned to play table tennis not by watching tutorials, but by failing millions of times in a virtual simulator before ever facing a real champion. That is exactly what Ace is: an autonomous robot developed by researchers, whose results have just been published in the journal Nature. Ace is no sideshow gimmick: it competes against human players of competitive caliber — and holds its own.
But beyond the athletic feat, what makes Ace fascinating is the mathematical machinery running under the hood. To understand why this robot marks a breakthrough, we need to dive into a branch of artificial intelligence that bears an uncanny resemblance to the way we ourselves learn: reinforcement learning.
Learning like a child — only much faster
Reinforcement learning — known in French as apprentissage par renforcement — is a machine-learning method in which an agent — here, the robot — learns to act by interacting with its environment. There is no teacher demonstrating the right moves. There is simply a rule: some actions earn points (a reward), others cost points. The agent explores, makes mistakes, adjusts, and tries again — until it finds the strategy that maximizes its gains over the long run.
This is exactly how a child learns to walk: not by reading a biomechanics textbook, but by falling, getting back up, and gradually working out what works. The difference is that Ace could go through this learning process millions of times in simulation, at a speed no flesh-and-blood human could match.
The world as a sequence of decisions
For an algorithm to learn this way, it needs a mathematical framework. The one used by Ace — and by almost all modern reinforcement-learning systems — is called a Markov decision process, or MDP. The name sounds intimidating, but the idea is simple.
An MDP breaks the world down into a series of moments. At each moment, the agent observes the state of the world (where is the ball? how fast? in which direction?), chooses an action (how to angle the racket, how hard to hit), and receives in return a reward (did it return the ball onto the table?). The next state depends only on the present state and the chosen action — not on the entire history. That is the Markov property: the past doesn't matter, only the present counts for predicting the future.
"The agent doesn't need to remember every previous exchange. It only needs to know where things stand right now to make the best possible decision."
—
This simplification is both a strong assumption and the key that makes the problem mathematically tractable. In table tennis, the state of the world changes hundreds of times per second — hence the need for reaction times under a millisecond.
Ace's eyes: a camera that doesn't film
Before it can even decide what to do, Ace first has to see. And that's where it breaks new ground on the hardware side. Conventional cameras capture images at fixed intervals — 30, 60, sometimes 240 times per second. That's fine, but nowhere near enough to track a ping-pong ball fired at full speed.
Ace uses an event-based camera. Instead of photographing the whole scene at fixed intervals, this type of sensor only reports the pixels whose brightness changes — and it does so with a temporal resolution on the order of a microsecond (one millionth of a second). The result is an ultra-fast, low-noise stream of information, perfectly suited to fast-moving objects. It's a bit as if, instead of watching a film frame by frame, you perceived only what moves in the scene — and instantly.
This perception is then fused with the decision-making algorithms to produce a motor response in record time. Perception and action are no longer two separate steps: they form a continuous loop.
The robot's policy: learning to choose
At the heart of the system is what researchers call a policy (in French, politique) — a mathematical function that, given an observed state, indicates which action to take. The goal of reinforcement learning is precisely to optimize this policy — to find the one that maximizes the cumulative reward over an entire rally, or even an entire match.
To refine this policy, modern algorithms use techniques known as policy gradient methods — literally, the "gradient of the policy". The idea: work out in which direction to nudge the policy so that future rewards increase. It's a form of gradient descent, the same mathematical tool that lets neural networks learn to recognize faces or translate text — applied here not to static pixels, but to sequences of decisions in an unpredictable physical world.
The particular challenge of table tennis is that the environment is stochastic — meaning it contains an irreducible element of randomness. The ball can veer slightly depending on the spin applied, the opponent can vary their play, playing conditions fluctuate. The algorithm therefore has to learn a robust policy, capable of adapting in real time to these variations.
Facing a human: the moment of truth
Training a robot in simulation is one thing. Putting it up against real human players is another. The researchers had Ace play matches against players of competitive caliber. The results, published in Nature, show that Ace can sustain long rallies, adapt its tactics to its opponent's style of play, and place shots that are difficult to return.
This is not merely an athletic feat. It demonstrates that embodied artificial intelligence — AI that acts in the physical world, with a body, sensors, and real-time constraints — can reach a level of performance comparable to that of human experts in highly dynamic tasks. An important step toward robots capable of assisting humans in complex, unpredictable environments: surgery, logistics, home care.
Key takeaways
For math enthusiasts
A Markov decision process is formally defined by a quintuple (S, A, T, R, γ), where S is the state space (the position and velocity of the ball, the robot's posture…), A is the action space (the racket's possible movements), T : S × A × S → [0,1] is the transition function giving the probability of moving from a state s to a state s' after action a, R : S × A → ℝ is the reward function, and γ ∈ [0,1) is the discount factor that weights future rewards.
The goal is to find a policy π : S → A (or a distribution over A) that maximizes the expected cumulative discounted reward: E[Σ γᵗ R(sₜ, aₜ)]. To do this, algorithms of the policy gradient type compute the gradient of this expectation with respect to the parameters θ of the policy πθ, then update θ in the direction of this gradient. The fundamental policy gradient theorem (Sutton et al., 1999) establishes that this gradient equals: ∇θ J(θ) = E[∇θ log πθ(a|s) · Qπ(s,a)], where Qπ(s,a) is the action-value function — the expected cumulative reward starting from state s, choosing action a, and then following policy π. It is this formula that lets Ace adjust its strategy shot after shot, without ever needing a human supervisor.
