Feedback synthesis for underactuated systems using sequential second-order needle variations

Giorgos Mamakoukas, Malcolm A. MacIver, Todd D. Murphey

In the world of robotics, underactuated systems—machines with fewer control inputs than degrees of freedom—are everywhere. Think of a drone that can't move sideways without tilting, or a ship that can't slide laterally. Controlling these systems is notoriously difficult because the "obvious" path to a goal is often blocked by the laws of physics.

A recent paper by Mamakoukas et al., titled "Feedback Synthesis For Underactuated Systems Using Sequential Second-Order Needle Variations," introduces a powerful new way to navigate these constraints by looking past simple gradients and into the second-order geometry of the system.

TL;DR

The authors develop a nonlinear feedback control method that uses "second-order needle variations" to exploit the hidden ways underactuated systems can move. By mathematically linking optimal control theory to Lie brackets, they prove that their method can find control solutions even when standard first-order methods get stuck.

What Problem Does This Paper Address?

Most feedback controllers rely on the "gradient" of a cost function—essentially looking at which direction reduces the error most quickly right now. However, for underactuated systems, the gradient is often zero in the direction you actually need to go. For example, a car cannot move directly sideways; a first-order controller might simply give up because no immediate steering input results in sideways motion.

To move in these "forbidden" directions, you need to exploit nonlinear interactions, often described by Lie brackets (e.g., "steer left, move forward, steer right, move backward" to parallel park). Traditional ways to handle this, like Nonlinear Model Predictive Control (NMPC), are often too computationally heavy for real-time use, while other methods like Sliding Mode Control can cause damaging high-frequency "chattering" in motors.

Methods

The researchers utilize needle variations—infinitesimal, high-amplitude bursts of control. While previous work focused on first-order needle variations (the Mode Insertion Gradient), this paper derives the Mode Insertion Hessian (MIH), a second-order sensitivity measure.

The Key Innovation: The authors prove that the MIH explicitly contains Lie bracket terms. This bridges the gap between optimal control (minimizing costs) and geometric control (understanding reachability).

They packaged this into Algorithm 1, a real-time feedback loop that:

  1. Simulates the system's current trajectory and "co-states" (sensitivities) over a short horizon.
  2. Calculates an analytical, second-order control response.
  3. Uses a simple line search to determine exactly how long to apply that control "needle" to guarantee the cost goes down.

Datasets (Simulation Models)

As is standard in control theory research, the method was evaluated on complex physical simulations rather than static datasets:

  • 2D Kinematic Differential Drive: A "Roomba-style" robot used to test basic convergence and obstacle avoidance.
  • 3D Kinematic Rigid Body: A 6-DOF system (using quaternions) to test motion in orientations where first-order methods are singular.
  • 3D Dynamic Underwater Vehicle: A high-fidelity "fish" model with 13 states, including mass, inertia, and fluid drift, testing the algorithm's performance in unpredictable environments.

Key Insights and Ideas

  • The Lie Bracket Connection: The most significant "aha" moment is the proof that second-order variations "see" the nonlinear controllability of the system. If a system can be moved in a certain direction via a Lie bracket, the second-order controller is mathematically guaranteed to find a way to reduce the cost.
  • Provable Descent: Unlike many heuristics, this method has a formal guarantee: if the system is controllable, there always exists a second-order needle action that reduces the objective function.
  • Computational Efficiency: Because the control law is derived analytically (closed-form), it is incredibly fast. In tests, it ran roughly 40 times faster than iLQG (Iterative Linear Quadratic Gaussian), a popular but heavy optimization-based controller.

Key Results

  • 100% Success Rate: In Monte Carlo simulations of the 2D cart, the second-order method converged to the target in 1000/1000 trials, whereas first-order methods frequently failed to turn the vehicle toward the goal.
  • Overcoming Singularity: In the 3D "fish" simulation, first-order controllers were unable to displace the system in the $y$-direction (sideways). The second-order controller successfully navigated the system to the target by exploiting heave and roll maneuvers.
  • Obstacle Avoidance: The controller successfully navigated around static and moving obstacles simply by adding "penalty" terms to the cost function, without needing a separate, complex motion planner.

Limitations and Future Work

  • Control Discontinuity: Needle variations are, by definition, "pulses." Applying these directly to hardware could be hard on motors. The authors suggest using filters to smooth these pulses into "motor-friendly" signals.
  • Convexity Assumptions: The proof for global convergence currently assumes the cost function is convex in the unconstrained sense. Generalizing this to more complex, non-convex landscapes is a future goal.
  • Real-world Testing: While the simulations are high-fidelity, the authors are planning to move these algorithms onto physical underwater "fish-robots" to test performance against real-world sensor noise and water turbulence.

Summary updated

Abstract

This paper derives nonlinear feedback control synthesis for general control affine systems using second-order actions, the second-order needle variations of optimal control, as the basis for choosing each control response to the current state. A second result of this paper is that the method provably exploits the nonlinear controllability of a system by virtue of an explicit dependence of the second-order needle variation on the Lie bracket between vector fields. As a result, each control decision necessarily decreases the objective when the system is nonlinearly controllable using first-order Lie brackets. Simulation results using a differential drive cart, an underactuated kinematic vehicle in three dimensions, and an underactuated dynamic model of an underwater vehicle demonstrate that the method finds control solutions when the first-order analysis is singular. Finally, the underactuated dynamic underwater vehicle model demonstrates convergence even in the presence of a velocity field.
BibTeX
@article{mamakoukas2018feedback,
  author = {Giorgos Mamakoukas and Malcolm A. MacIver and Todd D. Murphey},
  title = {{Feedback synthesis for underactuated systems using sequential second-order needle variations}},
  journal = {The International Journal of Robotics Research},
  year = {2018},
  doi = {10.1177/0278364918776083},
}

Loading related content...

Figure