Abstract
We propose a trajectory-based optimal control method for periodic tasks for systems with discontinuous dynamics. A general method, dynamic programming, suffers from the problem of dimensionality. We use local models of the optimal control law to construct a local controller. We combine a parametric trajectory optimization method and differential dynamic programming (DDP) to find the optimal periodic trajectory in a periodic task. By formulating the optimal control problem with an infinite time horizon, DDP generates time-invariant local models of the optimal control law. For DDP, the value function at dynamics discontinuities is approximated by a second order Taylor series. The utility of the proposed method is evaluated using simulated walking control of a five-link biped robot. The results show lower torques and more robustness from the proposed controller than a PD servo controller.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bellman R. Dynamic Programming. Princeton, NJ: Princeton Univ. Press, 1957
Larson R E. State Increment Dynamic Programming. New York: American Elsevier Pub. Co., 1968
Dyer P, McReynolds S R. The Computation and Theory of Optimal Control. San Diego, CA: Academic Press, 1970
Jacobson D H, Mayne D Q. Differential dynamic programming. In: Modern Analytic and Computational Methods in Science and Mathematics, No. 24. Elsevier, 1970
Betts J T. Practical Methods for Optimal Control Using Nonlinear Programming. Philadelphia, PA: Society for Industrial and Applied Mathematics, 2001
Hargraves C R, Paris S W. Direct trajectory optimization using nonlinear programming and collocation. J Guid Control Dyn, 1987, 10: 338–342
Gill P E, Murray W, Saunders M A. SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM J Opt, 1997, 12: 979–1006
Stolle M, Atkeson C G. Policies based on trajectory libraries. In: Proc. Int. Conf. Robot. Autom., Orlando, FL, USA, 2006. 3344–3349
Tassa Y, Erez T, Smart W. Receding horizon differential dynamic programming. In: Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2008, 20: 1465–1472
Grossman R L, Valsamis D, Qin X. Persistent stores and hybrid systems. In: Proc. Int. Conf. Decision and Control, New York, NY, USA, 1993. 2298–2302
Schierman J, Ward D, Hull J, et al. Integrated adaptive guidance and control for re-entry vehicles with flight test results. J Guid Control Dyn, 2004, 27: 975–988
Frazzoli E, Dahleh M, Feron E. Maneuver-based motion planning for nonlinear systems with symmetries. IEEE Trans Robot, 2005, 21: 1077–1091
Stolle M, Tappeiner H, Chestnutt J, et al. Transfer of policies based on trajectory libraries. In: Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst, New York, NY, USA, 2007. 2981–2986
Safonova A, Hodgins J K. Construction and optimal search of interpolated motion graphs. ACM Trans Graph, 2007, 26: 106
Tedrake R, Manchester I R, Tobenkin M, et al. LQR-trees: Feedback motion planning via sums-of-squares Verification. Int J Rob Res, 2010, 29: 1038–1052
Atkeson C G. Using local trajectory optimizers to speed up global optimization in dynamic programming. In: Cowan J D, Tesauro G, Alspector J, eds. Advances in Neural Information Processing Systems. San Fransisco, CA: Morgan Kaufmann Publishers, Inc., 1994, 6: 663–670
Atkeson C G, Morimoto J. Nonparametric representation of policies and value functions: a trajectory-based approach. In: Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2003. 1643–1650
Vukobratović M, Borovac B, Surla D, et al. Biped locomotion: dynamics, stability, control and application. In: Scientific Fundamentals of Robotics. Berlin: Springer, 1990
Hirai K, Hirose M, Haikawa Y, et al. The development of Honda humanoid robot. In: Proc Int Conf Robot Autom, Leuven, Belgium, 1998, 2: 1321–1326
Huang Q, Yokoi K, Kajita S, et al. Planning walking patterns for a biped robot. IEEE Trans Robot Autom, 2001, 17: 280–289
Kajita S, Tani K. Experimental study of biped dynamic walking. IEEE Control Syst Mag, 1996, 16: 13–19
Djoudi D, Chevallereau C, Aoustin Y. Optimal reference motions for walking of a biped robot. In: Proc. Int. Conf. Robot. Autom., New York, NY, USA, 2005. 2002–2007
Bessonnet G, Seguin P, Sardain P. A parametric optimization approach to walking pattern synthesis. Int J Rob Res, 2005, 24: 523–536
Chevallereau C, Djoudi D, Grizzle J. Stable bipedal walking with foot rotation through direct regulation of the zero moment point. IEEE Trans Robot, 2008, 24: 390–401
Loffler K, Gienger M, Pfeiffer F, et al. Sensors and control concept of a biped robot. IEEE Trans Indust Electr, 2004, 51: 972–980
Westervelt E R, Grizzle J W, Koditschek D E. Hybrid zero dynamics of planar biped walkers. IEEE Trans Autom Contr, 2003, 48: 42–56
Raibert M H. Legged robots. Commun ACM, 1986, 29: 499–514
Pratt J, Pratt G. Intuitive control of a planar bipedal walking robot. In: Proc. IEEE Int. Conf. Robot. Autom., New York, NY, USA, 1998. 2014–2021
Dimitrov D, Ferreau H J, Wieber P B, et al. On the implementation of model predictive control for on-line walking pattern generation. In: Proc. IEEE Int. Conf. Robot. Autom., Pasadena, CA, USA, 2008. 2685–2690
Whitman E, Atkeson C G. Control of a walking biped using a combination of simple policies. In: Proc. Int. Conf. Humanoid Robots, Paris, 2009. 520-527
Moore M, Wilhelms J. Collision detection and response for computer animation. Comput Graph, 1988, 22: 289–298
Liu C G, Atkeson C G. Standing balance control using a trajectory library. In: Proc. Int. Conf. Intell. Robots Syst., St. Louis, 2009. 3031–3036
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, C., Atkeson, C.G. & Su, J. Neighboring optimal control for periodic tasks for systems with discontinuous dynamics. Sci. China Inf. Sci. 54, 653–663 (2011). https://doi.org/10.1007/s11432-011-4185-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11432-011-4185-z