Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

We describe a novel quadrotor Micro Air Vehicle (MAV) system that is designed to use computer vision algorithms within the flight control loop. The main contribution is a MAV system that is able to run both the vision-based flight control and stereo-vision-based obstacle detection parallelly on an embedded computer onboard the MAV. The system design features the integration of a powerful onboard computer and the synchronization of IMU-Vision measurements by hardware timestamping which allows tight integration of IMU measurements into the computer vision pipeline. We evaluate the accuracy of marker-based visual pose estimation for flight control and demonstrate marker-based autonomous flight including obstacle detection using stereo vision. We also show the benefits of our IMU-Vision synchronization for egomotion estimation in additional experiments where we use the synchronized measurements for pose estimation using the 2pt+gravity formulation of the PnP problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. www.pixhawk.ethz.ch.

  2. www.asctec.de.

  3. www.mikrokopter.de.

  4. www.microdrones.com.

  5. ardrone.parrot.com.

References

  • Achtelik, M., Achtelik, M., Weiss, S., & Siegwart, R. (2011). Onboard IMU and monocular vision based control for MAVs in unknown in- and outdoor environments. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 3056–3063).

    Chapter  Google Scholar 

  • Bachrach, A., de Winter, A., He, R., Hemann, G., Prentice, S., & Roy, N. (2010). Range—robust autonomous navigation in GPS-denied environments. In Robotics and automation (ICRA), 2010 IEEE international conference on (pp. 1096–1097). doi:10.1109/ROBOT.2010.5509990.

    Chapter  Google Scholar 

  • Bills, C., Chen, J., & Saxena, A. (2011). Autonomous MAV flight in indoor environments using single image perspective cues. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 5776–5783).

    Chapter  Google Scholar 

  • Blösch, M., Weiss, S., Scaramuzza, D., & Siegwart, R. (2010). Vision based MAV navigation in unknown and unstructured environments. In Robotics and automation (ICRA), 2010 IEEE international conference on (pp. 21–28). doi:10.1109/ROBOT.2010.5509920.

    Chapter  Google Scholar 

  • Bosch, S., Lacroix, S., & Caballero, F. (2006). Autonomous detection of safe landing areas for an UAV from monocular images. In Intelligent robots and systems, 2006 IEEE/RSJ international conference on (pp. 5522–5527). doi:10.1109/IROS.2006.282188.

    Chapter  Google Scholar 

  • Bouabdallah, S., Murrieri, P., & Siegwart, R. (2004). Design and control of an indoor micro quadrotor. In Proceedings of int. conf. on robotics and automation. URL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.89.842.

    Google Scholar 

  • Bouabdallah, S., & Siegwart, R. (2007). Full control of a quadrotor. In Intelligent robots and systems, 2007. IROS 2007. IEEE/RSJ international conference on (pp. 153–158). doi:10.1109/IROS.2007.4399042.

    Chapter  Google Scholar 

  • Conte, G., & Doherty, P. (2008). An integrated UAV navigation system based on aerial image matching. In Proceedings of the IEEE aerospace conference. URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.119.3963&rep=rep1&type=pdf.

    Google Scholar 

  • Dryanovski, I., Morris, W., & Xiao, J. (2011). An open-source pose estimation system for micro-air vehicles. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 4449–4454).

    Chapter  Google Scholar 

  • Ducard, G., & D’Andrea, R. (2009). Autonomous quadrotor flight using a vision system and accommodating frames misalignment. In Industrial embedded systems, 2009. SIES ’09. IEEE international symposium on (pp. 261–264). doi:10.1109/SIES.2009.5196224.

    Chapter  Google Scholar 

  • Eberli, D., Scaramuzza, D., Weiss, S., & Siegwart, R. (2011). Vision based position control for MAVs using one single circular landmark. Journal of Intelligent and Robotic Systems, 61(1–4), 495–512.

    Article  Google Scholar 

  • Fowers, S., Lee, D. J., Tippetts, B., Lillywhite, K., Dennis, A., & Archibald, J. (2007). Vision aided stabilization and the development of a quad-rotor micro UAV. In International symposium on computational intelligence in robotics and automation, 2007. CIRA 2007 (pp. 143–148). doi:10.1109/CIRA.2007.382886

    Chapter  Google Scholar 

  • Heng, L., Meier, L., Tanskanen, P., Fraundorfer, F., & Pollefeys, M. (2011). Autonomous obstacle avoidance and maneuvering on a vision-guided MAV using on-board processing. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 2472–2477).

    Chapter  Google Scholar 

  • Hofiann, G., Rajnarqan, D., & Waslander, S. (2004). The Stanford testbed of autonomous rotorcraft for multi agent control (starmac). In Proceedings of digital avionics systems conference (DASC04). URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1390847.

    Google Scholar 

  • Hrabar, S., & Sukhatme, G. (2009). Vision-based navigation through urban canyons. Journal of Field Robotics, 26(5), 431–452. doi:10.1002/rob.20284.

    Article  Google Scholar 

  • Huang, A., Olson, E., & Moore, D. (2010). LCM: Lightweight Communications and Marshalling. In Intelligent robots and systems (IROS), 2010 IEEE/RSJ international conference on (pp. 4057–4062).

    Chapter  Google Scholar 

  • Johnson, A., Montgomery, J., & Matthies, L. (2005). Vision guided landing of an autonomous helicopter in hazardous terrain. In Robotics and automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE international conference on (pp. 3966–3971). doi:10.1109/ROBOT.2005.1570727.

    Chapter  Google Scholar 

  • Kanade, T., Amidi, O., & Ke, Q. (2004). Real-time and 3d vision for autonomous small and micro air vehicles. In Decision and control, 2004. CDC. 43rd IEEE conference on (vol. 2, pp. 1655–1662). doi:10.1109/CDC.2004.1430282.

    Google Scholar 

  • Kemp, C. (2006). Visual control of a miniature quad-rotor helicopter. Ph.D. thesis, Churchill College, University of Cambridge.

  • Kukelova, Z., Bujnak, M., & Pajdla, T. (2010). Closed-form solutions to the minimal absolute pose problems with known vertical direction. In ACCV.

    Google Scholar 

  • Kukelova, Z., Bujnak, M., & Pajdla, T. (2011). Closed-form solutions to minimal absolute pose problems with known vertical direction. In Computer vision–ACCV 2010. URL http://www.springerlink.com/index/M012M78244081306.pdf.

    Google Scholar 

  • Lobo, J., & Dias, J. (2007). Relative pose calibration between visual and inertial sensors. International Journal of Robotics Research, 26(6), 561–575.

    Article  Google Scholar 

  • Meier, L., Tanskanen, P., Fraundorfer, F., & Pollefeys, M. (2011). Pixhawk: A system for autonomous flight using onboard computer vision. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 2992–2997).

    Chapter  Google Scholar 

  • Mellinger, D., & Kumar, V. (2011). Minimum snap trajectory generation and control for quadrotors. In Proceedings of the IEEE international conference on robotics and automation (ICRA).

    Google Scholar 

  • Mellinger, D., Shomin, M., Michael, N., & Kumar, V. (2010). Cooperative grasping and transport using multiple quadrotors. In Proceedings of the international symposium on distributed autonomous robotic systems.

    Google Scholar 

  • Montemerlo, M., Roy, N., & Thrun, S. (2003). Perspectives on standardization in mobile robot programming: the Carnegie Mellon navigation (Carmen) toolkit. In Intelligent robots and systems, 2003. (IROS 2003). Proceedings. 2003 IEEE/RSJ international conference on (vol. 3, pp. 2436–2441). doi:10.1109/IROS.2003.1249235.

    Chapter  Google Scholar 

  • Proctor, A. A., Johnson, E. N., & Apker, T. B. (2006). Vision-only control and guidance for aircraft. Journal of Field Robotics, 23(10), 863–890. doi:10.1002/rob.20155.

    Article  MATH  Google Scholar 

  • Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., & Ng, A. (2009). Ros: An open-source robot operating system.

    Google Scholar 

  • Roy, N., He, R., Bachrach, A., & Achtelik, M. (2009). On the design and use of a micro air vehicle to track and avoid adversaries. International Journal of Robotics Research. URL http://ijr.sagepub.com/cgi/content/abstract/29/5/529.

  • Saripalli, S., Montgomery, J., & Sukhatme, G. (2002). Vision-based autonomous landing of an unmanned aerial vehicle. In Robotics and automation, 2002. Proceedings. ICRA ’02. IEEE international conference on (vol. 3, pp. 2799–2804). doi:10.1109/ROBOT.2002.1013656.

    Google Scholar 

  • Scherer, S., Singh, S., Chamberlain, L., & Elgersma, M. (2008). Flying fast and low among obstacles: Methodology and experiments. The International Journal of Robotics Research, 27(5), 549–574. doi:10.1177/0278364908090949.

    Article  Google Scholar 

  • Shen, S., Michael, N., & Kumar, V. (2011). Autonomous multi-floor indoor navigation with a computationally constrained MAV. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 20–25).

    Chapter  Google Scholar 

  • Volpe, R., Nesnas, I., Estlin, T., Mutz, D., Petras, R., & Das, H. (2001). The CLARAty architecture for robotic autonomy. In Aerospace conference, 2001, IEEE proceedings (vol. 1, pp. 1/121–1/132). doi:10.1109/AERO.2001.931701.

    Chapter  Google Scholar 

  • Wagner, D., & Schmalstieg, D. (2007). Artoolkitplus for pose tracking on mobile devices. In Proceedings of 12th computer vision winter workshop. URL http://www.icg.tu-graz.ac.at/Members/daniel/ARToolKitPlusMobilePoseTracking.

    Google Scholar 

  • Li, W., Zhang, T., & Klihnlenz, K. (2011). A vision-guided autonomous quadrotor in an air-ground multi-robot system. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 2980–2985).

    Chapter  Google Scholar 

  • Wenzel, K., Masselli, A., & Zell, A. (2011). Automatic take off, tracking and landing of a miniature UAV on a moving carrier vehicle. Journal of Intelligent Robotic Systems, 61, 221–238. doi:10.1007/s10846-010-9473-0.

    Article  Google Scholar 

  • Williams, B., Hudson, N., Tweddle, B., Brockers, R., & Matthies, L. (2011). Feature and pose constrained visual aided inertial navigation for computationally constrained aerial vehicles. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 431–438).

    Chapter  Google Scholar 

Download references

Acknowledgements

We would like to thank our students (in alphabetical order) Bastian Bücheler, Andi Cortinovis, Christian Dobler, Dominik Honegger, Fabian Landau, Laurens Mackay, Tobias Nägeli, Philippe Petit, Martin Rutschmann, Amirehsan Sarabadani, Christian Schluchter and Oliver Scheuss for their contributions to the current system and the students of the previous semesters for the foundations they provided. Raffaello d’Andrea and Sergei Lupashin (ETH IDSC) provided valuable feedback.

This work was supported in part by the European Community’s Seventh Framework Programme (FP7/2007-2013) under grant #231855 (sFly) and by the Swiss National Science Foundation (SNF) under grant # 200021-125017.

Author information

Authors and Affiliations

  1. ETH Zurich, CAB G 86.3, Universitaetstr. 6, 8092, Zurich, Switzerland

    Lorenz Meier, Petri Tanskanen, Lionel Heng, Gim Hee Lee, Friedrich Fraundorfer & Marc Pollefeys

Authors
  1. Lorenz Meier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petri Tanskanen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lionel Heng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gim Hee Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Friedrich Fraundorfer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marc Pollefeys
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lorenz Meier.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

<Pixhawk_Alpha_Autonomous.avi: This video shows one of the first autonomous flights using computer vision localization with ARToolkit+ markers. It shows the general flight behaviour of our vision based system. The MAV is not thetered, images are fully processed online. The MAV flies several rounds, defined by pre-set waypoints.> (AVI 11.3 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Meier, L., Tanskanen, P., Heng, L. et al. PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision. Auton Robot 33, 21–39 (2012). https://doi.org/10.1007/s10514-012-9281-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-012-9281-4

Keywords

Search

Navigation