This paper describes a method for 4D imaging, which is used to study respiratory organ motion, a key problem in various treatments. Whilst the commonly used imaging methods rely on simplified breathing patterns to acquire one breathing cycle, the proposed method was developed to study irregularities in organ motion during free breathing over tens of minutes. The method does not assume a constant breathing depth or even strict periodicity and does not depend on an external respiratory signal. Time-resolved 3D image sequences were reconstructed by retrospective stacking of dynamic 2D images using internal image-based sorting. The generic method is demonstrated for the liver and for the lung. Quantitative evaluations of the volume consistency show the advantages over one-dimensional measurements for image sorting. Dense deformation fields describing the respiratory motion were estimated from the reconstructed volumes using non-rigid 3D registration. All obtained motion fields showed variations in the range of minutes such as drifts and deformations, which changed both the exhalation position of the liver and the breathing pattern. The obtained motion data are used in proton therapy planning to evaluate dose delivery methodologies with respect to their motion sensitivity. Besides this application, the new possibilities of studying respiratory motion are valuable for other applications such as the evaluation of gating techniques with respect to residual motion.