The processing of color image data using directional information is studied. The class of vector directional filters (VDF), which was introduced by the authors in a previous work, is further considered. The analogy of VDF to the spherical median is shown, and their relation to the spatial median is examined. Moreover, their statistical and deterministic properties are studied, which demonstrate their appropriateness in image processing. VDF result in optimal estimates of the image vectors in the directional sense; this is very important in the case of color images, where the vectors' direction signifies the chromaticity of a given color. Issues regarding the practical implementation of VDF are also considered. In addition, efficient filtering schemes based on VDF are proposed, which include adaptive and/or double-window structures. Experimental and comparative results in image filtering show very good performance measures when the error is measured in the L*a*b* space. L*a*b* is known as a space where equal color differences result in equal distances, and therefore, it is very close to the human perception of colors. Moreover, an indication of the chromaticity error is obtained by measuring the error on the Maxwell triangle; the results demonstrate that VDF are very accurate chromaticity estimators.