Object detection in millimeter-wave Interferometric Synthetic Aperture Radiometer (InSAR) imaging is always a crucial task. Facing unpredictable and numerous objects, traditional object detection models running after the InSAR system accomplishing imaging suffer from disadvantages such as complex clutter backgrounds, weak intensity of objects, Gibbs ringing, which makes a general purpose saliency detection system for InSAR necessary. This letter proposes a spectrum-based saliency detection algorithm to extract the salient regions from unknown backgrounds cooperating with sparse sensing InSAR imaging procedure. Directly using the interferometric value and sparse information of scenes in the basis of the Discrete Cosine Transform (DCT) domain adopted by InSAR imaging procedure, the proposed algorithm isolates the support of saliency region and then inversely transforms it back to calculate the saliency map. Comparing with other detecting algorithms which run after accomplishing imaging, the proposed algorithm will not be affected by information-loss accused by imaging procedure. Experimental results prove that it is effective and adaptable for millimeter-wave InSAR imaging.