Cell-cell communication (CCC) occurs across different biological scales, ranging from interactions between large groups of cells to interactions between individual cells, forming a hierarchical structure. Globally, CCC may exist between clusters or only subgroups of a cluster with varying size, while locally, a group of cells as sender or receiver may exhibit distinct signaling properties. Current existing methods infer CCC from single-cell RNA-seq or Spatial Transcriptomics only between predefined cell groups, neglecting the existing hierarchical structure within CCC that are determined by signaling molecules, in particular, ligands and receptors. Here, we develop CrossChat, a novel computational framework designed to infer and analyze the hierarchical cell-cell communication structures using two complementary approaches: a global hierarchical structure using a multi-resolution clustering method, and multiple local hierarchical structures using a tree detection method. This framework provides a comprehensive approach to understand the hierarchical relationships within CCC that govern complex tissue functions. By applying our method to two nonspatial scRNA-seq datasets sampled from COVID-19 patients and mouse embryonic skin, and two spatial transcriptomics datasets generated from Stereo-seq of mouse embryo and 10x Visium of mouse wounded skin, we showcase CrossChat's functionalities for analyzing both global and local hierarchical structures within cell-cell communication.