High-energy (TeV-PeV) and ultra-high-energy (> 100 PeV) neutrinos of cosmic origin have a vast potential to probe neutrino physics in regimes of energy and distance otherwise unreachable. Notable examples include precision tests using the flavor composition of the neutrino flux, new neutrino-neutrino interactions, and measuring the neutrino-nucleon cross section. Today, we are already tapping into this potential at high energies, thanks to spectacular discoveries by the IceCube neutrino telescope. In the coming decade, we may extend our reach to ultra-high energies, thanks to an ambitious experimental program under planning. I will survey the rich landscape of high-energy physics with cosmic neutrinos, from the perspectives of theory and experiment. By means of illustration, I will campaign for the notion that in order for the field to mature, we must embrace the natural synergy between low-energy and high-energy neutrino experiments, and adopt an earnest, comprehensive treatment of the relevant particle-physics and astrophysical unknowns. If we do, then our progress, while hard-earned, could lead to transformative insight.