Oxide growth with semiconductorlike accuracy has led to atomically precise thin films and interfaces that exhibit a plethora of phases and functionalities not found in the oxide bulk material. This has yielded spectacular discoveries such as the conducting, magnetic, and even superconducting interfaces separating two prototypical insulating perovskite materials. All these investigations, however, consider the static state at the interface, although studies on fast oxide interface dynamics would introduce a powerful degree of freedom to understanding the nature of the interface state. Here, we show that the polarization state at the interface can be optically enhanced or attenuated within picoseconds. Our observations are explained by a model based on charge propagation effects in the interfacial vicinity and transient polarization buildup at the interface.