Feature Enhancement and Foreground-Background Separation for Weakly Supervised Temporal Action Localization

Weakly-supervised Temporal Action Localization (W-TAL) is a significant task in video understanding, intending to recognize the category and pinpoint the temporal boundaries of action segments in untrimmed videos based on the video-level labels. Due to the lack of frame-level annotations, recognizing the spatiotemporal relationships among action snippets and precisely separating the foreground and background are two arduous challenges. In this paper, we propose a novel Feature Enhancement and Foreground-Background Separation (FE-FBS) method to address these problems. Specifically, we construct a spatiotemporal cooperation enhancement scheme utilizing residual graph convolutional to capture the spatial and temporal dependencies between the current snippet and others, generating snippet features with spatial and temporal correlations, thereby ensuring a complete feature representation for action localization. Furthermore, we propose a two-branch explicit foreground-background joint attention mechanism to guide foreground and background modeling, combined with an inverse enhancement strategy to enhance action weight for better foreground and background distinction, thereby improving the accuracy of action localization. We use THUMOS’14 and ActivityNet v1.3 datasets to achieve accuracy rates of 66.7% and 38.9%, respectively, and compare our method to other methods, gaining superiority.