A Real-Time Scene Recognition System Based on RGB-D Video Streams

Depth data captured by the cameras such as Microsoft Kinect can bring depth information than traditional RGB data, which is also more robust to different environments, such as dim or dark lighting conditions. In this technical demonstration, we build a scene recognition system based on real-time processing of RGB-D video streams. Our system recognizes the scenes with video clips, where three types of threads are implemented to ensure the realtime. This system first buffers the frames of both RGB and depth videos with the capturing threads. When the buffered videos reach the certain length, the frames will be packed into clips and forwarded in a pre-trained C3D model to predict scene labels with the scene recognition thread. Finally, the predicted scene labels and captured videos are illustrated in our user interface with illustration thread.