3D CNNs vs two-stream action recognition

WHAT THIS TESTS Whether you understand how each architecture injects temporal information and can weigh accuracy against computational and preprocessing cost.

A GOOD ANSWER COVERS Action recognition needs both appearance and motion. 3D Convolutional Networks like C3D and I3D extend convolution to the time axis, so filters span height, width, and several frames, learning spatiotemporal features end to end directly from RGB clips. Advantages: a single network learns motion jointly with appearance, no hand-crafted preprocessing. Disadvantages: 3D kernels have far more parameters, are data hungry, slow, and memory intensive. Two-Stream Networks instead use two parallel 2D CNNs: a spatial stream on single RGB frames captures appearance, and a temporal stream on a stack of precomputed optical flow fields captures motion; their predictions are fused. Advantages: explicit motion via flow gives strong accuracy and reuses 2D pretraining. Disadvantages: optical flow must be computed offline, which is expensive and storage heavy, and the two streams add complexity. I3D bridges both, inflating ImageNet-pretrained 2D filters into 3D and even adding a flow stream.

COMMON WRONG ANSWERS Ignoring the heavy offline cost of computing optical flow for the two-stream approach. Claiming 3D CNNs are always cheaper. Saying two-stream learns motion end to end, when its flow is precomputed.

LIKELY FOLLOW-UPS How does I3D's inflation transfer 2D pretraining? Why is optical flow so costly? What do efficient variants like R(2+1)D or SlowFast change? How does temporal receptive field affect long actions?

ONE CONCRETE EXAMPLE Classifying a tennis swing, a 3D CNN ingests a 16-frame RGB clip and learns the racket motion in its kernels. A two-stream model feeds one RGB frame to the spatial stream and a stack of optical flow frames of the arm swing to the temporal stream, then averages, but the flow had to be computed first.