Apply a CIFilter to a live camera feed

WHAT THIS TESTS This probes the live-capture pipeline in AVFoundation and the critical real-time concern: the delegate queue must keep up or frames back up and the session stalls.

A GOOD ANSWER COVERS Create an AVCaptureSession, add an AVCaptureDeviceInput wrapping the camera AVCaptureDevice, and add an AVCaptureVideoDataOutput for raw frames. On the output, call setSampleBufferDelegate(_:queue:) passing your object and a dedicated serial DispatchQueue; the queue must not be the main queue because per-frame work there would freeze the UI and the capture would stall. Frames arrive in the delegate method captureOutput(_:didOutput:from:) as CMSampleBuffer objects. You pull the CVPixelBuffer, wrap it in a CIImage, apply your CIFilter, and render the output, often with a CIContext into a Metal view or back into a pixel buffer. Set videoSettings to a pixel format the GPU likes, like BGRA. Start with session.startRunning on a background queue.

COMMON WRONG ANSWERS Setting the sample buffer delegate queue to DispatchQueue.main, which causes dropped frames and UI freezes. Doing slow CPU work synchronously in the callback so buffers accumulate. Not enabling alwaysDiscardsLateVideoFrames, letting the pipeline fall behind. Retaining CMSampleBuffers too long, exhausting the limited buffer pool and pausing capture.

LIKELY FOLLOW-UPS Why alwaysDiscardsLateVideoFrames true keeps latency low by dropping frames you cannot process in time. Using a Metal-backed CIContext for GPU rendering. How to handle orientation. Configuring the session preset for resolution. Why you must not block the callback waiting on the GPU.

ONE CONCRETE EXAMPLE You configure let output = AVCaptureVideoDataOutput(), set output.alwaysDiscardsLateVideoFrames = true and output.videoSettings for BGRA, then output.setSampleBufferDelegate(self, queue: DispatchQueue(label: "camera.processing")). In captureOutput you do guard let pb = CMSampleBufferGetImageBuffer(sampleBuffer), make a CIImage(cvPixelBuffer: pb), apply filter.setValue and read outputImage, and render it with a shared Metal CIContext into your preview. Because processing runs on the serial background queue with late frames discarded, the filtered feed stays smooth and the UI never blocks.