CPU vs GPU vs Edge TPU for inference.
inference hardware trade-offs.
CPU is flexible but slow, GPU offers massive parallelism at high power, Edge TPU gives efficient low-power int8 inference but is constrained; choose by latency, power, cost, and model fit.
WHAT THIS TESTS The interviewer wants you to reason about compute hardware along throughput, power, cost, and flexibility axes, then apply that to a concrete edge deployment.
A GOOD ANSWER COVERS A CPU is general-purpose and flexible, runs any model and any operator, but has limited parallelism, so deep networks are slow and inefficient for heavy vision workloads. A GPU has thousands of cores giving massive parallelism ideal for large CNNs and high throughput, but it draws significant power, generates heat, and costs more, which matters at the edge. A specialized accelerator like the Coral Edge TPU or an NPU is designed specifically for neural inference, delivering high performance per watt; the trade-off is that it typically requires int8 quantization, supports only a subset of operators, and constrains model architecture, so models must be compiled and may need adaptation. For a smart security camera the deciding factors are latency, since detection should be real time; power and thermal budget, since the device may be small or battery or PoE powered; cost at scale; whether processing is on-device for privacy and offline operation or offloaded to the cloud; and whether your model can be quantized and compiled for the accelerator.
COMMON WRONG ANSWERS Saying GPU is always best, ignoring power and cost. Forgetting the Edge TPU's quantization and limited-op constraints. Overlooking the cloud-versus-edge decision and its privacy and connectivity implications.
LIKELY FOLLOW-UPS Latency versus throughput, batching at the edge, quantization-aware training for the TPU, and hybrid edge-plus-cloud pipelines.
ONE CONCRETE EXAMPLE For a wall-powered indoor security camera doing on-device person detection, a Coral Edge TPU running an int8 MobileNet-SSD gives real-time inference at low power and keeps video local for privacy, whereas a full GPU would be overkill on power and cost and a bare CPU would miss the frame rate.
Read the original → aegisai.in
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.