CPU vs GPU vs Edge TPU for inference.
inference hardware trade-offs.
CPU is flexible but slow, GPU offers massive parallelism at high power, Edge TPU gives efficient low-power int8 inference but is constrained; choose by latency, power, cost, and model fit.
What's really being asked
The interviewer wants you to reason about compute hardware along throughput, power, cost, and flexibility axes, then apply that to a concrete edge deployment.
The full answer
A CPU is general-purpose and flexible, runs any model and any operator, but has limited parallelism, so deep networks are slow and inefficient for heavy vision workloads. A GPU has thousands of cores giving massive parallelism ideal for large CNNs and high throughput, but it draws significant power, generates heat, and costs more, which matters at the edge. A specialized accelerator like the Coral Edge TPU or an NPU is designed specifically for neural inference, delivering high performance per watt; the trade-off is that it typically requires int8 quantization, supports only a subset of operators, and constrains model architecture, so models must be compiled and may need adaptation. For a smart security camera the deciding factors are latency, since detection should be real time; power and thermal budget, since the device may be small or battery or PoE powered; cost at scale; whether processing is on-device for privacy and offline operation or offloaded to the cloud; and whether your model can be quantized and compiled for the accelerator.
The mistakes people make
Saying GPU is always best, ignoring power and cost. Forgetting the Edge TPU's quantization and limited-op constraints. Overlooking the cloud-versus-edge decision and its privacy and connectivity implications.
What usually comes next
Latency versus throughput, batching at the edge, quantization-aware training for the TPU, and hybrid edge-plus-cloud pipelines.
A concrete example
For a wall-powered indoor security camera doing on-device person detection, a Coral Edge TPU running an int8 MobileNet-SSD gives real-time inference at low power and keeps video local for privacy, whereas a full GPU would be overkill on power and cost and a bare CPU would miss the frame rate.
Interview question
What is a key constraint of deploying a model on a Google Coral Edge TPU compared to a GPU?
- a.It cannot run any convolutional layers
- b.It only runs models written in a proprietary language
- c.It draws far more power than a GPU
- d.It typically requires int8 quantization and supports only a subset of operatorsCorrect
Why? this is the answer
Edge TPUs are efficient but require quantized int8 models and support limited ops, so models must be compiled and may need adaptation. They are low-power, not high-power, and do run convolutions.
Just read this? Test yourself on what you have been reading.
Read the original → aegisai.in
- #edge-ai
- #inference-hardware
- #tpu
- #gpu
- #deployment
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles