pandas .apply() versus vectorized operations

WHAT THIS TESTS This checks whether you understand the performance model of pandas: that operations pushed down to vectorized, C-implemented routines vastly outperform calling a Python function repeatedly, and whether you know when each is appropriate.

A GOOD ANSWER COVERS The .apply() method runs a user-supplied Python function across an axis of a DataFrame, once per row or per column, or per element for a Series. Its strength is flexibility: you can express arbitrary custom logic that has no built-in equivalent. Its weakness is performance. Under the hood .apply() is essentially an interpreted Python loop, so the Python-level function-call overhead is paid for every row, and it cannot exploit the columnar, C-level optimizations that vectorized operations use. Vectorized operations, such as arithmetic on whole columns, boolean comparisons, string accessors, or np.where, operate on entire arrays at once in compiled code, often one to two orders of magnitude faster. The rule of thumb is to reach for vectorized expressions first and treat .apply() as a fallback. Note that .apply() with axis=1 over rows is especially slow; df.apply over columns or built-in aggregations are faster, and itertuples beats iterrows when iteration is truly needed.

COMMON WRONG ANSWERS Believing .apply() is itself vectorized or fast. Saying it should always be avoided, when sometimes there is no vectorized alternative. Confusing applymap, apply, and map. Recommending iterrows, which is even slower.

LIKELY FOLLOW-UPS How do you vectorize a conditional that you wrote with apply? When is a lookup table or merge better than apply? What does the GIL have to do with this?

ONE CONCRETE EXAMPLE Necessary case: you must call an external geocoding library on each address string; there is no array operation for that, so df['addr'].apply(geocode) is the right tool. Avoid case: to compute a discounted price you might write df.apply(lambda r: r.price * 0.9, axis=1), but the vectorized df['price'] * 0.9 produces the identical result far faster because it runs in compiled code over the whole column rather than looping in Python.