Python Programming Questions

Difference & when to prefer

• List comprehension: produces a list eagerly; concise and fast for moderate-sized results when you need random access or repeated iteration.
• Generator expression: lazy iteration, low memory; prefer for large streams or pipeline processing.
• map/filter: functional style; can be slightly faster in some cases and composes well with other functions or builtins; returns iterator in Py3.

Examples (square 0..9):
List comprehension:

squares = [x*x for x in range(10)]
print(squares)
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Generator expression:

squares_gen = (x*x for x in range(10))
print(list(squares_gen))
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

map/filter:

squares_map = list(map(lambda x: x*x, range(10)))
print(squares_map)
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

All three print the identical list, confirming the three forms are interchangeable here; only their evaluation style (eager list, lazy generator, eager map/filter) differs.

Approach

I’d normalize by lowercasing, removing punctuation, and collapsing repeated spaces. Then I’d use a set of normalized keys to keep only the first occurrence. A set is a data structure that gives fast membership checks, so this stays simple and efficient for a few thousand strings.

import re
from typing import List


def normalize_title(title: str) -> str:
    cleaned = re.sub(r'[\W_]+', ' ', title.lower())
    return ' '.join(cleaned.split())


def dedupe_titles(titles: List[str]) -> List[str]:
    seen = set()
    result = []

    for title in titles:
        norm = normalize_title(title)
        if norm in seen:
            continue
        seen.add(norm)
        result.append(norm)

    return result


titles = ['  Weekly Update!', 'weekly update', 'Budget Review', 'Budget review  ']
print(dedupe_titles(titles))

The regex pattern r'[\W_]+' is what actually strips punctuation: \W (capital W) matches any character that is NOT a letter, digit, or underscore, so it catches punctuation, symbols, and stray whitespace runs in one go; adding _ to the character class folds underscores into that same "replace with a space" rule too, since \W alone would NOT match an underscore (underscores count as word characters). The + means one-or-more, so any run of these unwanted characters collapses to a single space rather than leaving multiple spaces behind, and ' '.join(cleaned.split()) then trims leading/trailing spaces and collapses any remaining internal runs down to exactly one space each.

Example

Input: [' Weekly Update!', 'weekly update', 'Budget Review', 'Budget review ']

Output:

['weekly update', 'budget review']

That is the literal stdout of print(dedupe_titles(titles)) from the code block above.

Why this works: the first weekly update is kept, and the later duplicate is skipped because it normalizes to the same key.

Complexity: O(n * m) time, where m is string length, and O(n) extra space for the set.

If the team wants to keep original casing, I would store the first original string in result instead of the normalized version.

Direct answer

list.append(x) is $O(1)$ amortized; list.pop() (no argument, removing the last element) is $O(1)$; list.pop(0) and list.insert(0, x) are both $O(n)$, because removing or inserting at the front requires shifting every remaining element by one position; list.index(x) is $O(n)$, a linear scan from the start looking for a match. The two snippets below each hide an $O(n)$ operation inside a loop that runs $n$ times, so each function is $O(n^2)$ overall, not the $O(n)$ it might look like at a glance.

Structured elaboration

Why each complexity is what it is: Python's list is a dynamic array, backed by a contiguous block of memory holding references to the elements, plus some spare, over-allocated capacity at the end.

• append(x): writes into the spare capacity at the end and bumps a length counter; no shifting needed. When the spare capacity runs out, CPython reallocates a larger block (roughly 1.125x growth) and copies every existing element once, but this only happens occasionally, so the cost of those occasional $O(n)$ copies, averaged (amortized) over all the append calls, works out to $O(1)$ per call.
• pop(): removes the last element and decrements the length counter; nothing else moves, so it's a true $O(1)$, not just amortized.
• pop(0): removes the first element, then every one of the remaining $n-1$ elements has to shift left by one slot to close the gap, which is $O(n)$.
• insert(0, x): the mirror image; every existing element has to shift right by one slot to make room at the front, before x is written into the now-empty first slot, which is $O(n)$.
• index(x): there is no auxiliary structure telling you where a value lives, so the only way to find it is to check elements one at a time from the front until a match is found (or the end is reached), which is $O(n)$ in the worst case (element near the end, or absent).

Reading the two snippets:

def build_prefix_list(n):
    result = []
    for i in range(n):
        result.insert(0, i)     # O(n) shift, called n times
    return result

def contains_any(items, targets):
    hits = []
    for t in targets:
        if t in items:          # O(len(items)) scan, called len(targets) times
            hits.append(t)
    return hits

build_prefix_list calls insert(0, i) inside a loop that runs $n$ times. Each call's cost grows with however many elements are already in the list (0, then 1, then 2, ..., up to $n-1$), so the total work is $0 + 1 + 2 + \cdots + (n-1) = O(n^2)$, not $O(n)$.

contains_any does t in items inside a loop over targets; in on a list is a linear scan, $O(\text{len(items)})$ per check. If len(items) is roughly $n$ and len(targets) is also roughly $n$, the total work is $O(n) \cdot O(n) = O(n^2)$.

Worked example

print(build_prefix_list(5))                       # [4, 3, 2, 1, 0]
print(contains_any([1, 2, 3, 4, 5], [3, 9, 5]))     # [3, 5]

Fixed versions, same output, each $O(n)$ overall:

def build_prefix_list_fixed(n):
    result = list(range(n))   # build in natural order, O(n) total
    result.reverse()           # O(n) once, not O(n) per element inserted
    return result

def contains_any_fixed(items, targets):
    item_set = set(items)      # O(len(items)) once, upfront
    return [t for t in targets if t in item_set]   # O(1) average per membership check

print(build_prefix_list_fixed(5))                        # [4, 3, 2, 1, 0]
print(contains_any_fixed([1, 2, 3, 4, 5], [3, 9, 5]))      # [3, 5]

build_prefix_list_fixed builds the list in its natural (ascending) order with list(range(n)), which is $O(n)$, then reverses the whole list once with .reverse(), itself $O(n)$; total $O(n)$ instead of $O(n^2)$. contains_any_fixed pays the cost of building a set from items once ($O(n)$), then each membership check against that set is $O(1)$ average, for $O(n)$ total instead of $O(n^2)$.

Trade-offs & pitfalls

• The fix for "repeated insert(0, ...)" is almost always "build forwards, then reverse once" or, if you need frequent insertion and removal from both ends, switch the data structure entirely to collections.deque, which supports $O(1)$ append and pop from either end (at the cost of $O(n)$ random-access indexing, which list gives you for free).
• The fix for "repeated in against a list inside a loop" is almost always "build a set (or dict) once, outside the loop, and check membership against that instead." This is one of the single most common accidental-$O(n^2)$ patterns in data-processing code, and it is easy to miss because each individual line looks innocent.
• list.index(x) has the identical shape of risk: calling it repeatedly inside a loop over the same list is $O(n^2)$; if you need repeated lookups by value, build a dict mapping value to index once, upfront.
• Amortized $O(1)$ for append is a statement about the average cost across many calls, not a guarantee about any single call; an individual append that triggers a resize does real $O(n)$ work at that moment. This rarely matters in practice but is worth knowing when reasoning about worst-case latency for a single operation rather than total throughput.

Why vectorized ops are faster:

• Vectorized NumPy/Pandas operations run in optimized C/Fortran loops avoiding Python per-element overhead.
• They leverage contiguous memory, CPU cache, SIMD (Single Instruction, Multiple Data: a CPU feature that applies one instruction to several values in a single step, instead of one value at a time), and multi-threaded BLAS (Basic Linear Algebra Subprogram, a standard library of fast, hardware-tuned matrix/vector math routines that numpy calls into) for numeric work.
• Example: adding two arrays uses a single C loop vs Python loop calling millions of Python operations.

Worked example, verified on CPython 3.12 (showing what actually gets computed, not a timing benchmark: exact speed depends on hardware and array size, so run time.perf_counter() yourself to see the real gap on your own machine rather than trusting a hardcoded number here):

import numpy as np

a = np.array([1.0, 2.0, 3.0, 4.0])
b = np.array([10.0, 20.0, 30.0, 40.0])

def add_loop(a, b):
    return [x + y for x, y in zip(a, b)]

def add_vectorized(a, b):
    return a + b

print(add_loop(a, b))
# [11.0, 22.0, 33.0, 44.0]
print(add_vectorized(a, b))
# [11. 22. 33. 44.]

Both compute the identical values; add_loop does it via four separate Python-level additions, each one dispatched, type-checked, and reference-counted by the interpreter, while add_vectorized's a + b dispatches once into a single compiled C loop over the two contiguous memory buffers, which is the mechanism, not a specific timing number, that makes the vectorized version pull ahead as array size grows.

When vectorization can be slower:

• Small arrays: overhead of creating intermediate arrays and function call overhead can dominate; a simple Python loop or in-place updates may be faster.
• Complex element-wise logic with branching: vectorization may require multiple large temporaries or complex masking, increasing memory bandwidth and runtime.

Where vectorization stops winning, made concrete: a piecewise function like "if x < 0, return 0; elif x < 10, return x; else return x**2" vectorizes via np.where, but every branch has to be computed for every element before the mask selects which result to keep:

x = np.array([-3.0, 5.0, 15.0])
result = np.where(x < 0, 0.0, np.where(x < 10, x, x ** 2))
print(result)
# [  0.   5. 225.]

Even though only one branch is "true" for any given element, x ** 2 is computed for all three elements (including -3.0 and 5.0, whose squared values are simply discarded by the outer np.where), and each np.where call allocates a full temporary array the size of x. For a genuinely small array or a function with many branches, that wasted work and temporary-array allocation can cost more than a plain Python loop that only evaluates the one branch each element actually needs; this is the concrete shape of "vectorization can be slower."

Example: computing a piecewise function with many branches can be slower vectorized due to multiple masks and temporary arrays; using numba (a library that compiles a plain Python function to machine code the first time it runs, letting a genuinely branchy per-element loop run at near-C speed without vectorizing it at all) for a compiled loop may be faster and more memory-efficient.

Direct answer

Define a custom exception when a caller needs to programmatically distinguish and handle a specific failure mode; reuse a built-in (ValueError, TypeError, KeyError) when the failure is a generic, well-understood violation with no library-specific handling to offer. Keep except clauses as narrow as the exception you can actually recover from, never bare except:. Use raise NewError(...) from original whenever you translate a low-level exception into a library-level one, so the original traceback and type are preserved for debugging instead of discarded.

Structured elaboration

When to define a custom exception type

• Define one when a caller might reasonably want to catch this specific failure and do something different for it than for other failures (retry, fall back, surface a specific user-facing message). If every caller would handle it the same way as a generic ValueError, a custom type adds ceremony without adding value.
• Root the hierarchy at a single package-level base (for example MyLibError(Exception)), with specific failures subclassing it (ModelLoadError(MyLibError), InvalidDatasetError(MyLibError)). This lets a caller catch the single base class to mean "anything this library can go wrong in" without having to enumerate every subclass, while still allowing narrower catches where useful.
• Name exceptions for what failed semantically, not for the internal mechanism that detected it; a caller should be able to catch ModelLoadError without knowing or caring whether the implementation currently reads checkpoints from disk, S3, or a database.

How narrow an except clause should be

• Catch the most specific exception type you can actually do something about. A library function should generally let exceptions it cannot meaningfully handle propagate (or wrap them in its own type), rather than catching broadly and hiding the failure.
• Bare except: (or except Exception: used as a catch-all) at the library level is almost always wrong: it catches things like KeyboardInterrupt-adjacent control-flow signals in the case of bare except:, and in the case of except Exception: it hides programming errors (a typo causing an AttributeError) behind the same handling path as an expected, recoverable failure.
• Application-level code (the outermost layer, close to a user or an operator) is where broader catches are more defensible, specifically to provide a fallback, a user-facing error message, or a metric increment, since at that point there is nowhere further up to propagate to.

Exception chaining with raise ... from ...

• raise ModelLoadError(...) from original_exc sets original_exc as the new exception's __cause__, so the traceback shown to a developer includes both: "the following exception occurred while handling this one," preserving the original type, message, and traceback instead of losing them.
• Omitting from (raise ModelLoadError(...) inside an except block) still implicitly chains the original as __context__, shown as "during handling of the above exception, another exception occurred"; explicit from is preferred when the translation is intentional, since it documents the relationship as deliberate rather than incidental. raise ... from None suppresses the chain entirely, which is appropriate only when the original exception is genuinely irrelevant noise (rare in a library boundary).

Worked example

class LibraryError(Exception):
    '''Base class for all errors raised by this library.'''

class ModelLoadError(LibraryError):
    '''Raised when a model checkpoint cannot be loaded.'''

def load_checkpoint(path):
    raise FileNotFoundError(path)

try:
    load_checkpoint("/tmp/does-not-exist.pt")
except FileNotFoundError as e:
    raise ModelLoadError(f"could not load checkpoint at {e}") from e

This raises ModelLoadError: could not load checkpoint at /tmp/does-not-exist.pt, and the traceback CPython prints includes both exceptions, joined by the line "The above exception was the direct cause of the following exception:", with the original FileNotFoundError and its own traceback shown first. A caller who only knows about this library's API can catch ModelLoadError (or the LibraryError base) without needing to know the failure originated from a missing file rather than, say, a corrupted checkpoint format; a developer debugging the failure still sees the full original traceback via the chained __cause__.

Trade-offs & pitfalls

• A hierarchy that is too deep (many single-use subclasses that no caller ever catches individually) adds API surface for no behavioral benefit; a hierarchy that is too flat (one exception type for every failure) forces every caller to parse the message string to distinguish cases, which is fragile and not something the standard library's own conventions encourage.
• Catching broadly "to be safe" inside a library function is the single most common way debugging information gets lost: an unrelated bug (a typo, an off-by-one) gets silently reclassified as the same expected failure the except clause was written for, and the real bug ships unnoticed.
• Changing which exception type a public function raises (or removing a subclass from the hierarchy) is a breaking API change for any caller who catches it specifically; treat the exception hierarchy itself as part of the library's versioned public contract, not as an implementation detail.