Programming Fundamentals Questions

Direct answer
A closure is a function bundled together with references to the variables from its enclosing scope that it uses, captured by reference (not by value), so it keeps seeing the CURRENT value of those variables even after the enclosing function has returned. That captured reference can create a reference cycle, which is why a closure or callback can keep an object alive longer than you expect.

Structured elaboration

• What gets captured: a closure captures the variable itself (technically, the enclosing scope's cell), not a snapshot of its value at creation time. Two closures created from the same enclosing call share independent state; two closures created from the SAME variable in a loop share the same captured cell, which is the classic 'all my callbacks report the same, final loop value' bug.
• Why closures can leak memory: a closure keeps a live reference to everything it captures for as long as the closure itself is reachable. If you then store that closure back onto an object it captured (a callback registered on the very object it was built from), you've created object -> closure -> object, a reference cycle.
• Why reference counting alone can't free a cycle: CPython's primary memory management is reference counting, an object is freed the instant its reference count hits zero. In a cycle, each object holds a reference to the other, so neither one's count ever reaches zero on its own, even after nothing OUTSIDE the cycle references either of them. This is precisely why CPython also runs a separate cyclic garbage collector (gc module) that periodically looks for groups of objects that reference each other but are unreachable from anywhere else, and frees them as a group.
• Mitigation strategies: avoid storing a closure back onto the object it captures when you can restructure to avoid it; use weakref for a back-reference that shouldn't keep the target alive (a common pattern for observer/callback registries); or simply trust the cyclic collector for genuinely short-lived cycles and only investigate further if profiling shows real, growing retention in a long-running process.

Worked example

class Node:
    def __init__(self, name):
        self.name = name
        self.on_event = None

def wire(node):
    def handler():                 # closure: captures `node`
        return f"{node.name} handled"
    node.on_event = handler        # node -> handler -> node : a cycle
    return handler

Verified by running it with gc.disable() and a weakref to the node: after del n (dropping the only external reference), the node is STILL alive (ref() is not None is True) because the cycle keeps both objects' reference counts above zero. Re-enabling the collector and calling gc.collect() reclaims it (ref() is None becomes True immediately after), confirming the cyclic collector, not reference counting, is what actually frees this pattern.

Trade-offs & pitfalls
In a long-running service, this usually shows up as slow, steady memory growth rather than an obvious crash, because the cyclic collector DOES eventually run and free most cycles; the real danger is cycles involving objects with a __del__ method (historically these were UNCOLLECTABLE by the cyclic GC before Python 3.4, and even post-3.4 they add real collection overhead) or large cycles that make each collection pass more expensive as the live object graph grows. The fix is rarely 'stop using closures', it's to be deliberate about back-references specifically, using weakref where a callback registry would otherwise hold the only thing keeping a large object graph alive.

Direct answer
A bare except: (or except Exception: with a silent return None) doesn't just handle the error you intended, it catches every error that happens to occur in that block and treats all of them identically, so a genuine bug (wrong type, a typo'd attribute, a logic error) gets misclassified as 'expected failure' and hidden from anyone who could act on it. The fix is to catch only the SPECIFIC exception you actually expect, and make failure visible (log it, re-raise it, or return a value the caller is forced to check) instead of silently returning a value indistinguishable from a normal result.

Structured elaboration

• Why this is dangerous, not just untidy: None is frequently also a valid, meaningful return value elsewhere in the codebase. A caller receiving None from this function cannot tell 'there was no data' from 'something crashed while getting the data', those are very different situations that need very different handling, and the swallowed exception has erased the distinction.
• Why 'catch everything' is worse than it looks: a bare except catches ValueError (probably intended) but ALSO TypeError, AttributeError, even KeyboardInterrupt in some forms, errors that indicate a real bug in the calling code, not a data-quality issue the function was designed to tolerate. Narrowing to the specific expected exception type is what lets a genuine bug surface loudly instead of being absorbed by the same catch-all.
• What replaces it: catch only the exception type(s) you actually expect and know how to handle; log enough context to diagnose it (what input caused it); then either re-raise (if the caller has no way to proceed without this data) or return an explicit, unambiguous sentinel that cannot be confused with a valid result (not bare None if None is otherwise meaningful).
• The review conversation: once the technical fix is clear, raising it with the author is a normal, low-friction code-review comment focused on the concrete failure mode ('this will hide a real TypeError as if it were expected'), not a judgment about the person, that's what keeps the fix landing quickly.

Worked example

def bad_parse(raw):
    try:
        return int(raw)
    except Exception:
        return None            # catches ValueError AND TypeError identically

def good_parse(raw, logger):
    try:
        return int(raw)
    except ValueError:
        logger.warning("could not parse %r as int", raw)
        raise                  # or: return a sentinel the caller is forced to check

Verified: bad_parse(None) returns None silently, giving no signal that int(None) actually raised a TypeError (passing None where a string/number was expected, a real bug at the call site, not a data-quality issue). good_parse(None, logger) instead lets that TypeError propagate uncaught (confirmed: it raises TypeError, not swallowed), because the function only catches ValueError. good_parse("not-a-number", logger) correctly logs a warning and re-raises ValueError (confirmed by execution), a case the function WAS designed to handle, with a visible trail.

Trade-offs & pitfalls
The judgment call is choosing between re-raising and returning a sentinel: re-raise when the caller genuinely cannot proceed without valid data (most cases); return an explicit sentinel only when the caller has a real, intentional fallback path for 'this record was unparseable' and the sentinel can't be confused with a legitimate value. What never belongs in either path is catching a broader exception type than you can actually reason about, that's the pattern that turns a narrow, expected failure mode into a general-purpose bug hiding place.

Direct answer
Plain assignment doesn't copy anything, it just gives a second name to the same object. A shallow copy creates a new outer container but reuses references to the same nested objects inside it, so mutating a nested element through either the original or the shallow copy is visible in both. A deep copy recursively copies every nested object too, giving you a fully independent structure.

Structured elaboration

• Assignment (b = a): a and b are now two names for the exact same object; b is a is True. There is no 'original' versus 'copy', they're the same thing.
• Shallow copy (copy.copy(a), or list(a), or a[:] for a list): creates a genuinely new outer object (b is a is now False), but for every element that is itself a mutable object (a nested list, a dict, a custom object), the copy holds a reference to the SAME nested object, not a copy of it (b[0] is a[0] is True).
• Deep copy (copy.deepcopy(a)): recursively walks the structure and makes a new copy of every nested mutable object too, so nothing is shared (b[0] is a[0] is False).
• The bug shape this causes: code that shallow-copies a nested structure believing it now has an independent snapshot, then mutates the original, and the 'snapshot' silently changes too, because the shallow copy's nested elements were never actually copied.

Worked example

import copy
original = [[1, 2, 3], [4, 5, 6]]
shallow = copy.copy(original)
deep = copy.deepcopy(original)

original[0].append(999)   # mutate a NESTED element of the original

Verified results after that mutation: original == [[1, 2, 3, 999], [4, 5, 6]], shallow == [[1, 2, 3, 999], [4, 5, 6]] (the nested list was shared, so the shallow copy sees the change too), deep == [[1, 2, 3], [4, 5, 6]] (fully independent, unaffected).

A realistic version of this bug: code takes a shallow copy of a dataset as a 'before' snapshot, then runs an in-place normalization pass over the dataset:

def normalize_inplace(rows):
    for row in rows:
        total = sum(row)
        for i in range(len(row)):
            row[i] = row[i] / total if total else 0

After running normalize_inplace on the dataset, the shallow-copied 'snapshot' taken beforehand is bitwise identical to the now-normalized dataset (verified by running it: snapshot == dataset evaluates True after normalization), because normalize_inplace mutates each row list in place, and the shallow copy's rows are the SAME row objects as the original's. The 'backup' was never a backup.

Trade-offs & pitfalls
The fix depends on what you actually need: if you truly need an independent snapshot, use copy.deepcopy (accepting its cost, see the mutability discussion) or rebuild the structure by copying each nested piece explicitly. If deep-copying every row of a large dataset is too expensive, the more scalable fix is usually to stop mutating in place at all, have normalize_inplace return a new structure instead of mutating its argument, which sidesteps the shallow/deep copy question entirely by removing the shared-mutable-state pattern that created the risk.

Direct answer
Convert the recursive walk into an explicit loop that maintains its own stack (a plain list/array), pushing left children as you descend and popping/visiting/moving right exactly where the recursive calls would have happened, so the traversal order is identical but the call depth is no longer tied to the language's function-call stack.

Structured elaboration
The recursive in-order traversal is: recurse left, visit the node, recurse right. Each recursive call pushes a real stack frame, so a left-skewed tree of depth d uses O(d) frames and blows the interpreter's recursion limit once d exceeds it (Python's default is 1000). The iterative version replaces those implicit call-stack frames with an explicit stack you manage yourself, which lives on the heap and has no language-imposed depth limit (bounded only by available memory, not by a fixed call-depth ceiling):

1. Walk left as far as possible, pushing every node visited onto the stack (mirrors descending through the left-recursion calls without visiting yet).
2. Pop the top of the stack, that's the next node to visit, in the exact order the recursive version would have visited it.
3. Move to that node's right child and repeat from step 1 (mirrors the recurse-right call).
4. Stop when the stack is empty and there's no current node left to descend into.

Worked example

def inorder_iterative(root):
    out = []
    stack = []
    node = root
    while stack or node is not None:
        while node is not None:
            stack.append(node)
            node = node.left
        node = stack.pop()
        out.append(node.val)
        node = node.right
    return out

Verified against a known small balanced tree (root 4, left subtree 2 with children 1 and 3, right subtree 6 with children 5 and 7): both the recursive version and this iterative version return [1, 2, 3, 4, 5, 6, 7], identical order. Verified against a deliberately pathological case, a left-skewed tree of depth 3000 (each node's left child is the next node down, no right children): the recursive version raises RecursionError: maximum recursion depth exceeded at Python's default limit of 1000, while the iterative version completes and returns all 3000 values in order (len(result) == 3000, result == list(range(1, 3001))), confirming it has no equivalent depth ceiling.

Trade-offs & pitfalls
The iterative version is not simply 'better', it trades the recursive version's direct correspondence to the problem's structure (which makes it easy to verify by inspection) for an explicit stack whose invariant (everything on the stack is an ancestor of the current node, still awaiting its visit-and-descend-right) is easy to get subtly wrong, common bugs are popping before fully descending left, or forgetting to move to node.right after visiting and looping forever on the same node. This conversion is worth doing specifically when input depth is attacker- or user-controlled and therefore cannot be assumed small (a request-driven tree/graph walk), not as a blanket 'recursion is bad' rule for every tree operation.

Direct answer
The four core built-in containers split along two axes: mutability (can you change it after creation?) and whether elements need to be ordered/duplicable versus unique/hashable. A list is a mutable ordered sequence, a tuple is an immutable ordered sequence, a set is a mutable unordered collection of unique hashable elements, and a dict is a mutable unordered mapping of unique hashable keys to values.

Structured elaboration

The O(1)-average / O(n)-worst-case split for set/dict comes from hashing: normally a hash lookup goes straight to (approximately) the right bucket, but if many keys collide into the same bucket, resolving the collision degenerates toward a linear scan. list/tuple index access is O(1) because the underlying storage is one contiguous block, computing an offset from the index is arithmetic, not a search; searching a list BY VALUE (x in my_list) is O(n) because there's no shortcut, every element may need to be checked.

Worked example
Hashability is the concrete reason tuples, not lists, can be dict keys or set members: {(1, 2): 'a point'} works because a tuple's contents can't change after creation, so its hash value is stable for its lifetime; {[1, 2]: 'a point'} raises TypeError: unhashable type: 'list' because a list's contents CAN change, so Python refuses to let it serve as a hash key at all (verified: t = (1, 2, 3) then t[0] = 99 raises TypeError: 'tuple' object does not support item assignment; {1, 2, 2, 3} == {1, 2, 3}, confirming a set silently drops the duplicate 2).

Trade-offs & pitfalls
The most common mistake is choosing list by default and doing repeated x in my_list membership checks in a hot path, that's O(n) per check and O(n*m) over m checks; switching to a set for membership-only use cases is one of the cheapest performance wins available. The second is using a mutable default in a spot that implicitly needs hashability (trying to use a list as a dict key, or storing lists inside a set) and hitting a TypeError that a tuple would have avoided entirely.