Python Programming Questions

Vectorized mask for rows containing any NaN:

import numpy as np

# a is shape (N, M)
mask = np.isnan(a).any(axis=1)

Explanation:

• np.isnan(a) produces boolean array shape (N, M).
• .any(axis=1) reduces per row to True if any element is NaN. A numpy axis names which direction a reduction collapses across for a 2D array: axis=0 collapses DOWN each column (one result per column), axis=1 collapses ACROSS each row (one result per row); since the question asks for a per-row answer, one True/False per row of the (N, M) array, axis=1 is the one that produces exactly N results, matching the number of rows.

Worked example, verified on CPython 3.12:

a = np.array([[1.0, 2.0], [np.nan, 4.0], [5.0, 6.0]])
print(np.isnan(a))
# [[False False]
#  [ True False]
#  [False False]]
print(np.isnan(a).any(axis=1))
# [False  True False]

Row 0 ([1.0, 2.0]) has no NaN, so it reduces to False; row 1 ([nan, 4.0]) has one, so it reduces to True; row 2 ([5.0, 6.0]) has none, so False again, exactly the boolean mask a caller would use to filter out or flag the bad row.

This is fully vectorized, memory-efficient (single boolean array) and avoids Python loops. For very large arrays consider processing in chunks to limit peak memory.

Project layout

• mypackage/
  • src/mypackage/init.py # package implementation
  • src/mypackage/module.py
  • tests/ # unit tests
    • test_module.py
  • pyproject.toml / setup.cfg

Where to place tests
Keep tests outside package source (e.g., tests/ at repo root). This prevents accidental import of tests and mirrors CI test discovery.

init.py & public API
Use init.py to expose a stable public API with explicit imports and all. Keep implementation modules internal (e.g., mypackage._internal).

A concrete worked example of what that __init__.py and its module actually contain:

# src/mypackage/module.py
def process(data):
    return [x * 2 for x in data]

def _internal_helper():   # leading underscore: not part of the public API
    pass

# src/mypackage/__init__.py
from .module import process

__all__ = ["process"]

A caller then does from mypackage import process, process is the stable public name, while mypackage.module._internal_helper stays a private implementation detail nobody outside the package should import directly; __all__ also controls exactly what from mypackage import * pulls in.

Testing in CI

• Use pytest, run in CI (GitHub Actions) with matrix for Python versions. Run mypy (checks your type hints without actually running the code) and unit tests; fail on coverage thresholds.

A minimal CI job doing exactly that:

# .github/workflows/test.yml
jobs:
  test:
    strategy:
      matrix:
        python-version: ["3.11", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pip install -e .[dev]
      - run: pytest --cov=mypackage --cov-fail-under=80
      - run: mypy src/mypackage

Type stubs & docs

• Add type hints in code; supply .pyi stubs (a .pyi file is a type-hint-only file, holding no real code, for code Python itself cannot read type hints from directly, such as a compiled C extension) for compiled extensions.
• Use Sphinx or MkDocs (both turn your docstrings into a browsable documentation website) for docs; generate API reference from docstrings. Include CI job to build docs.

Which of these matter most for a first working setup versus are polish added later: the src/ layout, an explicit __init__.py public API, and pytest running in CI are the essentials, without them the package is neither testable nor safely importable by other code. mypy, coverage thresholds, .pyi stubs, and generated documentation sites are all valuable, genuinely worth doing, but they are follow-up hardening once the basic package structure and test suite already exist, not blockers to a first working, publishable package.

Best practices: pin dev deps, run linters, and keep public surface minimal and documented.

Approach: build a directed graph of lock waits and detect cycles. Represent each lock as node; for each thread that holds locks A then requests B, add edges A -> B. A cycle implies possible deadlock.

def detect_deadlocks(thread_dump):
    # thread_dump: list of tuples (holds:list, wants:list)
    from collections import defaultdict, deque
    g=defaultdict(list)
    for holds, wants in thread_dump:
        for h in holds:
            for w in wants:
                g[h].append(w)
    # detect cycle via DFS
    visited=set(); stack=set()
    def dfs(u):
        if u in stack: return True
        if u in visited: return False
        visited.add(u); stack.add(u)
        for v in g.get(u, []):
            if dfs(v): return True
        stack.remove(u); return False
    return any(dfs(node) for node in g)

Note (a correction to the traversal, verified by actually running both versions): the loop reads g.get(u, []) rather than plain g[u]. Since g is a defaultdict(list), g[u] for a lock u that is only ever requested and never itself a holds key would silently insert a new empty entry for u into g the first time it is visited, mutating the dictionary while the outer for node in g is still iterating over it. Run directly, that raises RuntimeError: dictionary changed size during iteration on any thread_dump containing such a lock, for example the two-thread chain in the worked example just below. .get(u, []) reads the same list without ever creating a new key, so the traversal cannot perturb the structure it is iterating over.

Worked example, verified on CPython 3.12:

# Deadlock: A holds a lock and wants B's; B holds a lock and wants A's -> circular wait.
print(detect_deadlocks([(['A'], ['B']), (['B'], ['A'])]))
# True

# No deadlock: A wants what B holds, B wants what C holds -> a chain, not a cycle.
print(detect_deadlocks([(['A'], ['B']), (['B'], ['C'])]))
# False

For the deadlock case, the edge-building loop adds A -> B (from the first tuple) and B -> A (from the second), so g = {'A': ['B'], 'B': ['A']}. dfs('A') marks A visited and on-stack, follows the edge to B, marks B visited and on-stack, then follows B's edge back to A; since A is already on the current recursion stack, dfs returns True immediately, that revisit of an on-stack node is the cycle. For the no-deadlock case, g = {'A': ['B'], 'B': ['C']}; dfs('A') walks A -> B -> C, and C has no outgoing edges (g.get('C', []) returns [], since C never holds anything in this example) and is never back on the stack, so every branch returns False and no cycle is found.

Complexity: building edges O(E) where E = sum(hands*wants); cycle detection O(V+E).

Correctness: an edge A -> B means "a thread holding A is waiting to acquire B", i.e. that thread cannot proceed until whoever holds B releases it. Sufficiency (a cycle really does mean a stuck circular wait): if A -> B -> C -> A is a real cycle, the thread waiting on the A -> B edge cannot proceed until B frees up, but whoever holds B is itself stuck on the B -> C edge waiting for C, and whoever holds C is stuck on the C -> A edge waiting for A, which is held by the very first thread that is blocked; every thread on the cycle is waiting on the next one, forever, none of them can be the one to break the chain. Necessity (a stuck circular wait always shows up as a cycle here): if a set of threads really is deadlocked in a circular wait, then by definition each thread in that set holds one lock and is blocked wanting another lock in the same set, which is exactly one holds -> wants edge per thread; following those edges from any thread in the set must eventually revisit a thread already seen, since there are only finitely many threads in the set and every one of them has an outgoing edge to another member of the set, and a finite directed graph where every node in some subset has an outgoing edge back into that same subset necessarily contains a cycle. So "cycle in this graph" and "circular wait is possible" imply each other, given the model that a thread's current wait state is fully captured by its (holds, wants) entry.

Approach (iterative): Use a stack to process items in order, pushing list elements in reverse so original order is preserved. This avoids recursion depth limits.

Implementation:

def flatten_list(nested):
    """Flatten nested lists (iterative)."""
    result = []
    stack = list(reversed(nested))

    while stack:
        item = stack.pop()
        if isinstance(item, list):
            stack.extend(reversed(item))
        else:
            result.append(item)
    return result

Worked example, verified on CPython 3.12:

print(flatten_list([1, [2, [3, 4], 5], 6]))
# [1, 2, 3, 4, 5, 6]

Tracing the stack (pop() takes from the end, so pushing a sub-list's items in reverse order means they come back out of the stack in their original left-to-right order, since a stack is last-in-first-out: the last thing pushed is the first thing popped, so reversing before pushing cancels out the stack's own order-reversal): stack starts as [6, [2, [3, 4], 5], 1] (the whole input list, reversed). Popping 1 (a plain value) appends it to result, leaving stack = [6, [2, [3, 4], 5]]. Popping [2, [3, 4], 5] (a list) pushes its contents back on in reverse, [5, [3, 4], 2], giving stack = [6, 5, [3, 4], 2]. Popping 2 appends it (result = [1, 2]). Popping [3, 4] pushes [4, 3] back on, giving stack = [6, 5, 4, 3]. Popping 3, then 4, then 5, then 6 in that order appends each in turn, building result = [1, 2, 3, 4, 5, 6], matching the original left-to-right order exactly, and matching the printed output above.

Key points: preserves original order; handles arbitrary nesting; non-recursive.

Time Complexity: O(n) where n is total number of elements (visit each once).
Space Complexity: O(n) for result plus O(d) stack where d is nesting breadth (worst-case O(n)).

Edge cases: empty list, lists with non-list iterables (only lists are flattened per spec), deeply nested single chains handled without recursion.

Goal: compute rank within each 'category' in descending order efficiently.

Solution:

import pandas as pd

# assuming df has columns ['category', 'value']
df = pd.DataFrame({
    "category": ["A", "A", "A", "B", "B"],
    "value":    [10,  10,  8,   5,   5],
})

# use groupby + rank with method='dense' or 'first' depending on tie behavior
df['rank_within_group'] = df.groupby('category')['value']\
    .rank(method='dense', ascending=False).astype(int)

method='dense' is the crux of the tie-breaking behavior here: it means tied values SHARE a rank, and the next distinct value continues at the very next rank number with no gap. That is different from method='first' (ties broken by row order, so no two rows ever share a rank) and from the default method='average' (ties get the average of the ranks they would have spanned). For descending values like 10, 10, 8, dense ranks them 1, 1, 2, not 1, 2, 3 (that would be method='first' or 'min'-adjacent behavior) and not 1.5, 1.5, 3 (that would be the average default).

Worked example, verified with pandas on CPython 3.12, including a genuine tie so the behavior is visible rather than only named:

df = pd.DataFrame({
    "category": ["A", "A", "A", "B", "B"],
    "value":    [10,  10,  8,   5,   5],
})
df['rank_within_group'] = df.groupby('category')['value']\
    .rank(method='dense', ascending=False).astype(int)
print(df)

Output:

  category  value  rank_within_group
0        A     10                  1
1        A     10                  1
2        A      8                  2
3        B      5                  1
4        B      5                  1

Category A's two tied 10s both get rank 1 (the highest, since ascending=False), and the next distinct value, 8, gets rank 2 immediately, no rank 3 is skipped to "make room" for the tie. Category B's two tied 5s both get rank 1, since within group B, 5 is the only (and therefore highest) value present.

Performance & memory:

• groupby.rank is implemented in C and avoids Python-level loops; it's fast and memory-efficient compared to apply.
• For millions of rows, ensure 'category' is categorical to save memory and speed grouping:

df['category'] = df['category'].astype('category')

• If memory is tight, process in chunks by categories: iterate over df.groupby('category') or sort by category and value and rank per chunk.

Trade-offs:

• In-memory groupby requires holding column arrays; converting to categorical reduces memory but adds preprocessing cost.
• Chunking reduces peak RAM but loses single-call simplicity and may require external sorting.