Python Programming Questions

1) Multiprocessing (multiprocessing module)

• Benefit: sidesteps GIL (Global Interpreter Lock: a lock inside CPython that only lets one thread execute Python bytecode at a time, which is why adding more threads alone does not speed up pure-Python CPU-bound work) by using multiple processes; good for CPU-bound tasks that can be partitioned.
• Limitations: IPC and data serialization overhead, higher memory usage per process.
• When: embarrassingly parallel workloads (map-style), batch processing across cores.

2) C-accelerated libraries (NumPy / vectorization)

• Benefit: move heavy numeric loops into C (SIMD, Single Instruction Multiple Data, a CPU feature applying one operation to many values in one step; contiguous memory), drastically faster for array math.
• Limitations: requires expressing work in array form; not helpful for complex control flow.
• When: numerical computations over arrays, matrix ops.

3) Just-in-time / ahead-of-time compilation (Numba or Cython)

• Numba: JIT-compiles (just-in-time compiles: instead of compiling ahead of time before the program ever runs, the function is compiled straight to machine code the first time it is actually called) Python functions to machine code with little code changes; excellent for loops on numeric data.
  • Benefit: low development cost, big speedups.
  • Limitation: not all Python features supported; first-call compilation overhead.
• Cython: compile Python to C for maximum control and speed; can produce greatest gains but requires typing and build step.
  • When: algorithmic hotspots that remain after vectorization or need fine-grained control.

A concrete contrast (real values, not timing, since exact speed depends on hardware and is not something to hardcode here):

import numpy as np

def squares_loop(n):
    return [i * i for i in range(n)]

def squares_vectorized(n):
    return np.arange(n) ** 2

print(squares_loop(5))
# [0, 1, 4, 9, 16]
print(squares_vectorized(5))
# [ 0  1  4  9 16]

Both produce the identical five values; squares_loop pays Python's per-element interpreter overhead five separate times, while squares_vectorized dispatches once into a single compiled C loop, the same mechanism that makes strategy 2 (vectorization) and strategy 3 (Numba/Cython compilation) both faster than a plain interpreted loop: removing per-element interpreter overhead, not changing the underlying arithmetic.

Choose based on workload: use vectorization first, then Numba for loop-heavy numeric code, and multiprocessing when parallelism across cores is needed and data can be partitioned.

Situation: Overnight job reads a directory of daily status files. Some may be empty, malformed, or missing.

Task: Keep valid data moving, but tell the operator exactly what failed.

Action: I would loop over the expected file list and handle errors per file, not around the whole batch.

• If a file is missing, record missing and continue.
• If a file is empty, record empty and continue.
• If a line is malformed, skip just that file or just that line depending on the rule, and capture the filename and line number.
• Keep running totals in memory, plus a failed_files list for the final report.

Result: The job still produces a useful summary from good files, while the operator gets a clean failure section like status_2025-07-03.txt: malformed line 42 and status_2025-07-04.txt: missing. That is better than a hard stop because it keeps the daily workflow moving.

A small worked example, verified on CPython 3.12, showing the actual loop and its exact printed summary rather than only describing the shape:

def build_report(expected_files, file_contents):
    report = {"processed": 0, "total_value": 0, "failed": []}
    for name in expected_files:
        if name not in file_contents:
            report["failed"].append(f"{name}: missing")
            continue
        content = file_contents[name]
        if content.strip() == "":
            report["failed"].append(f"{name}: empty")
            continue
        try:
            value = int(content.strip())
        except ValueError:
            report["failed"].append(f"{name}: malformed line")
            continue
        report["processed"] += 1
        report["total_value"] += value
    return report

expected = [f"status_2025-07-{i:02d}.txt" for i in range(1, 11)]
contents = {name: "5" for name in expected}
contents["status_2025-07-05.txt"] = ""          # empty
del contents["status_2025-07-09.txt"]            # missing

print(build_report(expected, contents))

Output:

{'processed': 8, 'total_value': 40, 'failed': ['status_2025-07-05.txt: empty', 'status_2025-07-09.txt: missing']}

8 files processed (matching the 10 expected minus the 1 empty and 1 missing), each contributing a value of 5, giving total_value = 40; the two problem files are named explicitly in failed with the specific reason, exactly the breakdown a report shown to an operator needs.

Approach

I’d normalize by lowercasing, removing punctuation, and collapsing repeated spaces. Then I’d use a set of normalized keys to keep only the first occurrence. A set is a data structure that gives fast membership checks, so this stays simple and efficient for a few thousand strings.

import re
from typing import List


def normalize_title(title: str) -> str:
    cleaned = re.sub(r'[\W_]+', ' ', title.lower())
    return ' '.join(cleaned.split())


def dedupe_titles(titles: List[str]) -> List[str]:
    seen = set()
    result = []

    for title in titles:
        norm = normalize_title(title)
        if norm in seen:
            continue
        seen.add(norm)
        result.append(norm)

    return result


titles = ['  Weekly Update!', 'weekly update', 'Budget Review', 'Budget review  ']
print(dedupe_titles(titles))

The regex pattern r'[\W_]+' is what actually strips punctuation: \W (capital W) matches any character that is NOT a letter, digit, or underscore, so it catches punctuation, symbols, and stray whitespace runs in one go; adding _ to the character class folds underscores into that same "replace with a space" rule too, since \W alone would NOT match an underscore (underscores count as word characters). The + means one-or-more, so any run of these unwanted characters collapses to a single space rather than leaving multiple spaces behind, and ' '.join(cleaned.split()) then trims leading/trailing spaces and collapses any remaining internal runs down to exactly one space each.

Example

Input: [' Weekly Update!', 'weekly update', 'Budget Review', 'Budget review ']

Output:

['weekly update', 'budget review']

That is the literal stdout of print(dedupe_titles(titles)) from the code block above.

Why this works: the first weekly update is kept, and the later duplicate is skipped because it normalizes to the same key.

Complexity: O(n * m) time, where m is string length, and O(n) extra space for the set.

If the team wants to keep original casing, I would store the first original string in result instead of the normalized version.

Approach: schedule tasks, wait with timeout, cancel pending tasks after timeout, re-raise exceptions from completed tasks. (A coroutine is what you get back from calling an async def function; calling it does not run it yet, asyncio.create_task(c) is what actually schedules a coroutine to start running concurrently with everything else.)

Implementation:

import asyncio

async def gather_with_timeout(coros, timeout):
    tasks = [asyncio.create_task(c) for c in coros]
    done, pending = await asyncio.wait(tasks, timeout=timeout)
    # cancel pending
    for t in pending:
        t.cancel()
    # await cancellations to finish
    if pending:
        await asyncio.gather(*pending, return_exceptions=True)
    # propagate exceptions from done
    results = []
    for t in done:
        exc = t.exception()
        if exc:
            # re-raise first exception
            raise exc
        results.append(t.result())
    return tuple(results), tuple(pending)

# Usage example
async def main():
    async def work(n):
        await asyncio.sleep(n); return n
    done, pending = await gather_with_timeout([work(0.1), work(2)], timeout=1)
    print(done, pending)

# asyncio.run(main())

Notes: ensures pending tasks are cancelled and awaited to avoid warnings; exceptions from completed tasks propagate.

Running it: uncommenting asyncio.run(main()) and adding print(len(pending), pending[0].cancelled()) right after print(done, pending) shows what actually happens: work(0.1) finishes well inside the 1-second timeout and work(2) does not, so done prints as (0.1,) (the tuple of results from the finished work(0.1)) and the follow-up print shows 1 True, one pending task, and it reports cancelled() == True, confirming the still-running work(2) really was cancelled rather than left dangling.

Tests and benchmarks:

• Correctness tests: unit tests comparing NumPy reference outputs vs Numba outputs on varied inputs including edge cases (NaN, infinities, zero-length arrays, non-contiguous/strided arrays). Use property-based tests (the hypothesis library: instead of writing individual test inputs by hand, you describe a rule the output must always satisfy, and the library generates many random inputs trying to break that rule) for broader coverage.
• Numerical accuracy: assert tolerances (rtol/atol, relative and absolute tolerance) for floating ops, using np.testing.assert_allclose rather than exact equality, since compiled and interpreted floating-point code can differ in the last few bits even when both are correct.
• Performance benchmarks: microbenchmarks measuring time and compilation overhead (first call vs steady-state) using asv (airspeed velocity, a benchmarking tool that tracks a function's performance across commits, so a regression shows up tied to a specific commit rather than just "it got slower at some point") or pytest-benchmark.

A minimal correctness check, concretely (verified on CPython 3.12; the pattern below is the standard, documented Numba usage, whether or not numba happens to be installed in the environment reading this):

import numpy as np
from numba import njit

def f_numpy(x):
    return np.sqrt(x ** 2 + 1)

@njit
def f_numba(x):
    out = np.empty_like(x)
    for i in range(x.shape[0]):
        out[i] = (x[i] ** 2 + 1) ** 0.5
    return out

x = np.array([0.0, 1.0, 2.0, 3.0])
np.testing.assert_allclose(f_numpy(x), f_numba(x))
# passes silently if the two implementations agree within tolerance;
# assert_allclose raises AssertionError with the offending values if they don't

This is the shape every correctness test in this answer means: compute the same thing both ways, on real (including edge-case) input, and assert they agree, rather than eyeballing the code and assuming they do.

Gotchas & how to test:

• Compilation overhead: measure both cold (first call, which includes the one-time cost of compiling the function for that argument type) and hot runs (every call after, which reuse the already-compiled machine code); prefer caching compiled functions when appropriate.
• Unsupported Python features: ensure no reliance on Python objects in hot loops; test for object-mode fallback by enabling nopython=True in tests and catching TypingError. Numba's default @njit mode (short for nopython=True) compiles the whole function straight to machine code with no Python-object fallback; if some part of the function cannot be compiled that way, @njit raises TypingError immediately rather than silently degrading, which is exactly what you want in tests: a raised TypingError tells you precisely which line Numba could not compile, instead of the function quietly running slow through the interpreter (the older, more permissive default mode's "object-mode fallback").
• ABI/dtype differences: test with different dtypes and memory layouts (C/F contiguous, non-contiguous views). ABI (application binary interface) is the compiled-code calling convention two pieces of native code, here a NumPy build and a Numba-compiled function, must agree on to call each other correctly; a mismatch is rare but shows up as crashes or garbage values rather than a clean error. C-contiguous means an array's elements sit one after another in memory in row-major order (the default for a freshly created array); F-contiguous (Fortran-contiguous) means the same but column-major; a strided or non-contiguous view (e.g. arr[::2]) is neither, and a Numba function compiled assuming one layout can behave incorrectly or need an implicit, costly copy when handed the other, which is exactly why testing with a strided view as input matters, not just a fresh contiguous array.
• Thread-safety: test with multithreaded calls if using parallel=True; check for race conditions.

Automation: integrate benchmarks in CI (nightly) and gate correctness on PRs; document performance expectations and maintain regression alerts.