Python Programming Questions

Tests and benchmarks:

• Correctness tests: unit tests comparing NumPy reference outputs vs Numba outputs on varied inputs including edge cases (NaN, infinities, zero-length arrays, non-contiguous/strided arrays). Use property-based tests (the hypothesis library: instead of writing individual test inputs by hand, you describe a rule the output must always satisfy, and the library generates many random inputs trying to break that rule) for broader coverage.
• Numerical accuracy: assert tolerances (rtol/atol, relative and absolute tolerance) for floating ops, using np.testing.assert_allclose rather than exact equality, since compiled and interpreted floating-point code can differ in the last few bits even when both are correct.
• Performance benchmarks: microbenchmarks measuring time and compilation overhead (first call vs steady-state) using asv (airspeed velocity, a benchmarking tool that tracks a function's performance across commits, so a regression shows up tied to a specific commit rather than just "it got slower at some point") or pytest-benchmark.

A minimal correctness check, concretely (verified on CPython 3.12; the pattern below is the standard, documented Numba usage, whether or not numba happens to be installed in the environment reading this):

import numpy as np
from numba import njit

def f_numpy(x):
    return np.sqrt(x ** 2 + 1)

@njit
def f_numba(x):
    out = np.empty_like(x)
    for i in range(x.shape[0]):
        out[i] = (x[i] ** 2 + 1) ** 0.5
    return out

x = np.array([0.0, 1.0, 2.0, 3.0])
np.testing.assert_allclose(f_numpy(x), f_numba(x))
# passes silently if the two implementations agree within tolerance;
# assert_allclose raises AssertionError with the offending values if they don't

This is the shape every correctness test in this answer means: compute the same thing both ways, on real (including edge-case) input, and assert they agree, rather than eyeballing the code and assuming they do.

Gotchas & how to test:

• Compilation overhead: measure both cold (first call, which includes the one-time cost of compiling the function for that argument type) and hot runs (every call after, which reuse the already-compiled machine code); prefer caching compiled functions when appropriate.
• Unsupported Python features: ensure no reliance on Python objects in hot loops; test for object-mode fallback by enabling nopython=True in tests and catching TypingError. Numba's default @njit mode (short for nopython=True) compiles the whole function straight to machine code with no Python-object fallback; if some part of the function cannot be compiled that way, @njit raises TypingError immediately rather than silently degrading, which is exactly what you want in tests: a raised TypingError tells you precisely which line Numba could not compile, instead of the function quietly running slow through the interpreter (the older, more permissive default mode's "object-mode fallback").
• ABI/dtype differences: test with different dtypes and memory layouts (C/F contiguous, non-contiguous views). ABI (application binary interface) is the compiled-code calling convention two pieces of native code, here a NumPy build and a Numba-compiled function, must agree on to call each other correctly; a mismatch is rare but shows up as crashes or garbage values rather than a clean error. C-contiguous means an array's elements sit one after another in memory in row-major order (the default for a freshly created array); F-contiguous (Fortran-contiguous) means the same but column-major; a strided or non-contiguous view (e.g. arr[::2]) is neither, and a Numba function compiled assuming one layout can behave incorrectly or need an implicit, costly copy when handed the other, which is exactly why testing with a strided view as input matters, not just a fresh contiguous array.
• Thread-safety: test with multithreaded calls if using parallel=True; check for race conditions.

Automation: integrate benchmarks in CI (nightly) and gate correctness on PRs; document performance expectations and maintain regression alerts.

Direct answer

Python has four argument-passing shapes: positional (matched by order), keyword (matched by name), default values (make a parameter optional), and the catch-alls *args (extra positionals, collected as a tuple) and **kwargs (extra keywords, collected as a dict). They combine in one fixed signature order: positional-or-keyword params, then *args, then keyword-only params (which may have defaults), then **kwargs.

Structured elaboration

Signature order and what each piece means:

def process_batch(data, transform, *args, normalize=True, batch_size=100, **kwargs):
    ...

• data, transform: positional-or-keyword, required. Callers usually pass these positionally because order is natural (the data, then what to do to it).
• *args: soaks up any extra positional arguments beyond data and transform, exposed inside the function as a tuple. Anything named after a bare *args (or a bare *) becomes keyword-only, meaning it cannot be passed positionally.
• normalize, batch_size: keyword-only parameters with defaults, so they are optional and callers must name them.
• **kwargs: soaks up any remaining keyword arguments not already named in the signature, exposed as a dict.

When each pattern is idiomatic versus overkill:

• Positional: idiomatic for the 1-2 arguments every call needs, in an order a reader can memorize. Overkill once a function takes more than 3-4 positional arguments; callers start passing things in the wrong order silently.
• Keyword + default: idiomatic for tunable, optional behavior (batch_size, normalize) where the default covers the common case and the name documents intent at the call site.
• *args: idiomatic for genuinely variadic operations (print(*values), a sum_all(*numbers)) or for forwarding positional arguments through a wrapper. Overkill as a substitute for a real, named parameter list; it hides the function's actual contract from anyone reading a call site.
• **kwargs: idiomatic for forwarding options to an underlying API you do not want to re-declare (backend-specific keys like shuffle or device), or for a plugin-style function whose exact options vary by caller. Overkill as a way to avoid deciding what a function's parameters actually are; unvalidated **kwargs swallows typos (nomralize=True) silently instead of raising TypeError.

Worked example

def process_batch(data, transform, *args, normalize=True, batch_size=100, **kwargs):
    results = []
    for item in data:
        value = transform(item, *args)
        if normalize:
            value = value / batch_size
        results.append(value)
    return results, kwargs

out, extra = process_batch(
    [10, 20, 30],
    lambda x: x * 2,
    normalize=True,
    batch_size=10,
    shuffle=True,
    device="cpu",
)
print(out)    # [2.0, 4.0, 6.0]
print(extra)  # {'shuffle': True, 'device': 'cpu'}

Here data and transform are passed positionally because every call needs them and the order is obvious; normalize and batch_size are named because they are optional tuning knobs; shuffle and device are not declared parameters at all, they arrive through **kwargs and the function forwards or ignores them as extra.

Trade-offs & pitfalls

• A bare * with no name (def f(a, *, b):) is legal and forces everything after it to be keyword-only without collecting extra positionals; use it to make a signature self-documenting even when you do not need *args.
• **kwargs that is never validated is a common production bug: transform_data(dat=my_df) (a typo of data=) silently lands in kwargs instead of raising TypeError: missing required argument. Pop and check known keys explicitly (kwargs.pop("shuffle", False)) if you accept **kwargs, or avoid it and declare the real parameter list.
• Order matters and is enforced by the interpreter, not just style: positional-or-keyword parameters must come before *args, which must come before keyword-only parameters, which must come before **kwargs; writing them out of order is a SyntaxError at definition time.
• Mutable defaults are the sharpest edge here (def f(items=[]):): a default value is evaluated once, when the def statement runs, not on each call, so a mutable default is shared and can accumulate state across unrelated calls.

My decision

I would lean Go for this tool if the parsing is truly CPU-heavy, meaning the machine spends most of its time computing rather than waiting on disk, and the team wants one deployable binary. Go's compiled binary, static typing, and built-in concurrency fit unattended daily jobs well.

What matters beyond raw speed

• team familiarity and onboarding
• library support for the file formats you need
• packaging and deployment simplicity
• error handling and testability
• memory footprint and startup time
• how often the rules change

When I would pick Python instead
If the job mostly glues together existing Python libraries, or if the engineers need to change parsing rules every week, Python may be faster to evolve and easier to read.

Worked example
If the tool reads 500 large files every morning, I would favor Go when parsing and parallel file handling are the bottlenecks. If the same tool is edited often by a small team that values quick iteration over strict compilation, Python may win because maintenance cost matters more than raw throughput.

So I choose the language that minimizes total operating cost, not just the fastest loop.

Direct answer

Give the decorator function an optional first positional parameter (conventionally func=None) alongside keyword-only configuration parameters. When Python evaluates @retry with no parentheses, it calls retry(the_function) directly, so func arrives as the function itself. When Python evaluates @retry(max_attempts=5), it first calls retry(max_attempts=5) with func left at its default of None, and that call must return the actual decorator, which Python then applies to the function on the next line. The whole trick is: check whether func is None and either apply the wrapping immediately or hand back a configured decorator to be applied a moment later.

Structured elaboration

What makes this genuinely ambiguous is that @retry and @retry(max_attempts=5) invoke the same name in two structurally different ways, and Python gives you no explicit signal about which form was used; the only observable difference is what got passed as the first argument:

• No-parens form (@retry): Python evaluates retry as an expression, then immediately calls it with the decorated function as the sole argument: retry(some_function). Here, func is bound to a real callable.
• Parens form (@retry(max_attempts=5)): Python first calls retry(max_attempts=5) to produce a value, then applies that value as the decorator to the function on the next line, calling that value with the function. Here, func is not supplied at all (or would only be supplied as None by default), and the whole first call must return something else that is itself callable and ready to receive the function.

So the detection is: is the value bound to func a callable function (no-parens case), or is it the default None (parens case, meaning we're still waiting for both the config and the function)? Concretely:

• If func is None: this is the parenthesized call. Return the inner decorator function, which Python will apply to the actual function next.
• If func is not None: this is the no-parens call, func already is the function to wrap. Apply decorator(func) immediately and return the wrapped result.

functools.wraps is orthogonal to the ambiguity itself but is required either way, without it, the wrapped function's __name__, __doc__, and other metadata would silently become the wrapper's instead of the original function's, which breaks introspection, debugging, and tools that key off __name__.

Worked example

import functools
from typing import Callable, Optional

def retry(func: Optional[Callable] = None, *, max_attempts: int = 3):
    def decorator(fn: Callable):
        @functools.wraps(fn)
        def wrapped(*args, **kwargs):
            attempts = 0
            while True:
                try:
                    return fn(*args, **kwargs)
                except Exception:
                    attempts += 1
                    if attempts >= max_attempts:
                        raise
        return wrapped

    if func is None:
        return decorator      # parens form: still need the function
    return decorator(func)    # no-parens form: func is already the function

calls = {"a": 0, "b": 0}

@retry
def flaky_a():
    calls["a"] += 1
    if calls["a"] < 2:
        raise ValueError("boom")
    return "a-ok"

@retry(max_attempts=5)
def flaky_b():
    calls["b"] += 1
    if calls["b"] < 4:
        raise ValueError("boom")
    return "b-ok"

print(flaky_a(), calls["a"])   # a-ok 2
print(flaky_b(), calls["b"])   # b-ok 4
print(flaky_a.__name__)        # flaky_a  (functools.wraps preserved the name)

flaky_a succeeds on its second internal call (default max_attempts=3 is enough); flaky_b needed 4 internal calls, only reachable because max_attempts=5 was configured through the parens form.

Trade-offs & pitfalls

• The keyword-only * before max_attempts is load-bearing, not stylistic. Without it, retry(5) (a bare positional call) would ambiguously try to bind 5 to func, which is nonsensical; keyword-only parameters force all configuration to be named, so the only way func gets a non-None value is through the actual no-parens decoration path.
• This pattern does not extend to async def functions for free. The wrapped function here calls fn(*args, **kwargs) synchronously; wrapping an async def with this decorator would return a coroutine object without awaiting it, silently swallowing the retry logic. An async-aware version needs an async def wrapped that awaits fn(...), and ideally detection of which kind of function it's wrapping (inspect.iscoroutinefunction).
• Retrying on bare Exception is intentionally broad here for brevity but is a common wrong turn in production code: it will retry on bugs (a TypeError from a coding mistake) exactly as eagerly as on transient failures (a network timeout), masking real errors behind a few extra silent attempts. A production version should accept an explicit tuple of exception types to retry on.
• No backoff between attempts means a fast-failing dependency gets hammered immediately on every retry; production retry decorators almost always add a delay (ideally with jitter) between attempts.

Vectorized mask for rows containing any NaN:

import numpy as np

# a is shape (N, M)
mask = np.isnan(a).any(axis=1)

Explanation:

• np.isnan(a) produces boolean array shape (N, M).
• .any(axis=1) reduces per row to True if any element is NaN. A numpy axis names which direction a reduction collapses across for a 2D array: axis=0 collapses DOWN each column (one result per column), axis=1 collapses ACROSS each row (one result per row); since the question asks for a per-row answer, one True/False per row of the (N, M) array, axis=1 is the one that produces exactly N results, matching the number of rows.

Worked example, verified on CPython 3.12:

a = np.array([[1.0, 2.0], [np.nan, 4.0], [5.0, 6.0]])
print(np.isnan(a))
# [[False False]
#  [ True False]
#  [False False]]
print(np.isnan(a).any(axis=1))
# [False  True False]

Row 0 ([1.0, 2.0]) has no NaN, so it reduces to False; row 1 ([nan, 4.0]) has one, so it reduces to True; row 2 ([5.0, 6.0]) has none, so False again, exactly the boolean mask a caller would use to filter out or flag the bad row.

This is fully vectorized, memory-efficient (single boolean array) and avoids Python loops. For very large arrays consider processing in chunks to limit peak memory.