Python Programming Questions

Comparison:

• Method A (single-threaded generator): simple, low overhead, minimal memory, deterministic ordering, easy error handling; limited by single-core CPU and Python parsing speed.
• Method B (multi-process byte-range split): parallel CPU usage, faster parse throughput on multi-core, but more complex (must align splits to line boundaries), higher peak memory (multiple workers), and increased I/O contention.

Method A, concretely (the whole method really is this short, which is part of why it is the default):

import json

def read_jsonl(path):
    with open(path, 'r', encoding='utf-8') as f:
        for line in f:
            yield json.loads(line)

Method B, concretely, with real byte offsets so 'split misalignment' can be traced rather than only named: take a tiny 40-byte JSON-lines file, four records of exactly 10 bytes each, {"id": 1}\n{"id": 2}\n{"id": 3}\n{"id": 4}\n (bytes 0-9 are line 1, 10-19 line 2, 20-29 line 3, 30-39 line 4). Splitting this file evenly at byte 20 happens to land exactly on a line boundary, so nothing goes wrong there. Splitting unevenly, worker 0 gets raw bytes [0, 15), worker 1 gets [15, 40), lands worker 1's start offset at byte 15, verified to be the : character in the middle of line 2, not the start of any record:

def worker_range(path, start, end):
    with open(path, 'rb') as f:
        f.seek(start)
        if start != 0:
            f.readline()          # discard the partial line landed on, if any
        while f.tell() < end:
            line = f.readline()
            if not line:
                break
            yield json.loads(line)

Worker 1 seeks to byte 15 (mid-line-2), then its f.readline() reads and discards the remainder of line 2 (bytes 15-19, ': 2}\n'), landing the cursor exactly at byte 20, the true start of line 3; from there it reads whole lines normally. The other half of line 2 (bytes 10-15) was already skipped by worker 0, which stopped consuming at byte 20 without seeing it either, so line 2 as a whole is silently dropped. This is exactly the alignment bug: each worker must seek to (and discard up to) the next newline after its own raw start offset, or a record straddling a boundary is lost or corrupted.

When B is preferable:

• CPU-bound parsing (complex JSON) and available cores/IO bandwidth
• File stored on fast SSD or networked storage that supports concurrent reads

Pitfalls & failure modes:

• Split misalignment: worker starts mid-line: must seek to next newline
• Memory pressure: many workers each allocate buffers; can OOM
• Ordering: results may be out-of-order; need merging if order matters
• Partial line encoding issues (multibyte UTF-8 boundaries: UTF-8 is a variable-width encoding where one character can take anywhere from 1 to 4 bytes, so a byte-offset split chosen without regard to character boundaries can land in the middle of a multi-byte character, not just in the middle of a line; a correct implementation aligns on a newline byte, which is always a single, unambiguous ASCII byte and never appears as a continuation byte inside a multi-byte UTF-8 character, rather than aligning on an arbitrary byte count)
• Error recovery complexity: a worker crash loses its chunk

Guidelines: choose worker count to match CPU/IO balance, implement safe split alignment, stream results via queues, and fallback to single-threaded if resources constrained.

Bottleneck reasoning

A bottleneck is the part of the system that dominates total runtime. If the report generator scans the same task records over and over, each new question forces another full pass through the data. That scales poorly as the team grows.

5 scans * 100000 rows = 500000 row visits
1 scan * 100000 rows = 100000 row visits

Intuition: one pass is 5x less work in this example.

What I would change
I would switch the data flow from repeated scans to one ingestion pass that builds summary structures as it goes:

• counters for totals by status, owner, or date
• maps indexed by the fields used in reports
• one reducer function that updates all needed aggregates together

Then each summary question reads from the precomputed aggregates instead of touching the raw records again. That keeps queries fast and the code maintainable because the aggregation logic lives in one place.

Tradeoff
The main cost is a bit more memory for the summary maps. I would accept that tradeoff because it is much cheaper than rescanning the whole dataset repeatedly.

Example
If ten report questions all need ticket counts by team, I would build that count once during ingestion, then reuse it for every report instead of looping over the task table ten times.

Approach

I’d normalize by lowercasing, removing punctuation, and collapsing repeated spaces. Then I’d use a set of normalized keys to keep only the first occurrence. A set is a data structure that gives fast membership checks, so this stays simple and efficient for a few thousand strings.

import re
from typing import List


def normalize_title(title: str) -> str:
    cleaned = re.sub(r'[\W_]+', ' ', title.lower())
    return ' '.join(cleaned.split())


def dedupe_titles(titles: List[str]) -> List[str]:
    seen = set()
    result = []

    for title in titles:
        norm = normalize_title(title)
        if norm in seen:
            continue
        seen.add(norm)
        result.append(norm)

    return result


titles = ['  Weekly Update!', 'weekly update', 'Budget Review', 'Budget review  ']
print(dedupe_titles(titles))

The regex pattern r'[\W_]+' is what actually strips punctuation: \W (capital W) matches any character that is NOT a letter, digit, or underscore, so it catches punctuation, symbols, and stray whitespace runs in one go; adding _ to the character class folds underscores into that same "replace with a space" rule too, since \W alone would NOT match an underscore (underscores count as word characters). The + means one-or-more, so any run of these unwanted characters collapses to a single space rather than leaving multiple spaces behind, and ' '.join(cleaned.split()) then trims leading/trailing spaces and collapses any remaining internal runs down to exactly one space each.

Example

Input: [' Weekly Update!', 'weekly update', 'Budget Review', 'Budget review ']

Output:

['weekly update', 'budget review']

That is the literal stdout of print(dedupe_titles(titles)) from the code block above.

Why this works: the first weekly update is kept, and the later duplicate is skipped because it normalizes to the same key.

Complexity: O(n * m) time, where m is string length, and O(n) extra space for the set.

If the team wants to keep original casing, I would store the first original string in result instead of the normalized version.

My decision

I would lean Go for this tool if the parsing is truly CPU-heavy, meaning the machine spends most of its time computing rather than waiting on disk, and the team wants one deployable binary. Go's compiled binary, static typing, and built-in concurrency fit unattended daily jobs well.

What matters beyond raw speed

• team familiarity and onboarding
• library support for the file formats you need
• packaging and deployment simplicity
• error handling and testability
• memory footprint and startup time
• how often the rules change

When I would pick Python instead
If the job mostly glues together existing Python libraries, or if the engineers need to change parsing rules every week, Python may be faster to evolve and easier to read.

Worked example
If the tool reads 500 large files every morning, I would favor Go when parsing and parallel file handling are the bottlenecks. If the same tool is edited often by a small team that values quick iteration over strict compilation, Python may win because maintenance cost matters more than raw throughput.

So I choose the language that minimizes total operating cost, not just the fastest loop.

Goal: return k most frequent items from a possibly streaming iterable using O(k) extra memory where possible.

Approach: Use Misra-Gries (frequent algorithm) for streaming approximate top-k with O(k) memory; for exact top-k when feasible, use counting with a hashmap and a min-heap of size k (memory O(u) where u unique). For very large universes, prefer Misra-Gries. This is a genuinely advanced, fairly obscure streaming algorithm, most engineers have never needed it; skip straight to "hashmap + heap, exact counts" below unless the interviewer specifically probes for a single-pass, bounded-memory answer.

The core intuition, in plain language, before any code: Misra-Gries tracks at most k candidate items with counts. When a new, not-yet-tracked item arrives and there is no room left (k candidates already exist), instead of dropping the new item and keeping the old ones untouched, EVERY existing candidate's counter is decremented by one, as if the new item had "cancelled out" one occurrence of each current candidate, and the new item itself is not added. Any item that truly appears more than roughly n/k times (n being the total stream length) cannot be fully cancelled away by this process, no matter how the decrements land, so it is guaranteed to still be a candidate at the end; the guarantee is one-sided, though, some less-frequent items may also survive as candidates (false positives), which is exactly why the code takes a second pass over the real data to compute true counts for whatever candidates survived, rather than trusting the approximate counters directly.

Misra-Gries implementation (approximate, deterministic guarantees):

from collections import defaultdict

def top_k_frequent(stream, k):
    if k < 1:
        return []
    counters = {}
    for x in stream:
        if x in counters:
            counters[x] += 1
        elif len(counters) < k:
            counters[x] = 1
        else:
            # decrement all
            to_del = []
            for y in list(counters):
                counters[y] -= 1
                if counters[y] == 0:
                    del counters[y]
    # counters are candidates; to get actual counts, re-scan
    true_counts = defaultdict(int)
    for x in stream:
        if x in counters:
            true_counts[x] += 1
    return sorted(true_counts.items(), key=lambda t: -t[1])[:k]

Worked trace, verified on CPython 3.12: an 8-item stream with k=2 (so the true top item, 'a', appears 5 out of 8 times, far more than the other three items, each appearing once):

stream = ['a', 'a', 'b', 'c', 'a', 'd', 'a', 'a']
print(top_k_frequent(stream, 2))
# [('a', 5), ('d', 1)]

Tracing counters step by step: a (candidate, table has room) -> {'a': 1}. Second a (already tracked, increment) -> {'a': 2}. b (room) -> {'a': 2, 'b': 1}. c arrives with the table full (2 candidates already, k=2): every existing candidate is decremented, a drops to 1, b drops to 0 and is deleted, and c itself is never added -> {'a': 1}. Third a (tracked, increment) -> {'a': 2}. d (room again, since b was just evicted) -> {'a': 2, 'd': 1}. Fourth and fifth a (tracked, increment twice) -> {'a': 4, 'd': 1}. Final candidates: a and d. The second pass then counts each candidate's REAL frequency across the whole stream (a: 5, d: 1) and returns them sorted by that real count.

This trace also shows the false-positive behavior honestly: b and c both had a true count of 1, identical to d's true count of 1, but only d happened to survive as a candidate, purely because of when it arrived relative to the decrement-all events. The algorithm's real guarantee is only about the dominant item: a, at 5 out of 8 occurrences, appears far more often than roughly n/k = 4, and it survives every decrement round intact enough to still be tracked at the end. Nothing is guaranteed about which of the equally-infrequent items happen to survive alongside it, which is precisely why a second, exact pass over the real data is required before trusting any reported count, the approximate candidate set is a correct superset guarantee for truly frequent items, not a precise answer on its own.

Notes: The algorithm uses O(k) memory and one or two passes (approximate single-pass; exact requires second pass over data or storing counts). Choose Misra-Gries for streaming huge data; use Counter+heap for exact counts when unique items fit memory.