Python Programming Questions

Pytest unit tests for normalize_email(email: str)

import pytest
from yourmodule import normalize_email

def test_strip_whitespace_and_lower_domain():
    assert normalize_email('  User+tag@EXAMPLE.COM  ') == 'User+tag@example.com'

def test_empty_string():
    assert normalize_email('') == ''

def test_trim_local_only():
    assert normalize_email('  local  @DoMain.Com') == 'local@domain.com'

def test_invalid_type_raises():
    with pytest.raises(TypeError):
        normalize_email(None)

Notes: tests cover leading/trailing whitespace, uppercase domain, empty string, and invalid type. Adjust expected behavior if function normalizes tags or lowercases local part.

Approach

I’d normalize by lowercasing, removing punctuation, and collapsing repeated spaces. Then I’d use a set of normalized keys to keep only the first occurrence. A set is a data structure that gives fast membership checks, so this stays simple and efficient for a few thousand strings.

import re
from typing import List


def normalize_title(title: str) -> str:
    cleaned = re.sub(r'[\W_]+', ' ', title.lower())
    return ' '.join(cleaned.split())


def dedupe_titles(titles: List[str]) -> List[str]:
    seen = set()
    result = []

    for title in titles:
        norm = normalize_title(title)
        if norm in seen:
            continue
        seen.add(norm)
        result.append(norm)

    return result


titles = ['  Weekly Update!', 'weekly update', 'Budget Review', 'Budget review  ']
print(dedupe_titles(titles))

The regex pattern r'[\W_]+' is what actually strips punctuation: \W (capital W) matches any character that is NOT a letter, digit, or underscore, so it catches punctuation, symbols, and stray whitespace runs in one go; adding _ to the character class folds underscores into that same "replace with a space" rule too, since \W alone would NOT match an underscore (underscores count as word characters). The + means one-or-more, so any run of these unwanted characters collapses to a single space rather than leaving multiple spaces behind, and ' '.join(cleaned.split()) then trims leading/trailing spaces and collapses any remaining internal runs down to exactly one space each.

Example

Input: [' Weekly Update!', 'weekly update', 'Budget Review', 'Budget review ']

Output:

['weekly update', 'budget review']

That is the literal stdout of print(dedupe_titles(titles)) from the code block above.

Why this works: the first weekly update is kept, and the later duplicate is skipped because it normalizes to the same key.

Complexity: O(n * m) time, where m is string length, and O(n) extra space for the set.

If the team wants to keep original casing, I would store the first original string in result instead of the normalized version.

Approach

I would read the file line by line with bufio.Scanner. Opening errors are fatal, because there is no file to process. Line parse errors are recoverable, meaning a bad line does not stop the whole file, so I return them in a slice and keep the good records. If scanning itself fails, I return the partial data plus the scan error so the caller can decide whether to fail the job.

package main

import (
    "bufio"
    "fmt"
    "os"
    "strconv"
    "strings"
)

type Record struct {
    Name string
    Age  int
}

type ParseIssue struct {
    Line int
    Raw  string
    Err  error
}

func ParseFile(path string) ([]Record, []ParseIssue, error) {
    f, err := os.Open(path)
    if err != nil {
        return nil, nil, err
    }
    defer f.Close()

    scanner := bufio.NewScanner(f)
    scanner.Buffer(make([]byte, 1024), 1024*1024)

    var records []Record
    var issues []ParseIssue
    lineNum := 0

    for scanner.Scan() {
        lineNum++
        line := strings.TrimSpace(scanner.Text())
        if line == "" {
            continue
        }

        parts := strings.Split(line, ",")
        if len(parts) != 2 {
            issues = append(issues, ParseIssue{Line: lineNum, Raw: line, Err: fmt.Errorf("expected name,age")})
            continue
        }

        age, err := strconv.Atoi(strings.TrimSpace(parts[1]))
        if err != nil {
            issues = append(issues, ParseIssue{Line: lineNum, Raw: line, Err: err})
            continue
        }

        records = append(records, Record{
            Name: strings.TrimSpace(parts[0]),
            Age:  age,
        })
    }

    if err := scanner.Err(); err != nil {
        return records, issues, err
    }

    return records, issues, nil
}

Key points

• open failure: return immediately
• parsing failure: collect issue and continue
• scan failure: return partial results and an error

Example: with Ann,32, bad-line, and Bob,41, you get 2 records and 1 issue.

Complexity: O(n) time, O(k) memory for stored records and issues.

Experiment design
Compare three functions in a tight loop: (A) raise/catch exception on error, (B) return error code and check, (C) pre-validated path (no error). Use timeit and large N.

Code:

import timeit

def raise_path(n):
    for i in range(n):
        try:
            if i%100==0: raise ValueError
        except ValueError:
            pass

def errorcode_path(n):
    for i in range(n):
        ok = True            # the normal, no-error case
        if i%100==0: ok = False   # the simulated error case
        if not ok: pass

def prevalidated_path(n):
    for i in range(n):
        pass                 # no error branch at all: the baseline

n=1000000
print(timeit.timeit(lambda: raise_path(n), number=3))
print(timeit.timeit(lambda: errorcode_path(n), number=3))
print(timeit.timeit(lambda: prevalidated_path(n), number=3))

A correction to the code above (a genuine bug, not just a style choice): as originally sketched, errorcode_path set ok = False unconditionally at the top of every iteration and never set it back to True on the normal path, so if not ok: was True on every single iteration, not just the 1-in-100 simulated errors. That does not model "check an error code" at all, it just runs the pass branch every time. The fix is the one shown above: ok = True by default (the common, no-error case), flipped to False only on the simulated error, so errorcode_path and raise_path are actually testing the same 1% error frequency against each other. The third function, prevalidated_path, is also added here: it has no error branch whatsoever, and exists specifically as the baseline "cost of the loop itself, with no error handling of any kind" that the question's three-way comparison (raise/catch, return-code, pre-validated) asks for; the original sketch defined only the first two.

Interpretation

• Run this at varying simulated error frequencies (change i%100==0 to i%1==0 for 100% errors, or remove the if entirely for 0%) rather than trusting a single frequency; the shape of the gap between raise_path and errorcode_path as a function of error frequency is the actual finding, a single absolute number from one machine is not portable or reproducible elsewhere and should not be reported as "the" result. prevalidated_path's time is the floor: the cost of iterating n times with no error-handling machinery of any kind, useful as the baseline both other paths are measured against.
• The mechanism, not a specific number, is what to lead with: constructing and raising a real exception involves allocating an exception object, populating a traceback, and unwinding the stack to find a matching except, real, nonzero work that a plain boolean check never does; at 0% error frequency, the try block itself still costs something (setting up the exception-handling frame) even though nothing is ever raised, so raise_path at 0% errors is a fair comparison of that baseline try-frame overhead against errorcode_path's baseline if overhead. As error frequency rises toward 100%, the actual raise/unwind cost starts to dominate raise_path's total time in a way it never does for errorcode_path, which does the same constant amount of work (ok = True, one comparison) whether or not that iteration's error flag ends up True.
• For production ETL, prefer error-code or pre-validation for expected, frequent errors, reserve exceptions for truly exceptional control flow, situations that are rare enough that even a real per-raise cost barely matters in aggregate, and where the cleaner control-flow and forced handling (you cannot silently ignore a raised exception the way an unchecked error code can be ignored) outweigh that cost.
• Profile memory and CPU (not just the timeit total) to ensure GC or traceback construction isn't dominating the comparison for reasons unrelated to the actual branch being tested, for example, a test harness that also does unrelated allocation inside the timed loop.

Approach summary
Use an atomic filesystem operation to create a lockfile and store owner PID. "Atomic" here means the operating system guarantees that checking whether the file already exists and creating it happen as a single, indivisible step: the kernel resolves any race internally, so it is impossible for two processes to both be told "it didn't exist, you just created it" for the same file. Concretely, os.open(path, O_CREAT|O_EXCL) either creates the file and returns a valid file descriptor (this process is now the owner), or, if the file already exists, raises OSError with errno.EEXIST and creates nothing, there is no window in between where a second process could sneak in and also succeed. On Unix use os.open with O_EXCL|O_CREAT; on Windows use msvcrt.locking or CreateFile with exclusive flags. Implement acquire(timeout) with retries and stale-lock detection via PID and age.

Implementation (Unix-first, cross-platform fallback)

import os, time, errno
from pathlib import Path

class FileLock:
    def __init__(self, path):
        self.path = Path(path)
    def acquire(self, timeout=10):
        end = time.time()+timeout
        while time.time()<end:
            try:
                fd = os.open(self.path, os.O_CREAT|os.O_EXCL|os.O_WRONLY)
                os.write(fd, str(os.getpid()).encode())
                os.close(fd)
                return True
            except OSError as e:
                if e.errno!=errno.EEXIST: raise
            time.sleep(0.1)
        return False
    def release(self):
        try: self.path.unlink()
        except FileNotFoundError: pass

Worked trace: two processes racing to acquire the same lock, simulated in one process for illustration (the real guarantee comes from the OS's atomic O_CREAT|O_EXCL, not from anything special in this simulation, but the sequence of return values is exactly what would happen with two real, separate processes):

lock_a = FileLock("/tmp/demo.lock")
lock_b = FileLock("/tmp/demo.lock")

print(lock_a.acquire(timeout=1))   # True: lock_a's os.open call wins the race, file now exists
print(lock_b.acquire(timeout=0.3)) # False: lock_b's os.open calls all hit EEXIST and it
                                    # gives up once the 0.3s timeout elapses
lock_a.release()
print(lock_b.acquire(timeout=1))   # True: now that lock_a released (deleted the file),
                                    # lock_b's next os.open call succeeds

The first acquire call's os.open(..., O_CREAT|O_EXCL) either wins outright (file did not exist, now it does, True) or loses outright (EEXIST, False after retries time out); there is no third outcome where both processes believe they created the file, which is exactly the race the atomic system call rules out by construction.

Platform notes

• Unix: O_EXCL is atomic across processes. Use fcntl.flock for advisory locks (a lock that only stops other processes if they also choose to check for it before touching the file; unlike a mandatory, OS-enforced lock, the OS itself does not block a process that simply ignores the lock and opens the file directly) when needed.
• Windows: O_EXCL behaves differently; prefer msvcrt.locking or pywin32 CreateFile for exclusive access.

Stale locks
Check PID from file and whether process exists; remove stale if safe. For production, add jitter, robust error handling, and optional directory-level locking for network filesystems.