Cross-Browser and Cross-Platform Testing Questions

Direct answer

Run the same pinned computation on every target (operating system, CPU architecture, C library, compiler and flags), store the results losslessly, and compare each against one reference platform in units of ULP (unit in the last place, the gap between two neighbouring floating-point numbers). The tolerance depends on what the arithmetic guarantees: addition, subtraction, multiplication, division and square root are required to be correctly rounded by IEEE 754, so any gap there is a bug (budget 0 ULP); exp, log, sin and pow are only recommended to be correctly rounded, so give them a small budget (4 ULP here; the glibc manual states the goal as a result within a few ULP of the exact one); sums and dot products are bounded by an error formula derived from the algorithm. When a result exceeds its budget, find the layer that caused it by elimination (compiler contraction, operation order, math library), then pin that layer, or change the number representation.

Terms used below. IEEE 754 is the standard that defines how computers store and add numbers in binary; binary64 is its 64-bit format, the double. An operation is correctly rounded when its result is the exact mathematical answer rounded to the nearest number that can be stored. Associative means grouping does not matter, (a+b)+c = a+(b+c); floating-point addition is not. A reduction combines many values into one, such as a sum, and a dot product is a sum of pairwise products, x1*y1 + x2*y2 + .... A vectorised loop uses a CPU instruction that handles several numbers at once, each slot being a lane, so its additions happen in a different order from a plain loop. libm is the C math library (sin, exp, pow); glibc and musl are two implementations of the C library on Linux. Unit roundoff, u = 2^-53, is the largest relative error one correctly rounded operation can make. Pairwise summation adds in a balanced tree and compensated summation keeps a running correction term; both shrink the error of a long sum. -O3 asks the compiler for aggressive optimisation and -ffast-math lets it ignore IEEE rules. A relative tolerance allows an error proportional to the value's size, which cannot work when the expected value is 0. volatile in C forces the compiler to read and write a variable in memory instead of folding it away at compile time. nextafter returns the neighbouring storable number. aarch64, x86_64 and s390x are the 64-bit ARM, Intel/AMD and IBM mainframe CPU families; s390x is big-endian.

Two tiny examples first

# tiny.py
import math
from harness import ordered

print((0.1 + 0.2) + 0.3, 0.1 + (0.2 + 0.3))          # same numbers, different grouping

a = 1.0
b = math.nextafter(a, 2.0)                             # the next double above 1.0
print(f"gap above 1.0 = {b - a:.3g} = 2**-52 = {2.0 ** -52:.3g} = math.ulp(1.0) {math.ulp(1.0) == b - a}")
print("ordered(1.0) =", ordered(a), " ordered(next) =", ordered(b), " difference =", ordered(b) - ordered(a))
c = -1.0
d = math.nextafter(c, -2.0)                            # the next double below -1.0
print("ordered(-1.0) =", ordered(c), " ordered(next below) =", ordered(d), " difference =", ordered(c) - ordered(d))
print("ordered(0.0) =", ordered(0.0), " ordered(-0.0) =", ordered(-0.0))
print("ulp(1000.0) =", math.ulp(1000.0), " ulp(1e6) =", math.ulp(1e6))

Run in a python:3.12-slim container, it prints:

0.6000000000000001 0.6
gap above 1.0 = 2.22e-16 = 2**-52 = 2.22e-16 = math.ulp(1.0) True
ordered(1.0) = 4607182418800017408  ordered(next) = 4607182418800017409  difference = 1
ordered(-1.0) = -4607182418800017408  ordered(next below) = -4607182418800017409  difference = 1
ordered(0.0) = 0  ordered(-0.0) = 0
ulp(1000.0) = 1.1368683772161603e-13  ulp(1e6) = 1.1641532182693481e-10

Reading it:

• Line 1: each + rounds its result. Grouping the same three numbers differently rounds different intermediate values, so the sums differ in the last place (0.6000000000000001 against 0.6). A vectorised or multi-threaded loop regroups additions in exactly this way without anyone editing the source.
• Line 2: one ULP at 1.0 is the gap to the next storable number, 2^-52, about 2.22e-16.
• Lines 3 and 4: ordered() in the harness below reads a double's 64 bits as a signed integer (1.0 is the bit pattern 0x3FF0000000000000, which is 4607182418800017408) and flips negative values so the integers increase with the numbers. Neighbouring doubles then differ by exactly 1, so the ULP gap between two results is the absolute difference of their ordered() values. 0.0 and -0.0 both map to 0, so they are 0 ULP apart.
• Last line: an ULP grows with magnitude (1.14e-13 at 1000, 1.16e-10 at a million), which is why a fixed absolute tolerance is the wrong tool and an ULP budget is the right one.

Where platform differences in numeric code come from

Contraction, operation order and the math library are the three layers to eliminate first, because each can change with no edit to the source (a new compiler default, a different thread count, a different C library). Fast-math is a flag the team switches on itself, so it is checked by reading the build flags.

FMA contraction, measured

// fma.c
#include <math.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    volatile double a = atof(argv[1]), b = atof(argv[2]), c = atof(argv[3]);  /* runtime values: nothing is folded at compile time */
    double plain = a * b + c;          /* the compiler may fuse this into one multiply-add */
    double fused = fma(a, b, c);       /* one rounding, always */
    volatile double product = a * b;   /* the volatile store forces the product to be rounded */
    double unfused = product + c;      /* two roundings */
    printf("plain=%.17g fused=%.17g unfused=%.17g\n", plain, fused, unfused);
    return 0;
}

# run_fma.sh
for flags in "-O2" "-O2 -ffp-contract=off" "-O2 -std=c11"; do
  gcc $flags fma.c -o fma -lm && printf '%-24s' "[$flags]" && ./fma 0.1 10 -1
done

Run in gcc:14 containers (GCC 14.4.0), natively on aarch64 and emulated as x86_64:

aarch64
gcc (GCC) 14.4.0
[-O2]                   plain=5.5511151231257827e-17 fused=5.5511151231257827e-17 unfused=0
[-O2 -ffp-contract=off] plain=0 fused=5.5511151231257827e-17 unfused=0
[-O2 -std=c11]          plain=0 fused=5.5511151231257827e-17 unfused=0

x86_64
gcc (GCC) 14.4.0
[-O2]                   plain=0 fused=5.5511151231257827e-17 unfused=0
[-O2 -ffp-contract=off] plain=0 fused=5.5511151231257827e-17 unfused=0
[-O2 -std=c11]          plain=0 fused=5.5511151231257827e-17 unfused=0

The input is 0.1, 10 and -1. Two roundings give 0.1*10 = 1.0 then 1.0 - 1 = 0; one rounding gives 5.55e-17, the true value of the residual. On aarch64 the default -O2 build silently picks the fused answer, and adding -ffp-contract=off or -std=c11 restores 0. On x86_64 the same default build prints 0, so it did not fuse (GCC fuses only where the target has native FMA support). The same source therefore gives different numbers on two CPUs, with no bug in either.

The harness

# harness.py
import json
import math
import random
import struct
import sys

N = 20000
U = 2.0 ** -53                      # unit roundoff of binary64 (round-to-nearest)

# Allowed ULP gap against the reference platform. IEEE 754 requires correctly rounded
# + - * / and sqrt, so any gap there is a bug. exp/log/sin/pow are not required to be
# correctly rounded; the budget below is a few ULP, in line with what glibc documents.
POLICY = {"add": 0, "mul": 0, "div": 0, "sqrt": 0, "exp": 4, "log": 4, "sin": 4, "pow": 4}


def inputs():
    rng = random.Random(7)          # pinned seed: same inputs on every machine
    return ([rng.uniform(-50, 50) for _ in range(N)], [rng.uniform(0.1, 20) for _ in range(N)])


def compute():
    xs, ys = inputs()
    return {
        "add": [x + y for x, y in zip(xs, ys)],
        "mul": [x * y for x, y in zip(xs, ys)],
        "div": [x / y for x, y in zip(xs, ys)],
        "sqrt": [math.sqrt(y) for y in ys],
        "exp": [math.exp(x / 10) for x in xs],
        "log": [math.log(y) for y in ys],
        "sin": [math.sin(x) for x in xs],
        "pow": [math.pow(y, 1.7) for y in ys],
    }


def ordered(x):                      # maps doubles onto integers so that neighbours differ by 1
    i = struct.unpack("<q", struct.pack("<d", x))[0]
    return i if i >= 0 else -(i & 0x7FFFFFFFFFFFFFFF)


def ulp_gap(a, b):
    return abs(ordered(a) - ordered(b))


def compare(golden, candidate):
    ok = True
    for key, budget in POLICY.items():
        gaps = [ulp_gap(float.fromhex(a), b) for a, b in zip(golden[key], candidate[key])]
        worst, differing = max(gaps), sum(1 for g in gaps if g)
        status = "ok" if worst <= budget else "FAIL"
        ok &= worst <= budget
        print(f"{key:5} budget {budget} ULP  worst gap {worst}  elements differing {differing}/{N}  {status}")
    return ok


def reduction_report():
    xs, _ = inputs()
    vals = [x * 1e3 for x in xs]
    naive = 0.0
    for v in vals:
        naive += v
    lanes = [0.0] * 4                # a 4-lane vectorised loop adds in a different order
    for i, v in enumerate(vals):
        lanes[i % 4] += v
    lane_sum = ((lanes[0] + lanes[1]) + (lanes[2] + lanes[3]))
    exact = math.fsum(vals)          # correctly rounded sum, the yardstick
    bound = (N - 1) * U * sum(abs(v) for v in vals)    # worst-case error of a naive left-to-right sum
    ulp = math.ulp(exact)
    print(f"sum|x| = {sum(abs(v) for v in vals):.6g}  n-1 = {N - 1}  u = {U:.3g}")
    print(f"fsum {exact!r}  ulp(result) {ulp:.3g}")
    print(f"naive gap {ulp_gap(naive, exact)} ULP, 4-lane gap {ulp_gap(lane_sum, exact)} ULP")
    print(f"worst-case bound (n-1)*u*sum|x| = {bound:.3g} = {bound / ulp:,.0f} ULP of the result")


if __name__ == "__main__":
    mode = sys.argv[1]
    if mode == "dump":
        json.dump({k: [v.hex() for v in vs] for k, vs in compute().items()}, open(sys.argv[2], "w"))
    elif mode == "compare":
        golden = json.load(open(sys.argv[2]))
        cand = compute()
        if len(sys.argv) > 3 and sys.argv[3] == "inject":      # prove the check can fail
            cand["mul"][0] = math.nextafter(cand["mul"][0], math.inf)           # 1 ULP error in an exact operation
            cand["sin"][0] = math.nextafter(cand["sin"][0], math.inf, steps=5)  # 5 ULP error in a libm function
        print(f"platform byteorder={sys.byteorder}")
        sys.exit(0 if compare(golden, cand) else 1)
    else:
        reduction_report()

Design points: inputs come from a seeded generator so every machine sees identical data; results are stored as float.hex() strings, which are exact, unlike decimal printing; byte order is pinned with < in the ULP conversion; and the comparison reports the worst gap and how many elements differ per function, so a rare single-element difference is visible.

Results across platforms

The reference ("golden") file was produced on aarch64 with glibc (Python 3.12 slim image). Each row compares one other target against it, 20,000 values per function; + - * / sqrt showed 0 differing elements everywhere. Cells give elements that differ (of 20,000); the worst gap in every nonzero cell was 1 ULP, within the 4 ULP budget.

Reading it: exact operations agree on every platform and byte order, as IEEE 754 requires, so byte order is not a source of numeric difference here. The differences sit in library functions and are at most one ULP. The counts read as fractions of 20,000: 630 is 3.15 percent, about one input in 32 where musl's sin rounds the last bit differently from glibc's, and the identical 630 on aarch64 and x86_64 shows the difference follows the library and not the CPU. The exp and pow counts (13 and 8, which are 0.065 and 0.04 percent) are the same on x86_64 glibc and both musl rows, while the s390x glibc row agrees with the reference exactly, so glibc on a different CPU (x86_64) can still differ on a few inputs. The all-zero s390x row means this big-endian target produced the same bits for these 80,000 values. The x86_64 rows ran under emulation, so run the same command on a native x86_64 runner before treating them as final.

The check must be able to fail

Injecting a 1 ULP error into an exact operation and a 5 ULP error into sin (the inject option) makes the comparison fail with exit code 1. The same command sequence then prints the reduction report used in the next section:

platform byteorder=little
add   budget 0 ULP  worst gap 0  elements differing 0/20000  ok
mul   budget 0 ULP  worst gap 1  elements differing 1/20000  FAIL
div   budget 0 ULP  worst gap 0  elements differing 0/20000  ok
sqrt  budget 0 ULP  worst gap 0  elements differing 0/20000  ok
exp   budget 4 ULP  worst gap 0  elements differing 0/20000  ok
log   budget 4 ULP  worst gap 0  elements differing 0/20000  ok
sin   budget 4 ULP  worst gap 5  elements differing 1/20000  FAIL
pow   budget 4 ULP  worst gap 0  elements differing 0/20000  ok
exit=1
sum|x| = 5.00813e+08  n-1 = 19999  u = 1.11e-16
fsum -4428365.074153088  ulp(result) 9.31e-10
naive gap 20 ULP, 4-lane gap 8 ULP
worst-case bound (n-1)*u*sum|x| = 0.00111 = 1,193,972 ULP of the result

Variance in sums

The report shows a naive left-to-right sum is 20 ULP from the correctly rounded sum (math.fsum, which tracks multiple partial sums), and a 4-lane order (what a vectorised loop does) is 8 ULP away. The naive sum's worst-case error grows in proportion to $n\varepsilon\sum|x_i|$ (Wikipedia, pairwise summation); with $n = 20000$ terms (so a factor $n-1$) and unit roundoff $u=2^{-53}$ (the largest relative error of one correctly rounded operation) the script's bound is the 1,193,972 ULP printed, far above anything observed because real errors partly cancel.

Derivation: a left-to-right sum of n values does n-1 additions, each adding at most a relative error u to a partial sum no larger than sum|x|, so the worst-case total error is about (n-1) * u * sum|x|. The report prints sum|x| = 5.00813e+08 (20,000 values averaging about 25,000 in size), n-1 = 19,999 and u = 1.11e-16, so the bound is 19,999 x 1.11e-16 x 5.00813e8 = 1.11e-3. The sum itself, about -4.43 million, lies between 2^22 and 2^23, so its ULP is 2^(22-52) = 2^-30 = 9.31e-10, and 1.11e-3 / 9.31e-10 is about 1.19 million ULP.

How much variance is acceptable

1. Start from the requirement. If results are hashed, replayed or compared bit for bit (financial ledgers, consensus, golden files), the budget is zero and you must pin everything. If a downstream decision tolerates 1e-9 relative error, that is the ceiling.
2. Use the algorithm's bound as the "this is a bug" line. An observed gap above the analytic bound cannot be rounding.
3. Set the alarm well below the bound from measurement. Here: 0 ULP for exact operations, 4 ULP for library functions (observed worst is 1), and for reductions take the largest gap seen between sensible orderings (20 ULP) and multiply by 4 for headroom, so the alarm is 80 ULP. The factor of 4 is a judgement: it is wide enough that another thread count or a new data set does not trip it, and the alarm is still about 15,000 times below the 1,193,972 ULP bound, so a real defect such as a dropped term or a wrong precision does. Review it whenever the data changes. Prefer ULP gaps to relative tolerance: Python's math.isclose has abs_tol=0.0 by default, so a comparison against zero fails for any nonzero value unless you pass an absolute floor.

What to do when it is not acceptable

1. Save the first failing input as hex floats and the two outputs, so the case is reproducible without the full run.
2. Eliminate layers: rebuild with -ffp-contract=off (gap vanishes means contraction); fix the reduction order explicitly (a fixed pairwise tree independent of thread count or vector width); run the same input through a different libm (musl against glibc as above).
3. Remedy the layer, cheapest first: pin the flags in the build, vendor one math implementation for all platforms, or accumulate with a compensated or exact method such as math.fsum. If the values must be exactly reproducible, use integers, fixed point or decimal instead of binary floats.
4. If the gap is above the analytic bound, treat it as a defect (overflow, uninitialised data, wrong precision), not as noise, and do not widen the budget.

Trade-offs

A tight budget on library functions makes the suite fail on harmless one-ULP library updates; a loose one hides real bugs. Pin the C library version in CI images and revisit the budget on upgrades. Emulated architectures are slow but are good enough for these small deterministic suites.

Running the code

# save fma.c, run_fma.sh, harness.py and tiny.py in one folder, then:
docker run --rm --ulimit core=0 --platform linux/arm64 -v "$PWD":/w -w /w python:3.12-slim \
  sh -c 'timeout 100 python harness.py dump golden.json'            # reference, 3.9 MB
docker run --rm --ulimit core=0 --platform linux/s390x -v "$PWD":/w -w /w python:3.12-slim \
  sh -c 'timeout 150 python harness.py compare golden.json'         # any other target
docker run --rm --ulimit core=0 --platform linux/arm64 -v "$PWD":/w -w /w python:3.12-alpine \
  sh -c 'timeout 150 python harness.py compare golden.json'         # musl row (use linux/amd64 for the x86_64 musl row)
docker run --rm --ulimit core=0 -v "$PWD":/w -w /w python:3.12-slim \
  sh -c 'python tiny.py; python harness.py compare golden.json inject; echo "exit=$?"; python harness.py reduce'
docker run --rm --ulimit core=0 --platform linux/arm64 -v "$PWD":/w -w /w gcc:14 \
  sh -c 'uname -m; gcc --version | head -1; timeout 100 sh run_fma.sh'

Direct answer

Five families of platform difference break file and binary-data code: how paths and names are written and what is legal, whether names are case- and Unicode-sensitive, what text bytes are produced (encoding and line endings), how binary values are laid out (byte order, size, padding), and what the file system lets you do with open files and long paths. Defend against them by making the product's output explicit rather than platform-default (UTF-8, \n, little-endian standard-size struct formats), by running one portability suite on native Linux, Windows and macOS runners in CI, and by adding a big-endian job through emulation so byte-order assumptions fail loudly.

Terms used below. A code point is the number Unicode assigns to a character. A combining accent is a separate code point drawn over the letter before it, so é can be one code point (U+00E9) or two (e then U+0301); NFC (Normalization Form C) is the Unicode rule that merges such a pair into the single composed code point. Byte order (endianness) is the order a multi-byte number is stored in: little-endian stores the least significant byte first, so the 4-byte number 3 is 03 00 00 00; big-endian stores it last, 00 00 00 03. The locale encoding is the text encoding the operating system's settings pick when a program does not name one. A golden file is a stored known-good output that a test compares byte for byte, and a golden hex string is the same bytes written as hex digits inside the test. aarch64 is 64-bit ARM, s390x is IBM's mainframe CPU (big-endian), and QEMU is an emulator that lets one machine run programs built for another CPU. APFS is Apple's file system. MAX_PATH is the 260-character path limit of older Windows APIs, and the \\?\ prefix is a path spelling that asks Windows to skip that limit.

What differs, and the test that catches it

Names, case and text defaults (encoding and line endings) are the differences most teams meet first, because every developer laptop and the CI runner differ in them every day. Byte order and the 260-character limit appear later, when files move between machine types or the output directory gets deep. This ordering is a judgement from how the environments differ, not a measured count.

The portability suite

One small module with the product's rules, and one pytest file that checks each row. The Windows rules are checked on any OS through PureWindowsPath, a path class that applies Windows semantics without touching the disk.

# portability.py
import csv
import struct
import unicodedata
from pathlib import PureWindowsPath

BAD_CHARS = set('<>:"/\\|?*')


def portable_name(name):
    """Return a file name that is legal on Linux, macOS and Windows, or raise ValueError."""
    n = unicodedata.normalize("NFC", name)          # one spelling of "é", not two
    if not n or any(c in BAD_CHARS or ord(c) < 32 for c in n):
        raise ValueError(f"reserved character in {name!r}")
    if n.endswith((" ", ".")):
        raise ValueError(f"trailing space or period in {name!r}")
    if PureWindowsPath(n).is_reserved():              # CON, NUL, COM1 ... even as NUL.txt
        raise ValueError(f"reserved device name {name!r}")
    return n


def case_collisions(names):
    """Names that would be the same file on a case-insensitive file system."""
    seen = {}
    for n in names:
        seen.setdefault(unicodedata.normalize("NFC", n).casefold(), []).append(n)
    return [group for group in seen.values() if len(group) > 1]


def write_csv(path, rows):
    # Explicit encoding and newline: output bytes no longer depend on the OS or locale.
    with open(path, "w", encoding="utf-8", newline="") as f:
        csv.writer(f, lineterminator="\n").writerows(rows)


def pack_header(version, count):
    return struct.pack("<HI", version, count)         # standard sizes, little-endian, no padding


def check_path_budget(path, limit=250):
    """Fail early on paths that would approach the 260-character Windows MAX_PATH."""
    if len(str(path)) > limit:
        raise ValueError(f"path is {len(str(path))} characters, budget is {limit}")
    return path

# test_portability.py
import locale
import os
import struct
import sys
from pathlib import PurePosixPath, PureWindowsPath

import pytest

from portability import case_collisions, check_path_budget, pack_header, portable_name, write_csv


@pytest.mark.parametrize("bad", ["NUL.txt", "con", "COM1.log", "a:b.csv", "what?.csv", "a|b.csv", "x*y.csv", "tab\there.csv", "name.", "name "])
def test_names_illegal_somewhere_are_rejected_everywhere(bad):
    with pytest.raises(ValueError):
        portable_name(bad)


def test_unicode_spellings_collapse_and_case_variants_are_flagged():
    composed, decomposed = "caf\u00e9.csv", "cafe\u0301.csv"
    assert portable_name(decomposed) == composed
    assert case_collisions(["Report.csv", "report.csv", "other.csv"]) == [["Report.csv", "report.csv"]]


def test_this_file_system_probe(tmp_path):
    (tmp_path / "a.txt").write_text("1")
    (tmp_path / "A.txt").write_text("2")
    (tmp_path / "caf\u00e9").write_text("3")
    (tmp_path / "cafe\u0301").write_text("4")
    print(f"\nsep={os.sep!r} byteorder={sys.byteorder} locale_encoding={locale.getencoding()} "
          f"files_after_a/A_and_two_cafe_spellings={len(os.listdir(tmp_path))}")


def test_csv_bytes_are_identical_on_every_os(tmp_path):
    out = tmp_path / "out.csv"
    write_csv(out, [["id", "name"], [1, "caf\u00e9"]])
    assert out.read_bytes() == b"id,name\n1,caf\xc3\xa9\n"
    legacy = tmp_path / "legacy.csv"           # what a Windows-locale default produces
    with open(legacy, "w", encoding="cp1252", newline="\r\n") as f:
        f.write("id,name\n1,caf\u00e9\n")
    print("\nportable:", out.read_bytes(), "\nwindows-style:", legacy.read_bytes())
    assert out.read_bytes() != legacy.read_bytes()


def test_binary_header_has_fixed_layout():
    print("\n<HI ->", pack_header(1, 3).hex(), struct.calcsize("<HI"),
          "| native @HI ->", struct.pack("@HI", 1, 3).hex(), struct.calcsize("@HI"),
          "| >HI ->", struct.pack(">HI", 1, 3).hex())
    assert pack_header(1, 3).hex() == "010003000000"
    assert struct.unpack("<HI", bytes.fromhex("010003000000")) == (1, 3)


def test_windows_path_rules_checked_from_any_os():
    a, b = PureWindowsPath("C:/Data/Out.CSV"), PureWindowsPath("c:/data/out.csv")
    p, q = PurePosixPath("/data/Out.CSV"), PurePosixPath("/data/out.csv")
    print(f"\nwindows paths equal: {a == b}; posix paths equal: {p == q}")
    assert a == b and p != q


def test_output_path_budget():
    assert check_path_budget("/out/" + "x" * 100 + "/report.csv")
    with pytest.raises(ValueError):
        check_path_budget("/out/" + "x" * 300 + "/report.csv")


def test_no_handle_is_left_open_after_writing(tmp_path):
    out = tmp_path / "out.csv"
    write_csv(out, [["id"], [1]])
    replacement = tmp_path / "new.csv"
    write_csv(replacement, [["id"], [2]])
    os.replace(replacement, out)       # replace over an existing file
    os.remove(out)                     # delete straight after writing
    assert not out.exists()

Run in python:3.12-slim containers, once natively on aarch64 (little-endian) and once on s390x (big-endian, emulated with QEMU), it prints on aarch64 (verbatim; the time in the last line varies from run to run):

...........
sep='/' byteorder=little locale_encoding=UTF-8 files_after_a/A_and_two_cafe_spellings=4
.
portable: b'id,name\n1,caf\xc3\xa9\n' 
windows-style: b'id,name\r\n1,caf\xe9\r\n'
.
<HI -> 010003000000 6 | native @HI -> 0100000003000000 8 | >HI -> 000100000003
.
windows paths equal: True; posix paths equal: False
...
17 passed in 0.01s

and on s390x (also 17 passed) the lines that differ are:

sep='/' byteorder=big locale_encoding=UTF-8 files_after_a/A_and_two_cafe_spellings=4
<HI -> 010003000000 6 | native @HI -> 0001000000000003 8 | >HI -> 000100000003

How to read it:

• Byte by byte, struct.pack("<HI", 1, 3) is: H is a 2-byte unsigned number, so 1 becomes 01 00; I is a 4-byte unsigned number, so 3 becomes 03 00 00 00; with no padding the 6 bytes are 01 00 03 00 00 00, printed 010003000000. The native @HI format also aligns the 4-byte field to a multiple of 4, so 2 padding bytes (00 00) are inserted after the H: 01 00 + 00 00 + 03 00 00 00, 8 bytes in all. On a big-endian machine the same native layout is 00 01 + 00 00 + 00 00 00 03. >HI is 00 01 + 00 00 00 03.
• The explicit formats are identical on both machines: <HI is 010003000000 (6 bytes), >HI is 000100000003. The native format @HI is 8 bytes (two pad bytes) and is 0100000003000000 on little-endian but 0001000000000003 on big-endian. Any code that writes native layout to a file is wrong the moment the file moves.
• The CSV written with encoding="utf-8", newline="" and an explicit \n terminator is b'id,name\n1,caf\xc3\xa9\n' everywhere; the "Windows-style" bytes (\r\n, and \xe9 from the cp1252 code page) show what unpinned defaults can produce. The cp1252 case is simulated, because a Windows runner is needed to see a real locale default.
• The open-handle row has no output to read because on Linux nothing fails. The test passes since write_csv closes its file with with. If someone changed it to keep the file object open, the same test would still pass on Linux but the os.replace or os.remove line would raise PermissionError on the Windows runner, which is the only place that regression shows.
• On the Linux volume used here all four probe files exist (a/A, and both spellings of café), so Linux accepts what other file systems may merge; the product-level collision check is what protects it.

Automating it in CI

# .github/workflows/portability.yml
name: portability
on: [pull_request]
jobs:
  native:
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install pytest==8.3.3
      - run: python -m pytest -q -s
  big-endian:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-qemu-action@v3
      - run: >
          docker run --rm --platform linux/s390x -v "$PWD":/w -w /w python:3.12-slim
          sh -c "pip install -q pytest==8.3.3 && python -m pytest -q"

• GitHub's hosted runner labels include ubuntu-latest (x64), windows-latest (x64) and macos-latest (arm64, Apple M1 per the runner reference), plus arm64 Linux labels such as ubuntu-24.04-arm; the matrix therefore also covers x64 versus arm64 for free. Pin the action versions to the current releases.
• Treat red on any one OS as a blocker; fail-fast: false lets all three report.
• Keep golden files from being rewritten by checkout: mark them binary in .gitattributes so Windows line-ending conversion cannot change the bytes you compare.
• Run the same suite locally through the Linux container so developers see the Linux result before CI.

Trade-offs

Emulated big-endian is slow (use it for the small byte-layout suite only) and covers a rare target; if the product never runs on big-endian hosts the real requirement is a file format that is defined as little-endian, which the golden-bytes test enforces. Windows-only behaviours (reserved names, path length, locked files) can only be fully proven on a Windows runner, which is why the native matrix stays even though PureWindowsPath checks run anywhere.

Running the code

# put portability.py and test_portability.py in one folder, then (Linux or emulated big-endian):
docker run --rm --ulimit core=0 --platform linux/s390x -v "$PWD":/w -w /w python:3.12-slim sh -c \
  'pip install -q pytest==8.3.3 && timeout 150 python -m pytest -q -s -p no:cacheprovider'
# drop --platform for the host architecture

Direct answer

On a subscription product with a React web app and a React Native mobile app, I led the move from two separate implementations to one shared TypeScript package for the logic that has no platform in it (pricing, validation, date handling, API types), while keeping everything that touches the screen, storage or navigation separate per platform. I tested the shared package once in Node, the web app across Chromium, Firefox and WebKit, and the mobile app on an iOS simulator and an Android emulator, and each part ran in CI only when its inputs changed. The compatibility bug that made the case for the whole approach was a date that shifted by the user's time zone offset.

Situation

The web and mobile teams each had their own copy of the pricing rules and the form validation. They drifted: the invoice preview on the phone and the one on the web occasionally disagreed by a rounding rule, and each fix landed twice, weeks apart. My task was to remove the duplication without slowing either team, and to make cross-platform testing a routine step instead of something found by customers.

What I decided to share and what I did not

The rule I used: share what is a pure function of its inputs; do not share what calls a platform API. I tried sharing a component library first and dropped it within the first iteration, because the web and native primitives differ enough that the abstraction cost more than it saved. For the few places where behaviour had to differ I used a file-name convention rather than if statements: React Native documents that it picks .ios. and .android. files automatically, and .native. files for code shared between React Native and the web, so session-store.native.ts and session-store.ts sat side by side behind one interface, and I told the web bundler (the build tool that packs the web app's code into files for the browser) to ignore .native files so the web bundle did not carry unused code.

How I tested across environments

1. Shared package: unit tests run once in Node, plus a file of golden pricing cases (fixed input and expected-price pairs that both apps' tests read) that the web tests and the mobile tests both read, so a disagreement between the apps fails a build.
2. Web: Playwright with three browser projects (Chromium, Firefox, WebKit) at a desktop viewport and a phone viewport, so six runs per change. I picked which browsers got the full suite from our own traffic data and gave the others a smoke set. Because Playwright's WebKit is derived from upstream WebKit and is not the Safari release, I also kept a small real-Safari check on a real iPhone for the release candidate.
3. Mobile: component tests on every change, and end-to-end flows on one iOS simulator and one Android emulator per merge, with a short list of real devices before release.

Build and CI

A single monorepo (one repository holding several packages) with workspaces (the package manager feature that links those packages together locally): packages/shared, apps/web, apps/mobile. A change under packages/shared ran every job; a change under one app ran only that app's jobs, with dependency caching on both. The shared package was versioned with the repository, so there was never a question of which version an app was built against.

A compatibility problem I solved

The API returned timestamps like 2019-01-01T00:00:00 with no time zone marker, and both apps called Date.parse on them. Date.parse returns the number of milliseconds since the Unix epoch, the moment 00:00 UTC on 1 January 1970, and the script below divides by 3,600,000 to print hours. To check one value by hand: 2019-01-01 is 49 years after the epoch, including 12 leap years (1972 to 2016), so 49 x 365 + 12 = 17,897 days, and 17,897 x 24 = 429,528 hours. Midnight in Los Angeles in January is 08:00 UTC, so a time read as Los Angeles local time is 8 hours later: 429,536. The shared package got a single parsing helper and a test that I proved with this script in the three Playwright engines, with the browser's time zone set to Los Angeles (UTC-8 in January):

// dates.mjs
import { chromium, firefox, webkit } from 'playwright';

const inputs = ['2019-01-01', '2019-01-01T00:00:00', '2019-01-01 00:00', '1970/01/01', '2019-01-01T00:00:00Z'];

for (const [name, type] of [['chromium', chromium], ['firefox', firefox], ['webkit', webkit]]) {
  const browser = await type.launch();
  const context = await browser.newContext({ timezoneId: 'America/Los_Angeles' });   // UTC-8 in January
  const page = await context.newPage();
  const result = await page.evaluate((list) => list.map((s) => [s, Date.parse(s)]), inputs);
  console.log(`${name.padEnd(9)}` + result.map(([s, v]) => `${JSON.stringify(s)}=${Number.isNaN(v) ? 'NaN' : v / 3600000 + 'h'}`).join('  '));
  await browser.close();
}

Run in the Playwright 1.48.2 container (Chromium 130, Firefox 131, WebKit 18.0), it printed the same lines on each of 10 runs:

chromium "2019-01-01"=429528h  "2019-01-01T00:00:00"=429536h  "2019-01-01 00:00"=429536h  "1970/01/01"=8h  "2019-01-01T00:00:00Z"=429528h
firefox  "2019-01-01"=429528h  "2019-01-01T00:00:00"=429536h  "2019-01-01 00:00"=429536h  "1970/01/01"=8h  "2019-01-01T00:00:00Z"=429528h
webkit   "2019-01-01"=429528h  "2019-01-01T00:00:00"=429536h  "2019-01-01 00:00"=429536h  "1970/01/01"=8h  "2019-01-01T00:00:00Z"=429528h

Values are hours since the Unix epoch, so 429536 minus 429528 is the 8-hour Los Angeles offset. A date-only string (2019-01-01) and an explicit Z string are read as UTC (429528 h), but a date-time string with no offset is read as local time and lands 8 hours later (429536 h). That is the ECMAScript (the language standard JavaScript follows) rule MDN documents: date-only forms are UTC, date-time forms without an offset are local. So the same API value meant a different instant for every user depending on their time zone. The fix had two parts: the API now sends an explicit Z or offset, and the shared helper rejects strings without one and is the only place dates are parsed. MDN also documents that formats outside the standard ones, such as 2019-01-01 00:00 and 1970/01/01, are implementation-defined and "may not work across all browsers". All three engines here parsed both of them the same way, which says nothing about Safari: Playwright's WebKit is derived from upstream WebKit but is not the Safari release, so a Safari-only parsing difference would not show in this lane, and that is why the real-device check stayed.

The second problem was session storage. The browser keeps the web session in a cookie or web storage and the phone keeps it in the platform's credential storage, and the shared API client had been calling one of them directly. I put a small SessionStore interface in the shared package with two implementations picked by file name, tested the API client against an in-memory fake, and tested each real implementation only in its own environment (a Playwright test on web, an end-to-end flow on the simulator and emulator).

Result

• Pricing and validation went from two implementations to one, so a fix lands once.
• Both apps' test suites read the same golden pricing cases, and the invoice-preview disagreements stopped appearing in testing.
• The time-zone bug has a regression test that runs in three browser engines at once.
• Each pull request ran only the jobs affected by its changed paths, so the extra matrix did not make every change wait for every environment.

What I would do differently

I would add the real-Safari check on day one instead of after the first Safari-specific report, and write the golden-case contract tests before extracting the shared package so the extraction was proven by them, not followed by them. I would also stop trying to share UI components earlier, since I spent a sprint on an abstraction I then removed.

Pitfalls when telling this story

Name the rule you used to decide what to share, not just what you shared. Say which browsers got the full suite and why (traffic), and what you could not cover (real devices) and how you compensated. Keep every claim about results to what you actually measured or counted.

Running the code

# save dates.mjs and a package.json containing {"type":"module"} in one folder, then:
docker run --rm --ulimit core=0 -v "$PWD":/w -w /w mcr.microsoft.com/playwright:v1.48.2-jammy sh -c \
  'npm i --silent playwright@1.48.2 && timeout 100 node dates.mjs'