Computer Vision Questions

Direct answer

Split the one million images by photographer (so no photographer appears on both sides), fine-tune a pretrained image backbone at a resolution high enough to keep fine detail, use only augmentations that cannot change the quality label, handle the class imbalance with the metric and the decision threshold rather than by trusting accuracy, and get the "actionable feedback" from a second part of the system: separate attribute outputs for things a photographer can change (sharpness, exposure, composition) plus localization maps showing where the problem is. A single "professional vs amateur" score cannot tell anyone what to fix, because it only says that the photo resembles one group.

Terms. Transfer learning: start from a network pretrained on a large image collection and continue training on your task, instead of starting from random weights. Fine-tuning: continuing training of the pretrained weights. Augmentation: random changes to training images (flip, crop, color change) that should leave the label unchanged. Imbalance: one class being much rarer than the other. Precision: of the photos the model calls professional, the share that really are. Recall: of the photos that really are professional, the share the model finds. F1: one number combining the two (their harmonic mean, equal to $2TP / (2TP + FP + FN)$, where TP, FP and FN count true positives, false positives and false negatives). Prevalence: how common the rare class is (5 percent here). Threshold: the score above which the model says "professional". PR-AUC: area under the precision-recall curve (sweep the threshold, plot precision against recall), a score of how well the model ranks the rare class; a model that ranks at random scores about the prevalence, not 0.5. Head: the last small layer or layers of a network that turn its features into an output; "attribute heads" are extra heads, one per attribute.

1. Splits

• Group by photographer, and by near-duplicate. Images from one photographer share camera, editing style and subject matter; the label is often almost a property of the photographer. A random split lets the model recognize the photographer, not the quality. Also group exact and near-duplicate images (for example re-uploads) and keep them on one side.
• Three sets: train, validation (model and threshold selection), and a test set of photographers never seen, touched once. If the product will see new uploads over time, make the test set the most recent period.
• Label quality check. "Professional" is a proxy for quality (it describes the photographer, not each image). Have people rate a sample of images for the attributes you will give feedback on; this small set is also your most trustworthy evaluation.

2. Augmentation: only what cannot flip the label

Quality is carried by exactly the properties that common augmentations alter. Blur, downscaling and re-upscaling, JPEG compression (lossy saving that adds blocky or smeared artifacts, which can themselves be a quality cue), strong brightness changes and aggressive crops remove or fake the cues (sharpness, exposure, composition) the model should learn. Keep: horizontal flip, small-range color jitter, and mild crops that keep the composition, at a fixed training resolution chosen so that fine detail survives. If you want robustness to compression or resizing, apply them to BOTH classes at test time, as a separate robustness evaluation, not as free augmentation.

3. Transfer learning

Start with a pretrained backbone (the convolutional or transformer feature extractor), replace the head with a new classifier, train the head first with the backbone frozen, then unfreeze and train all with a smaller learning rate for the backbone than for the head. With a million labeled images, fine-tuning the whole backbone is usually feasible for a mid-sized network; whether it beats tuning only the top layers is something to check on the validation set, and a very large backbone may need more compute or regularisation than the data alone justifies. Use the highest input resolution the compute allows, since downscaling erases sharpness cues. If cost forces a smaller input, add a second branch: take a few small crops (for example 224 by 224 pixels each) straight from the full-resolution photo so no pixels are shrunk, pass them through a small network to get sharpness features, and join those features to the whole-image features just before the final classifier.

4. Imbalance

Say the real prevalence of professional photos (the demo assumes 5 percent). A model that always answers "amateur" gets accuracy equal to the majority share, so accuracy is useless; report PR-AUC, and precision and recall at the chosen threshold. The program prints the baseline that makes PR-AUC readable: random scores get 0.050 (the prevalence), so the model's 0.316 is about six times better than chance but far from perfect. Reweighting the loss or resampling changes the probabilities the model outputs, and you still need a threshold chosen on the validation set from the cost of each mistake.

5. Actionable feedback

• Attribute heads. Add outputs (trained on the human-rated set, with the backbone shared) for sharp vs blurred, well-exposed vs over or under-exposed, and composition. Feedback is then "this photo is soft" rather than "your photo scored 0.31".
• Measurable cues for exactness. Some attributes have direct measures: the variance of the Laplacian (a second-derivative filter that responds to edges) is a standard sharpness cue, and the share of near-black or near-white pixels measures clipped exposure. They are cheap, explainable, and a sanity check on the learned heads.
• Where. Gradient-based heat maps (Grad-CAM, which colours each region by how strongly its features push the network's score for the chosen class up; features that push it down are clipped to zero by a ReLU, so the map shows only supporting evidence) or an attention map (the weights a transformer-style backbone assigns to each image patch) over the image to show the region driving the score. They indicate where the model looked, not necessarily what is wrong, so use them as a hint and validate with people.
• Suggestions come from attributes, not from the class score. Test the feedback on photographers: do they accept it, and does re-shooting improve the attribute?

Runnable demonstrations

All three are synthetic stand-ins. Part 1 builds 150 photographers with 30 images each; each photographer has a style vector that shifts all their features, the label is a property of the photographer, and there is only a weak image-level signal; a random forest (an ensemble of decision trees) is cross-validated with a random split (KFold, which deals images into folds with no regard for who took them) and with a split by photographer (GroupKFold, which keeps every image of a photographer in the same fold, so the test fold contains only unseen photographers). Part 2 uses 5 percent positives in a 5-feature Gaussian problem with logistic regression. The line that picks the threshold tries every 50th distinct validation score as a cut-off, computes F1 for each (f1_at), and keeps the best; it uses validation data only, so the test set stays untouched. class_weight="balanced" makes each rare-class error count more in the training loss. Part 3 builds a texture image, verifies a hand-written Laplacian against scipy.ndimage.laplace, and measures what the "harmless" operations do to the cues.

import numpy as np
from scipy.ndimage import gaussian_filter, laplace, zoom
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score
from sklearn.model_selection import GroupKFold, KFold, cross_val_score

# ---- 1) Splits: the same photographer must not sit on both sides ----
def photographers(seed, n_people=150, per=30, d=20):
    g = np.random.default_rng(seed)
    style = g.normal(0, 2.0, size=(n_people, d))                  # each photographer's camera / editing signature
    pro = g.random(n_people) < 0.5                                # the label is mostly a property of the photographer
    X, y, grp = [], [], []
    for p in range(n_people):
        quality = 0.8 * (1 if pro[p] else -1)                     # weak, genuinely image-level quality signal
        X.append(style[p] + quality * np.eye(d)[0] + g.normal(0, 1.0, size=(per, d)))
        y += [int(pro[p])] * per; grp += [p] * per
    return np.vstack(X), np.array(y), np.array(grp)

rnd, grouped = [], []
for seed in range(3):
    X, y, grp = photographers(seed)
    rf = RandomForestClassifier(n_estimators=60, random_state=0, n_jobs=1)
    rnd.append(cross_val_score(rf, X, y, cv=KFold(4, shuffle=True, random_state=0)).mean())
    grouped.append(cross_val_score(rf, X, y, cv=GroupKFold(4), groups=grp).mean())
print(f"accuracy, random split {np.mean(rnd):.3f} vs split by photographer {np.mean(grouped):.3f} (3 seeds; same data, same model)")

# ---- 2) Imbalance: 5% professional ----
g = np.random.default_rng(0)
def imbalanced(n):
    y = (g.random(n) < 0.05).astype(int)
    return g.normal(0, 1, size=(n, 5)) + y[:, None] * np.array([1.2, 0.8, 0, 0, 0]), y
Xtr, ytr = imbalanced(20000); Xva, yva = imbalanced(20000); Xte, yte = imbalanced(20000)
print(f"\nalways predicting 'amateur': accuracy {1 - yte.mean():.3f}, recall of professionals 0.000")
print(f"test prevalence of professionals {yte.mean():.3f}; PR-AUC of random scores {average_precision_score(yte, g.random(len(yte))):.3f}")
for name, cw in (("plain", None), ("class_weight=balanced", "balanced")):
    m = LogisticRegression(class_weight=cw, max_iter=1000).fit(Xtr, ytr)
    s = m.predict_proba(Xte)[:, 1]; sv = m.predict_proba(Xva)[:, 1]
    def f1_at(t):                                                 # F1 = 2 TP / (2 TP + FP + FN) = 2 TP / (flagged + real positives)
        tp = ((sv >= t) & (yva == 1)).sum()
        return 2 * tp / ((sv >= t).sum() + yva.sum())
    thr = max(np.unique(sv)[::50], key=f1_at)                     # try every 50th distinct validation score, keep the F1-best
    pred = s >= thr
    rec = pred[yte == 1].mean(); prec = yte[pred].mean()
    print(f"{name:22s} accuracy at 0.5 {((s>=.5)==yte).mean():.3f} | PR-AUC {average_precision_score(yte, s):.3f} | at the validation-F1 threshold: precision {prec:.2f}, recall {rec:.2f}")

# ---- 3) Actionable feedback needs measurable cues, and augmentation must not change them ----
def lap_var(img):                                                 # variance of the Laplacian: a standard sharpness cue
    return float(laplace(img).var())
def clipped(img, lo=0.02, hi=0.98):                               # share of pixels near pure black / pure white
    return float(((img <= lo) | (img >= hi)).mean())
rng = np.random.default_rng(0)
sharp = np.clip(gaussian_filter(rng.random((128, 128)), 0.7) * 1.6 - 0.3, 0, 1)
k = np.array([[0, 1, 0], [1, -4, 1], [0, 1, 0]], float)           # 5-point Laplacian by hand
pad = np.pad(sharp, 1, mode="symmetric")
manual = sum(k[i, j] * pad[i:i + 128, j:j + 128] for i in range(3) for j in range(3))
print("\nhand-written Laplacian equals scipy.ndimage.laplace:", bool(np.allclose(manual, laplace(sharp))))
down_up = zoom(zoom(sharp, 0.5, order=1), 2.0, order=1)
print(f"sharpness (Laplacian variance): original {lap_var(sharp):.4f}, blur sigma 1 {lap_var(gaussian_filter(sharp, 1)):.4f}, "
      f"blur sigma 2 {lap_var(gaussian_filter(sharp, 2)):.4f}, 2x down-then-up resize {lap_var(down_up):.4f}")
print(f"clipped-pixel share: original {clipped(sharp):.3f}, brightness x1.5 {clipped(np.clip(sharp * 1.5, 0, 1)):.3f}")

Output:

accuracy, random split 0.951 vs split by photographer 0.556 (3 seeds; same data, same model)

always predicting 'amateur': accuracy 0.949, recall of professionals 0.000
test prevalence of professionals 0.051; PR-AUC of random scores 0.050
plain                  accuracy at 0.5 0.951 | PR-AUC 0.316 | at the validation-F1 threshold: precision 0.28, recall 0.49
class_weight=balanced  accuracy at 0.5 0.768 | PR-AUC 0.316 | at the validation-F1 threshold: precision 0.29, recall 0.48

hand-written Laplacian equals scipy.ndimage.laplace: True
sharpness (Laplacian variance): original 0.1647, blur sigma 1 0.0080, blur sigma 2 0.0003, 2x down-then-up resize 0.0188
clipped-pixel share: original 0.004, brightness x1.5 0.227

Reading it. The same data and model score 0.951 under a random split and 0.556 under a split by photographer. The gap is exaggerated by construction here (the label was built to be mostly a photographer property), but the direction is the real risk: the random-split number measures photographer recognition. On the imbalanced data, always answering "amateur" is 0.949 accurate with zero recall; the plain logistic regression's 0.951 is barely above that, and its PR-AUC is 0.316, against 0.050 for random scores. Class weighting dropped accuracy at the 0.5 cut-off to 0.768 but left PR-AUC unchanged at 0.316, and at a threshold chosen on validation they give precision 0.28 and 0.29 and recall 0.49 and 0.48 on this draw. On other random draws of the same construction they land anywhere from about 0.26 to 0.36 precision and 0.32 to 0.52 recall, so read them as roughly one flagged photo in three being truly professional (against one in twenty by chance) and somewhere between a third and a half of the professionals found. The weights moved the default 0.5 cut-off and left the ranking, which is what PR-AUC measures, almost unchanged (the two PR-AUC values stay within about 0.002 of each other across other random draws). Whether 0.28 precision is usable depends on the product: for a 'ready to publish' badge, where a wrong badge is costly, it is too low, and this toy is hard on purpose (the features carry only a weak signal), so these values show how to read the metrics rather than what a good photo-quality model scores. In part 3 the Laplacian variance falls from 0.1647 to 0.0080 under a blur with sigma 1, and to 0.0188 under a 2x down-then-up resize, so those operations change the sharpness label of a photo; a 1.5x brightness change raises the clipped-pixel share from 0.004 to 0.227, changing the exposure label.

Trade-offs and mistakes

• The model learns the proxy. If professionals shoot certain subjects (weddings, studio), the model may classify the subject. Check by evaluating within subject categories, and by looking at what the attribute heads say rather than the overall score.
• Fairness and harm. Telling a photographer they are "amateur" is a judgment; present feedback as attribute guidance, and audit by camera type (phone vs professional camera), since equipment correlates with the label.
• Common mistakes. Random splits, accuracy as the headline number, blur and resize augmentation, a score with no explanation, and tuning the threshold on the test set.

Direct answer

Consistent labels at scale come from three things working together: written instructions that settle every ambiguous case before annotators meet it, measurement that tells you which annotators and which images disagree (gold tasks, overlap, review), and a store that keeps every label's author, version and format. Treat labeling as a measured process, not a one-time hand-off.

Terms used below. A bounding box is a rectangle around an object, stored as corner coordinates. A polygon is a closed outline of the object, stored as a list of points; rasterized (filled in on a pixel grid) it becomes a mask. IoU (intersection over union) is the overlap area of two shapes divided by their combined area, from 0 (disjoint) to 1 (identical). A gold task is an item whose correct label is already known, mixed secretly into the normal queue.

Labeling instructions

Write the guide as decisions with pictures, not prose. One entry of a guide, shown as an illustration (the image ids are placeholders for real example images):

CLASS: pallet  (flat wooden or plastic platform that goods sit on)
Draw:      a box. Include the visible blocks and slats. Exclude goods stacked on top and the shadow.
Positive:  [ex-pallet-01] pallet empty on the floor      [ex-pallet-02] pallet carrying boxes (box the pallet only)
Negative:  [ex-pallet-03] a wooden crate (has solid walls: label "crate")   [ex-pallet-04] a pile of loose planks
Occluded:  half hidden behind a forklift -> box the VISIBLE part only; set attribute occluded = true.
Truncated: cut off by the image edge -> box up to the edge; set truncated = true.
Smaller than 12 x 12 pixels: skip.
Unsure:    choose "pallet", set attribute needs_review = true, and write the image id in the decision log.

Each rule in the entry is one decision the annotator would otherwise make differently from the next person. The rest of the guide follows the same pattern:

• Class definitions with a positive and a negative example each, and a "when unsure choose X, then flag it" rule.
• Geometry rules. Boxes are tight to the visible extent (say whether to include shadows, handles, reflections). Occluded objects: box the visible part only, or the inferred full extent, but pick one and say it. Truncated objects at the image border: label them, with a flag. Minimum size below which an object is skipped. Crowds of identical objects: one box per instance when separable, one "crowd" region otherwise.
• Polygon rules. Vertex density (enough points that the outline stays within a pixel or two of the edge), no self-crossing, holes handled explicitly, touching instances never share a boundary pixel.
• A decision log. Every question an annotator raises is answered once, written down, and added to the guide with a date, so the answer is the same for everyone afterwards.

Quality control

A measured example

The program builds synthetic data: 30 objects with known boxes, three simulated annotators whose corner placement errors have standard deviations of 2, 3 and 12 pixels (A, B and C), and 6 of the 30 objects treated as gold tasks. It computes pairwise agreement, accuracy on the gold tasks, a consensus box (the coordinate-wise median of the three: for each of the four box coordinates separately, take the middle of the three annotators' values), and a polygon comparison with instance matching. Gold tasks and the consensus are scored here against the simulated truth, which is the stand-in for an expert reference. A last section (6 and 7) repeats the consensus and gold-task questions on 200 objects, to separate what is a property of the method from what is luck on 30 or 6 items.

import numpy as np
from itertools import combinations
from matplotlib.path import Path
from scipy.optimize import linear_sum_assignment

rng = np.random.default_rng(0)

def iou_box(a, b):
    # boxes are (x1, y1, x2, y2) in pixels
    ix = max(0.0, min(a[2], b[2]) - max(a[0], b[0]))
    iy = max(0.0, min(a[3], b[3]) - max(a[1], b[1]))
    inter = ix * iy
    union = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - inter
    return inter / union

# ---- 1. Three annotators box 30 objects; annotator C is sloppy; 6 of the 30 are gold tasks
truth = []
for _ in range(30):
    x, y = rng.uniform(10, 150, 2); w, h = rng.uniform(30, 80, 2)
    truth.append(np.array([x, y, x + w, y + h]))
noise = {"A": 2.0, "B": 3.0, "C": 12.0}               # std of corner jitter in pixels
labels = {k: [t + rng.normal(0, s, 4) for t in truth] for k, s in noise.items()}
gold = set(range(6))

print("1) pairwise IoU between annotators (mean over 30 objects):")
for p, q in combinations("ABC", 2):
    print(f"   {p}-{q}: {np.mean([iou_box(labels[p][i], labels[q][i]) for i in range(30)]):.3f}")

print("2) accuracy on the 6 gold tasks (mean IoU to the reference box, hit rate at IoU>=0.7):")
for k in "ABC":
    ious = [iou_box(labels[k][i], truth[i]) for i in gold]
    print(f"   {k}: mean IoU {np.mean(ious):.3f}, hit rate {np.mean(np.array(ious) >= 0.7):.2f}")

consensus = [np.median([labels[k][i] for k in "ABC"], axis=0) for i in range(30)]   # coordinate-wise median
cons_iou = np.mean([iou_box(consensus[i], truth[i]) for i in range(30)])
single = {k: np.mean([iou_box(labels[k][i], truth[i]) for i in range(30)]) for k in "ABC"}
print(f"3) mean IoU to truth: consensus {cons_iou:.3f}; single annotators",
      {k: round(float(v), 3) for k, v in single.items()})
flag = [i for i in range(30) if min(iou_box(labels[p][i], labels[q][i]) for p, q in combinations("ABC", 2)) < 0.6]
ab = np.mean([iou_box(np.mean([labels["A"][i], labels["B"][i]], axis=0), truth[i]) for i in range(30)])
print(f"   mean of A and B only: {ab:.3f}")
print("4) objects sent to an adjudicator (any pair IoU < 0.6): all three annotators", len(flag),
      "of 30; A and B only", sum(iou_box(labels["A"][i], labels["B"][i]) < 0.6 for i in range(30)), "of 30")

# ---- 2. Polygons: rasterise to masks, compare masks, match instances between two annotators
def mask(poly, size=100):
    yy, xx = np.mgrid[0:size, 0:size]
    pts = np.c_[xx.ravel() + 0.5, yy.ravel() + 0.5]
    return Path(poly).contains_points(pts).reshape(size, size)

def iou_mask(m1, m2):
    return (m1 & m2).sum() / (m1 | m2).sum()

square = lambda x, y, s: np.array([[x, y], [x + s, y], [x + s, y + s], [x, y + s]], float)
ann1 = [mask(square(10, 10, 30)), mask(square(60, 20, 25)), mask(square(20, 60, 20))]
ann2 = [mask(square(12, 11, 30)), mask(square(21, 60, 22)), mask(square(70, 70, 15))]  # no match for #2, extra #3
M = np.array([[iou_mask(a, b) for b in ann2] for a in ann1])
r, c = linear_sum_assignment(-M)                       # best one-to-one matching by IoU
print("5) mask IoU matrix (rows annotator 1, columns annotator 2):")
print(np.round(M, 2))
matched = [(int(i), int(j), round(float(M[i, j]), 2)) for i, j in zip(r, c) if M[i, j] >= 0.5]
print("   matched pairs at IoU>=0.5 (i, j, IoU):", matched)
print("   instances without a partner: annotator 1:", 3 - len(matched), "| annotator 2:", 3 - len(matched))

# ---- 3. When does a median of three help, and how many gold tasks make a hit rate trustworthy? (200 simulated objects, new seed)
rng2 = np.random.default_rng(1)
truth2 = []
for _ in range(200):
    x, y = rng2.uniform(10, 150, 2); w, h = rng2.uniform(30, 80, 2)
    truth2.append(np.array([x, y, x + w, y + h]))
def annotate(sd): return [t + rng2.normal(0, sd, 4) for t in truth2]
def mean_iou(boxes): return np.mean([iou_box(b, t) for b, t in zip(boxes, truth2)])
equal = [annotate(3.0) for _ in range(3)]                       # three equally careful annotators
med_equal = [np.median([e[i] for e in equal], axis=0) for i in range(200)]
print(f"6) three equally careful annotators (sd 3): single annotator {mean_iou(equal[0]):.3f}, median of three {mean_iou(med_equal):.3f}")
mixed = [annotate(2.0), annotate(3.0), annotate(12.0)]          # the A, B, C situation again, on 200 objects
med_mixed = [np.median([m[i] for m in mixed], axis=0) for i in range(200)]
print(f"   one careful, one average, one sloppy (sd 2, 3, 12): best single {mean_iou(mixed[0]):.3f}, median of three {mean_iou(med_mixed):.3f}")
hits = np.array([iou_box(b, t) >= 0.7 for b, t in zip(mixed[1], truth2)])        # annotator B (sd 3) against the reference
z = 1.96
for n in (6, 20, 60, 200):
    k = int(hits[:n].sum()); p = k / n
    centre = (p + z * z / (2 * n)) / (1 + z * z / n)                               # Wilson score interval
    half = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
    print(f"7) annotator B on the first {n:3d} gold tasks: {k:3d} hits, rate {p:.2f}, 95 percent interval {centre - half:.2f} to {centre + half:.2f}")

Output:

1) pairwise IoU between annotators (mean over 30 objects):
   A-B: 0.810
   A-C: 0.499
   B-C: 0.486
2) accuracy on the 6 gold tasks (mean IoU to the reference box, hit rate at IoU>=0.7):
   A: mean IoU 0.870, hit rate 1.00
   B: mean IoU 0.770, hit rate 0.67
   C: mean IoU 0.430, hit rate 0.00
3) mean IoU to truth: consensus 0.874; single annotators {'A': 0.888, 'B': 0.837, 'C': 0.494}
   mean of A and B only: 0.901
4) objects sent to an adjudicator (any pair IoU < 0.6): all three annotators 23 of 30; A and B only 1 of 30
5) mask IoU matrix (rows annotator 1, columns annotator 2):
[[0.82 0.   0.  ]
 [0.   0.   0.  ]
 [0.   0.75 0.  ]]
   matched pairs at IoU>=0.5 (i, j, IoU): [(0, 0, 0.82), (2, 1, 0.75)]
   instances without a partner: annotator 1: 1 | annotator 2: 1
6) three equally careful annotators (sd 3): single annotator 0.827, median of three 0.884
   one careful, one average, one sloppy (sd 2, 3, 12): best single 0.885, median of three 0.872
7) annotator B on the first   6 gold tasks:   6 hits, rate 1.00, 95 percent interval 0.61 to 1.00
7) annotator B on the first  20 gold tasks:  20 hits, rate 1.00, 95 percent interval 0.84 to 1.00
7) annotator B on the first  60 gold tasks:  59 hits, rate 0.98, 95 percent interval 0.91 to 1.00
7) annotator B on the first 200 gold tasks: 193 hits, rate 0.96, 95 percent interval 0.93 to 0.98

What the numbers show:

• Agreement between annotators exposes C: A-B agree at 0.810 IoU while C agrees with either at about 0.49. The gold tasks confirm it (C: mean IoU 0.430, hit rate 0.00 at the 0.7 bar).
• A hit rate on six tasks is too coarse to act on. B scored 4 of 6 gold tasks (0.67), which looks borderline, but one task is 0.17 of the rate. Part 7 gives annotator B (the same error level) 6, 20, 60 and 200 gold tasks: the rate settles near 0.96 with a 95 percent interval of 0.93 to 0.98 at 200, while at 6 tasks the interval runs from 0.61 to 1.00. Size the gold set so the interval is narrow enough to separate a good annotator from a bad one; a few percent of a large queue gives that, six items does not.
• Median consensus helps only when the annotators are comparably good. On the 30 objects the median of three (0.874) did not beat A alone (0.888). Part 6 shows both cases on 200 objects: three equally careful annotators, median 0.884 against 0.827 for a single one (independent errors partly cancel, and the median ignores one outlier per coordinate); one careful, one average and one sloppy annotator, median 0.872 against 0.885 for the best single annotator, because with one wild annotator the middle value of each coordinate is usually one of the two careful values (the one nearer the outlier), not their average, and a single careful value is noisier than the best careful annotator alone. The sloppy annotator's own value is the middle one only occasionally, and when it is, it lies between the careful two and is accurate; the median protects against the wild value but loses the averaging gain it has when all three are careful. The mean of A and B only reaches 0.901 on the 30 objects. The fix for a bad annotator is retraining or removal from the pool, not averaging them in.
• With all three included, 23 of 30 objects fall below the 0.6 pairwise IoU flag, which would swamp the adjudicators; with A and B only, 1 of 30 does. Choose the flag threshold from the distribution of agreement among trusted annotators.
• Polygons: the mask IoU matrix compares every annotator-1 instance with every annotator-2 instance. The assignment step (linear_sum_assignment, the Hungarian algorithm: best one-to-one matching) pairs instance 0 with 0 (0.82) and instance 2 with 1 (0.75). One instance on each side has no partner. Those unmatched instances are the disagreements about whether an object exists, which matter more than boundary jitter; route them to review first. Agreement on masks is reported as the mean IoU of matched pairs plus the count of unmatched instances.

Annotation formats and tooling

• Keep one master store in a format that holds boxes, polygons, class ids, an attributes field (occluded, truncated, crowd), the annotator id, the guide version and a timestamp. The three common formats: COCO is one JSON file for the whole dataset with boxes as [x, y, width, height] in pixels; YOLO is one text file per image with one line per object, a class id and a normalized centre-based box; Pascal VOC is one XML file per image with corner boxes in pixels. COCO-style JSON is a common master, because it also holds polygons; export YOLO text or Pascal VOC XML for trainers that need them, and run a round-trip test after every export (convert, convert back, compare) so conversion bugs show up immediately.
• Tooling, as examples: an open-source labeling tool (for example CVAT or Label Studio) behind single sign-on (one company login for every tool, so an annotator's id is the same everywhere), with a task queue (the list of items waiting to be labeled) that supports gold-task injection (the system mixes gold items into the queue without telling annotators) and overlap assignment; model-assisted pre-labels that annotators correct (faster, but measure for anchoring: annotators accept wrong pre-labels instead of judging freshly, so keep a gold-task slice of images with deliberately wrong pre-labels); an object store for images; a dataset version for every export so a training run cites an exact label set.
• Metrics dashboard per annotator and per class: gold accuracy, agreement, review rejection rate, throughput. Pay for quality as well as volume or the dashboard will be gamed.

Mistakes to avoid

• Measuring agreement only on easy images. Oversample rare and ambiguous classes in the overlap set.
• Fixing the guide without relabeling: old labels follow the old rules, so version the guide and re-review the affected classes.
• Leaving gold tasks fixed for months; annotators memorize them.
• Letting the consensus rule hide a systematically bad annotator.
• Acting on a gold-task score from a handful of items.

The idea in plain words

A detector outputs boxes with confidence scores. Average precision (AP) summarises how good that ranked list is for one class: if you walk down the list from the most confident box to the least confident, how many of the real objects have you found (recall), and how many of the boxes you have accepted so far are right (precision)? AP is the area under the resulting precision-recall curve. Mean average precision (mAP) is the average of AP over classes. The IoU threshold (IoU is intersection over union, the overlap score between two boxes) fixes what "right" means for a box, so a stricter threshold gives a lower AP.

The computation, in the order it happens

For one class and one IoU threshold $t$:

1. Rank all predicted boxes of this class, across all images, by confidence, highest first.
2. Label each prediction as true positive (TP) or false positive (FP), going down the ranking. A prediction is a TP if its IoU with some still-unclaimed ground-truth box of the same class is at least $t$; that ground-truth box is then claimed. Otherwise (no overlap, too little overlap, or the object was already claimed by a higher-scoring box) it is an FP. A ground-truth box that no prediction claims is a false negative (a miss).
3. At each rank $k$ compute, with $\text{TP}_k$ and $\text{FP}_k$ the cumulative counts so far and $G$ the number of ground-truth boxes of the class:

4. Make the precision curve non-increasing (the "envelope"): replace each precision by the largest precision at that rank or any later rank. This removes the zig-zag where precision dips after each false positive.
5. Take the area under the envelope against recall. Two common ways: the exact area (sum of recall step times envelope precision), or the mean of the envelope precision sampled at 101 evenly spaced recall levels 0, 0.01, ..., 1 (the COCO-style choice; at a recall level never reached, precision counts as 0).
6. Average the per-class values to get mAP at this threshold.

The convention used here: envelope precision, 101-point sampling for AP (the exact area is also printed once to show the two differ slightly), mAP as the plain mean over classes.

AP50, AP75 and the COCO-style summary

• AP50 and AP75 are mAP computed with the single threshold $t = 0.50$ or $t = 0.75$.
• COCO-style mAP (written AP, or AP@[.5:.95]) is the average of mAP over the ten thresholds 0.50, 0.55, ..., 0.95. Per the evaluation code in the COCO API, it also uses 101 recall levels and reports results by object size (small, medium, large) and with a cap on detections per image (100 in the headline setting). Papers that report "mAP" without saying which setting they mean are not comparable with each other, so always state the threshold set, the recall sampling and the detections cap.

Worked example

One image, two classes: 4 cats and 3 dogs on the ground truth. The program builds 7 cat predictions and 4 dog predictions by sliding 100 by 100 boxes sideways from their ground-truth boxes by chosen amounts (a sideways shift of a fraction $f$ of the width gives IoU $(1-f)/(1+f)$), plus a duplicate of one cat and some clutter boxes, then runs the six steps above at every threshold. Class names are only labels, and the boxes are synthetic.

import numpy as np

def iou(a, b):
    iw = max(0.0, min(a[2], b[2]) - max(a[0], b[0]))
    ih = max(0.0, min(a[3], b[3]) - max(a[1], b[1]))
    inter = iw * ih
    ar = lambda r: (r[2] - r[0]) * (r[3] - r[1])
    return inter / (ar(a) + ar(b) - inter)

def box(x, y, shift=0.0):                       # 100 x 100 box, optionally shifted sideways
    return (x + 100 * shift, y, x + 100 + 100 * shift, y + 100)

# One image, two classes. Ground truth: 4 cats and 3 dogs.
gt = {"cat": [box(0, 0), box(200, 0), box(400, 0), box(600, 0)],
      "dog": [box(0, 300), box(200, 300), box(400, 300)]}
# Detections: (score, ground-truth index it was aimed at or None, sideways shift as a fraction of width)
dets_spec = {
    "cat": [(0.95, 0, 0.00), (0.90, 1, 0.10), (0.85, None, 0.0), (0.80, 2, 0.23),
            (0.70, 0, 0.05), (0.60, 3, 0.40), (0.50, None, 0.0)],
    "dog": [(0.90, 0, 0.02), (0.80, 1, 0.15), (0.70, None, 0.0), (0.55, 2, 0.30)],
}
def make_dets(cls):
    out = []
    for score, g, f in dets_spec[cls]:
        b = box(*gt[cls][g][:2], shift=f) if g is not None else box(900, 600)   # None = background clutter
        out.append((score, b))
    return sorted(out, key=lambda d: -d[0])

def match(cls, thr):
    """Greedy matching in score order. Each ground-truth box can be claimed once."""
    used, tp = set(), []
    for score, b in make_dets(cls):
        best, best_iou = None, 0.0
        for j, g in enumerate(gt[cls]):
            if j in used:
                continue
            v = iou(b, g)
            if v >= thr and v > best_iou:
                best, best_iou = j, v
        if best is not None:
            used.add(best)
        tp.append(best is not None)
    return np.array(tp)

def pr_curve(cls, thr):
    tp = match(cls, thr)
    ctp, cfp = np.cumsum(tp), np.cumsum(~tp)
    return ctp / len(gt[cls]), ctp / (ctp + cfp)           # recall, precision at each rank

def envelope(prec):                                         # make precision non-increasing
    return np.maximum.accumulate(prec[::-1])[::-1]

def ap_101(rec, prec):
    """COCO-style: mean over 101 recall levels 0, 0.01, ..., 1 of the best precision at recall >= level."""
    env = envelope(prec)
    idx = np.searchsorted(rec, np.linspace(0, 1, 101), side="left")
    return np.mean([env[i] if i < len(env) else 0.0 for i in idx])

def ap_area(rec, prec):
    """Area under the envelope, recall steps times the envelope precision."""
    env = envelope(prec)
    r = np.concatenate([[0.0], rec])
    return float(np.sum(np.diff(r) * env))

rec, prec = pr_curve("cat", 0.5)
print("cat at IoU 0.5: rank, score, recall, precision")
for k, (s, _) in enumerate(make_dets("cat")):
    print(f"  {k + 1}  {s:.2f}  {rec[k]:.2f}  {prec[k]:.3f}")
print("cat AP at 0.5: 101-point", round(ap_101(rec, prec), 4), " area", round(ap_area(rec, prec), 4))

thrs = np.round(np.arange(0.5, 0.96, 0.05), 2)
print("IoU thr  AP cat  AP dog  mAP")
rows = []
for t in thrs:
    a = [ap_101(*pr_curve(c, t)) for c in ("cat", "dog")]
    rows.append(a)
    print(f"  {t:.2f}   {a[0]:.3f}   {a[1]:.3f}   {np.mean(a):.3f}")
rows = np.array(rows)
print("AP50 (mAP at 0.50):", round(rows[0].mean(), 4))
print("AP75 (mAP at 0.75):", round(rows[5].mean(), 4))
print("mAP averaged over 10 thresholds 0.50:0.95:", round(rows.mean(), 4))

# Cross-check: the 101-point value against a much finer recall grid
fine = np.linspace(0, 1, 100001)
env = envelope(pr_curve("cat", 0.5)[1]); rc = pr_curve("cat", 0.5)[0]
idx = np.searchsorted(rc, fine, side="left")
fine_ap = np.mean([env[i] if i < len(env) else 0.0 for i in idx])
print("cat AP at 0.5 on a 100001-point grid:", round(float(fine_ap), 4))

# Trace of the 101-point sampling for the cat curve at IoU 0.5
rc, pc = pr_curve("cat", 0.5); env = envelope(pc)
levels = np.linspace(0, 1, 101)
first_rank = np.searchsorted(rc, levels, side="left")      # for each recall level: first rank whose recall reaches it
print("envelope precision by rank:", np.round(env, 3).tolist())
total = 0.0
for k in sorted(set(first_rank.tolist())):
    ls = levels[first_rank == k]; p = env[k] if k < len(env) else 0.0
    total += len(ls) * p
    where = f"rank {k + 1}" if k < len(env) else "never reached"
    print(f"levels {ls[0]:.2f} to {ls[-1]:.2f}: {len(ls)} levels, first reached at {where}, precision {p:.3f}, adds {len(ls) * p:.2f}")
print(f"sum {total:.2f} / 101 levels = {total / 101:.4f}")

The complete output:

cat at IoU 0.5: rank, score, recall, precision
  1  0.95  0.25  1.000
  2  0.90  0.50  1.000
  3  0.85  0.50  0.667
  4  0.80  0.75  0.750
  5  0.70  0.75  0.600
  6  0.60  0.75  0.500
  7  0.50  0.75  0.429
cat AP at 0.5: 101-point 0.6906  area 0.6875
IoU thr  AP cat  AP dog  mAP
  0.50   0.691   0.916   0.803
  0.55   0.691   0.663   0.677
  0.60   0.691   0.663   0.677
  0.65   0.505   0.663   0.584
  0.70   0.505   0.663   0.584
  0.75   0.505   0.337   0.421
  0.80   0.505   0.337   0.421
  0.85   0.257   0.337   0.297
  0.90   0.257   0.337   0.297
  0.95   0.257   0.337   0.297
AP50 (mAP at 0.50): 0.8032
AP75 (mAP at 0.75): 0.4208
mAP averaged over 10 thresholds 0.50:0.95: 0.5058
cat AP at 0.5 on a 100001-point grid: 0.6875
envelope precision by rank: [1.0, 1.0, 0.75, 0.75, 0.6, 0.5, 0.429]
levels 0.00 to 0.25: 26 levels, first reached at rank 1, precision 1.000, adds 26.00
levels 0.26 to 0.50: 25 levels, first reached at rank 2, precision 1.000, adds 25.00
levels 0.51 to 0.75: 25 levels, first reached at rank 4, precision 0.750, adds 18.75
levels 0.76 to 1.00: 25 levels, first reached at never reached, precision 0.000, adds 0.00
sum 69.75 / 101 levels = 0.6906

Reading it:

• The cat curve at IoU 0.5. Rank 1 and 2 are TPs, so recall is 0.25 then 0.50 with precision 1. Rank 3 (score 0.85) is clutter, an FP, and precision falls to 0.667. Rank 4 is a TP (recall 0.75, precision 0.75). Ranks 5 to 7 are FPs: rank 5 is a second box on a cat that rank 1 already claimed, and rank 7 is clutter; rank 6 aims at the fourth cat but is shifted so far that its IoU (about 0.43) is under 0.5, so recall never reaches 1.
• The envelope. Precision at rank 3 is 0.667, but a later rank has 0.75, so the envelope there is 0.75. The exact area is $0.25 \times 1 + 0.25 \times 1 + 0.25 \times 0.75 = 0.6875$, the 101-point value is 0.6906, and a 100001-point grid reproduces 0.6875 again. The 101-point value sits slightly above the exact area because of how the discrete recall levels fall on the steps, which is why only the same convention should be compared with itself.
• The envelope in code. np.maximum.accumulate(prec[::-1])[::-1] reads right to left: reverse the precision list, keep a running maximum (each entry becomes the largest value seen so far, scanning from the last rank backwards), and reverse it again. For the cat curve the precisions by rank are 1.000, 1.000, 0.667, 0.750, 0.600, 0.500, 0.429; scanning from rank 7 backwards the running maximum is 0.429, 0.500, 0.600, 0.750, 0.750, 1.000, 1.000, which reversed is the envelope the program prints: 1.0, 1.0, 0.75, 0.75, 0.6, 0.5, 0.429. In words: the envelope at a rank is the best precision you could still get by accepting more boxes, never less than what comes later.
• The 101-point sampling in code. np.searchsorted(rec, levels, side="left") answers, for each recall level, "at which rank does recall first reach this level?", and the code then takes the envelope precision at that rank (or 0.0 when the index runs past the last rank, because that recall is never reached). The trace in the last lines of the output does it level by level: levels 0.00 to 0.25 (26 of them, because level 0 counts) are first reached at rank 1, where the envelope is 1.0, adding 26.00; levels 0.26 to 0.50 (25 levels) at rank 2, precision 1.0, adding 25.00; levels 0.51 to 0.75 (25 levels) at rank 4, precision 0.75, adding 18.75; levels 0.76 to 1.00 are never reached, adding 0. The sum is 69.75, and 69.75 / 101 = 0.6906. The exact area uses 25% steps of recall instead (0.25 + 0.25 + 0.1875 = 0.6875); the 101 levels cannot split evenly into the four quarter-steps (26, 25, 25 and 25 levels instead of 25.25 each, so the precision-1.0 step gets slightly more weight), which accounts for the 0.0031 difference.
• Threshold effect. For the dog class AP falls from 0.916 at 0.50 to 0.663 at 0.55 and to 0.337 at 0.75 as the shifted boxes drop below the threshold one by one. mAP is 0.8032 at 0.50 (AP50), 0.4208 at 0.75 (AP75), and 0.5058 averaged over the ten thresholds.

Trade-offs, pitfalls and what mAP cannot see

• Ranking-only. AP uses confidence scores only through their order, so a badly calibrated detector (one whose confidence numbers do not match how often it is right, for example calling everything 0.99) can have a high AP and still need a different operating threshold in production. The operating threshold is the confidence cut-off you choose when the product runs: boxes scoring below it are discarded, boxes above it are shown, so it fixes the single precision-recall point users actually experience. Choose that threshold from the precision-recall curve for the product's cost of misses and false alarms.
• Averaging hides the failing class. Report per-class AP and, in COCO-style results, the size breakdown; a good mean can hide a class or object size that fails.
• Duplicates count as false positives (rank 5 above), so AP depends on the post-processing step, usually non-maximum suppression.
• Dataset-level, not image-level. Predictions from all images are ranked together. Averaging per-image AP gives a different number.
• Lenient thresholds hide localisation problems. If AP50 is good and AP75 is poor, improve box quality, not object finding.

The idea in plain words

Instance segmentation finds every object in an image, draws a box around it, gives it a class, and outlines it with a per-pixel mask. A model pretrained on COCO (a large public detection and segmentation dataset with 80 object categories; its scoring convention is used below) already knows how to find edges, textures, parts and object-like regions. With only 500 images of your own, you keep that knowledge and teach only what is new: your 5 classes and the look of your images. The recipe is: convert and split the data carefully, replace the class-specific output layers, freeze the earliest generic layers, train the rest with small learning rates and a faster one for the new layers, augment with label-consistent transforms, and validate with a split that can actually separate good from lucky.

1. Data preparation

• One annotation format. Convert polygons or masks into the format the training code expects (often COCO-style: for each instance a class id, a box and a mask). Check that every mask is non-empty, every box has positive width and height, and that boxes and masks agree (box equals the tight box of the mask).
• Class ids. The pretrained model's output layer was built for the COCO label set, so class ids are re-mapped to 1 to 5. Frameworks that reserve an index for background (a convention, not a universal) use 6 outputs; check yours.
• Annotation consistency. With 500 images, annotator disagreement is a visible part of the error. Write down what counts as an instance (occluded objects, crowds, partial objects at the border) and spot-check a sample.
• Split by source, not by image. Photos taken in the same session, of the same physical scene or from the same video are near-duplicates. If they are spread across train and validation, validation scores are inflated. Split by session, scene or patient, then stratify by class.
• Stratify so every class is measurable. A rare class with a handful of validation instances gives a meaningless score. Stratified means the split is made so each class keeps about the same share in every part. Example: in the program below the rarest class has 148 instances (8 percent of 1,779 instances over the 500 images); a 20 percent validation share targets 29.6 of them, and a recall measured on about 30 instances has a standard error near 0.07. The program compares a random split with a simple greedy class-balanced one, and a tiny hand-traceable version follows it.
• Hold out a test set that is touched once at the end. With 500 images, one reasonable layout is roughly 70, 15, 15 percent; or use k-fold cross-validation (split the data into k parts, train on k-1 of them, validate on the remaining part, rotate through all k, and average) on train plus validation and keep the test set fixed.

2. What to freeze, what to train

The early backbone stages (the stem, the first convolution layers that read the raw pixels, and the first stage, one block of layers that works at one image resolution) detect edges, colours and simple textures that are the same on your images as on COCO. They have little to gain from 500 images and a lot to lose from noisy gradients, so freeze them. The deeper backbone stages, the neck (the feature-pyramid layers that merge feature maps of different resolutions) and the heads (the box classifier, which decides the class of each box; the box regressor, which adjusts the box coordinates; and the mask predictor, which outputs the per-pixel mask) carry task-specific information. The new output layers (class scores for 5 classes, per-class mask logits, the raw per-pixel scores before they are turned into probabilities) must be initialised fresh because their shapes changed; train those and the heads fully, and train the upper backbone stages at a smaller rate. If validation AP stalls, unfreeze one more stage; if it overfits quickly (training loss falls, validation AP falls), freeze one more.

Normalisation layers. Backbones with BatchNorm (a layer that normalises activations with statistics gathered over the batch) are often run with their normalisation statistics frozen in detection fine-tuning, because batches of 2 to 8 large images are too small for stable batch statistics. Setting requires_grad = False is not enough: in train() mode the layer still updates its running mean and variance (the averages of the activations it has seen, which it stores and uses at test time) from your data. The program shows this and the fix (eval() on the frozen layers).

3. Learning rates

Use layer-wise (parameter-group) learning rates: the new heads at a higher rate, the pretrained layers at roughly a tenth of it, with a short warmup (start tiny and ramp up over the first steps) followed by a decay such as cosine (the rate follows half a cosine wave down to near zero). The numbers in the program are starting points to tune on your validation set, not measured optima: 1e-3 for new heads and 1e-4 for the pretrained backbone and neck with AdamW (an optimiser that adapts the step size per parameter and applies weight decay, a small pull of every weight towards zero that discourages overfitting, separately from the gradient); with SGD (plain stochastic gradient descent, one global step size) the absolute values differ. Four settings come up here: a smaller rate for pretrained layers than for new heads, frozen BatchNorm statistics, warmup, and cosine decay. A common practice, not something the program measures, is to get the first two right first, because they protect the pretrained features and the statistics from a small, noisy dataset; warmup mainly protects the first steps and cosine decay shapes the end of training, so tune those afterwards. A rate that is too high on the pretrained layers destroys the features you came for (watch for validation AP dropping in the first epochs); too low and nothing adapts.

4. Augmentation

Use label-consistent augmentation: horizontal flip when the objects are not left-right sensitive, scale jitter and random crop (with box clipping and dropping boxes with too little remaining), moderate brightness and colour jitter matched to the lighting variation you expect in deployment. Boxes and masks must be transformed together with the image; masks are resampled with nearest-neighbour. Stitching augmentations (mosaic, which tiles four images into one, and copy-paste, which pastes objects between images) can help when instances are few and small, but they also change the image statistics, so validate them. Fix the augmentation seed in debugging.

5. Validation

Report mask AP and box AP, per class, not only the mean. IoU (intersection over union) measures how well a predicted region matches a true one: the area they share divided by the area they cover together, 1 for identical and 0 for no overlap. A prediction counts as correct at an IoU threshold when its IoU with a real object reaches that value. Average precision (AP) for one class: sort the predictions by confidence and lower a confidence cut-off step by step; at each step precision is the share of predictions so far that are correct and recall is the share of real objects found so far; plotting precision against recall gives the precision-recall curve, and AP is the area under it. The COCO convention reports AP averaged over the IoU thresholds 0.5, 0.55, up to 0.95 (so tight masks are rewarded), and also AP at IoU 0.5 alone (a loose pass mark). On validation sets this small the standard error is large, so the program computes it. Choose the checkpoint and the learning rate on validation, and report the test set once.

Worked example

The first part builds 500 synthetic images with 1 to 6 instances each over 5 classes with an imbalanced mix (the instance counts are made up, but the imbalance shape is typical), then repeats a random 100-image validation draw and the greedy class-balanced draw 300 times and counts the rarest class's validation instances. The second part builds a small stand-in network that has the same parts as an instance-segmentation model (backbone stages, a neck, a box head, a mask head; a stand-in, not a real detector) to show freezing, parameter groups, frozen BatchNorm behaviour and a warmup plus cosine schedule.

import math
import numpy as np
import torch
from torch import nn
torch.set_num_threads(1)

# ---- Part 1: split 500 annotated images by image, with class balance in the validation set -----------------
N_IMG, K = 500, 5
class_p = np.array([0.40, 0.25, 0.15, 0.12, 0.08])      # imbalanced instance mix, a made-up but typical shape

def make_counts(rng):
    """counts[i, k] = number of instances of class k in image i."""
    counts = np.zeros((N_IMG, K), int)
    for i in range(N_IMG):
        for k in rng.choice(K, size=rng.integers(1, 7), p=class_p):
            counts[i, k] += 1
    return counts

def val_counts_random(rng, counts, n_val=100):
    idx = rng.permutation(N_IMG)[:n_val]
    return counts[idx].sum(0)

def val_counts_stratified(rng, counts, n_val=100):
    """Greedy: visit images rarest-class-first; an image goes to val while val still lacks its rarest class."""
    freq = counts.sum(0)
    rarest = np.array([np.nonzero(c)[0][np.argmin(freq[np.nonzero(c)[0]])] for c in counts])
    order = np.lexsort((rng.random(N_IMG), freq[rarest]))            # rarest first, random tie-break
    target = counts.sum(0) * n_val / N_IMG                           # 20 percent of each class's instances
    val = np.zeros(K, int); n_in_val = 0
    for i in order:
        if n_in_val < n_val and val[rarest[i]] < target[rarest[i]]:
            val += counts[i]; n_in_val += 1
    return val

rng = np.random.default_rng(0)
counts = make_counts(rng)
print("instances per class over all 500 images:", counts.sum(0).tolist())
rs = np.array([val_counts_random(np.random.default_rng(s), counts) for s in range(300)])
ss = np.array([val_counts_stratified(np.random.default_rng(s), counts) for s in range(300)])
print("validation set of at most 100 images, 300 repeats")
print("target val instances per class (20 percent):", np.round(counts.sum(0) * 0.2, 1).tolist())
for name, arr in (("random", rs), ("stratified", ss)):
    print(f"{name:10s}: rarest-class val instances min {arr[:, 4].min()}, mean {arr[:, 4].mean():.1f}, max {arr[:, 4].max()}")
p = 0.8
for k in (16, 30):
    print(f"standard error of a recall of {p} measured on {k} instances: {math.sqrt(p * (1 - p) / k):.3f}")

# ---- Part 2: what to freeze and which learning rate (a small stand-in network, not a real detector) --------
class Standin(nn.Module):
    """Stand-in with the same roles as a real instance-segmentation model: backbone stages, neck, box head, mask head."""
    def __init__(self, n_out):
        super().__init__()
        self.stem = nn.Sequential(nn.Conv2d(3, 8, 3), nn.BatchNorm2d(8))
        self.stage1 = nn.Sequential(nn.Conv2d(8, 16, 3), nn.BatchNorm2d(16))
        self.stage2 = nn.Sequential(nn.Conv2d(16, 32, 3), nn.BatchNorm2d(32))
        self.neck = nn.Conv2d(32, 16, 1)
        self.box_head = nn.Linear(16, 4 * n_out)       # replaced for the new label set
        self.mask_head = nn.Conv2d(16, n_out, 1)       # replaced for the new label set

model = Standin(n_out=6)                               # 5 classes + 1 background index (a convention some frameworks use)
for name in ("stem", "stage1"):                        # freeze the earliest, most generic stages
    for p_ in getattr(model, name).parameters():
        p_.requires_grad = False

groups = {"backbone (stage2)": model.stage2, "neck": model.neck, "new heads": nn.ModuleList([model.box_head, model.mask_head])}
lrs = {"backbone (stage2)": 1e-4, "neck": 1e-4, "new heads": 1e-3}   # starting values to tune, not measured optima
opt = torch.optim.AdamW([{"params": [q for q in g.parameters()], "lr": lrs[n]} for n, g in groups.items()], weight_decay=1e-4)
for (n, g), pg in zip(groups.items(), opt.param_groups):
    print(f"{n:18s} params {sum(q.numel() for q in g.parameters()):5d}  lr {pg['lr']:g}")
print("frozen params:", sum(q.numel() for q in list(model.stem.parameters()) + list(model.stage1.parameters())),
      "| trainable:", sum(q.numel() for q in model.parameters() if q.requires_grad))

# frozen layers still contain BatchNorm: train() would keep updating their running statistics
bn = model.stem[1]
before = bn.running_mean.clone()
model.train(); model.stem(torch.randn(4, 3, 16, 16))
print("frozen BN running_mean changed in train mode:", not torch.equal(before, bn.running_mean))
bn.eval(); before = bn.running_mean.clone(); model.stem(torch.randn(4, 3, 16, 16))
print("after bn.eval(): changed =", not torch.equal(before, bn.running_mean))

# warmup then cosine schedule over 300 steps; print a few values
total, warm = 300, 30
f = lambda s: (s + 1) / warm if s < warm else 0.5 * (1 + math.cos(math.pi * (s - warm) / (total - warm)))
sched = torch.optim.lr_scheduler.LambdaLR(opt, lambda s: f(s))
seen = {}
for s in range(total):
    seen[s] = [g["lr"] for g in opt.param_groups]
    opt.step(); sched.step()
print("head lr at steps 0, 29, 150, 299:", [f"{seen[s][2]:.2e}" for s in (0, 29, 150, 299)])

Output:

instances per class over all 500 images: [727, 417, 262, 225, 148]
validation set of at most 100 images, 300 repeats
target val instances per class (20 percent): [145.4, 83.4, 52.4, 45.0, 29.6]
random    : rarest-class val instances min 16, mean 29.3, max 41
stratified: rarest-class val instances min 30, mean 30.2, max 32
standard error of a recall of 0.8 measured on 16 instances: 0.100
standard error of a recall of 0.8 measured on 30 instances: 0.073
backbone (stage2)  params  4704  lr 0.0001
neck               params   528  lr 0.0001
new heads          params   510  lr 0.001
frozen params: 1440 | trainable: 5742
frozen BN running_mean changed in train mode: True
after bn.eval(): changed = False
head lr at steps 0, 29, 150, 299: ['3.33e-05', '1.00e-03', '5.87e-04', '3.38e-08']

Reading it:

• A random 100-image validation split gives the rarest class between 16 and 41 instances over 300 draws (target 29.6); the greedy class-balanced draw stays between 30 and 32. That is the point of stratifying: the worst random draw leaves the rare class with barely half its fair share.
• The standard error of a recall near 0.8 is $\sqrt{p(1-p)/n}$, where $p$ is the recall and $n$ the number of instances measured; it is 0.100 at 16 instances and 0.073 at 30. So with a 500-image dataset, per-class differences under about 0.1 are within noise, and a single validation run cannot rank two close configurations. Repeat with k-fold or several seeds before believing a small gain.
• The head group has the larger rate (1e-3) than the backbone and the neck (1e-4), and the schedule peaks at the end of warmup (1.00e-03 at step 29) and decays to near zero at step 299.
• The frozen BatchNorm's running_mean changes in train mode and does not change after eval().

A tiny split to trace by hand

The function in the program is dense, so here is the same logic on 10 images and 3 classes (class 2 is the rare one), with a validation budget of 3 images. Each row of counts is one image. The steps: count instances per class; set each class's validation target to 30 percent of its instances; find each image's rarest class; visit images whose rarest class is the rarest overall first; take an image into validation while that class is still below its target and the image budget is not used up.

import numpy as np

# Greedy class-balanced split on 10 images and 3 classes, small enough to follow by hand.
# counts[i, k] = number of instances of class k in image i. Class 2 is the rare one.
counts = np.array([[3, 0, 0], [2, 1, 0], [1, 0, 0], [2, 0, 0], [0, 2, 0],
                   [1, 1, 0], [0, 0, 1], [2, 0, 0], [1, 0, 1], [0, 1, 0]])
n_val = 3                                                       # validation budget in images
freq = counts.sum(0)                                            # instances per class over all images: [12, 5, 2]
target = freq * n_val / len(counts)                             # 30 percent of each class's instances
rarest = np.array([np.nonzero(c)[0][np.argmin(freq[np.nonzero(c)[0]])] for c in counts])   # each image's rarest class
order = np.lexsort((np.arange(len(counts)), freq[rarest]))      # visit images whose rarest class is rarest first (ties by index)
print("instances per class:", freq.tolist(), "| val target per class:", np.round(target, 2).tolist())
print("rarest class in each image:", rarest.tolist())
print("visit order:", order.tolist())
val = np.zeros(3, int); chosen = []
for i in order:
    take = len(chosen) < n_val and val[rarest[i]] < target[rarest[i]]
    print(f"image {i}: rarest class {rarest[i]}, val has {val[rarest[i]]} of target {target[rarest[i]]:.2f} -> {'take' if take else 'skip'}")
    if take:
        val += counts[i]; chosen.append(int(i))
print("validation images:", sorted(chosen), "| validation instances per class:", val.tolist())

Output:

instances per class: [12, 5, 2] | val target per class: [3.6, 1.5, 0.6]
rarest class in each image: [0, 1, 0, 0, 1, 1, 2, 0, 2, 1]
visit order: [6, 8, 1, 4, 5, 9, 0, 2, 3, 7]
image 6: rarest class 2, val has 0 of target 0.60 -> take
image 8: rarest class 2, val has 1 of target 0.60 -> skip
image 1: rarest class 1, val has 0 of target 1.50 -> take
image 4: rarest class 1, val has 1 of target 1.50 -> take
image 5: rarest class 1, val has 3 of target 1.50 -> skip
image 9: rarest class 1, val has 3 of target 1.50 -> skip
image 0: rarest class 0, val has 2 of target 3.60 -> skip
image 2: rarest class 0, val has 2 of target 3.60 -> skip
image 3: rarest class 0, val has 2 of target 3.60 -> skip
image 7: rarest class 0, val has 2 of target 3.60 -> skip
validation images: [1, 4, 6] | validation instances per class: [2, 3, 1]

Reading it: class 2 has 2 instances, so its target is 0.6, and image 6 (the first visited, one instance of class 2) is taken; image 8 is skipped because the class already has 1, above 0.6. Class 1 then fills from images 1 and 4. The result [2, 3, 1] covers the rare class (1 of its 2 instances) but overshoots class 1 (3 against a target of 1.5), because the greedy rule looks at only one class per image. It is an approximation that guarantees the rare class a place, which is what the 500-image program shows at scale.

Failure modes

Validation leakage through near-duplicate images; the box head predicting well but masks coarse because the mask head received too few updates (look at mask AP versus box AP separately); the rarest class never being predicted (check recall per class, lower the score threshold, or oversample, meaning show the images that contain it more often per epoch); overfitting visible as training loss falling while validation AP falls; and annotation noise capping the score regardless of the model.

Direct answer

I would commit to a single-stage detector (one network pass, no separate region-proposal step) with a lightweight depthwise-separable backbone (the convolutional feature extractor), a small three-level feature pyramid at strides 8, 16 and 32, and a narrow shared head (the layers that turn features into boxes and classes), at an input of roughly 512 pixels on the long side. Small objects are protected by keeping the stride-8 level and by not shrinking the input too far, not by adding a bigger network. Then I treat 20 frames per second as a measured on-device requirement: 50 ms per frame for everything, including camera copy, resize, inference, decoding boxes and non-maximum suppression (NMS, which removes duplicate boxes). Multiply-accumulate counts (MACs: one multiply plus one add) rank the options, but only a run on the target phone settles it.

Terms used below. Stride is how many input pixels one feature-map cell covers (stride 8: a 512-pixel image gives a 64x64 map). A feature pyramid (FPN) merges deep, meaning-rich low-resolution features with shallow, high-resolution ones so every scale gets a map. An anchor is a reference box a cell predicts offsets from; anchor-free heads predict the box directly from each cell. Quantization stores weights and activations as 8-bit integers instead of 32-bit floats.

Choosing each part, and why

1. Backbone. Depthwise-separable convolutions (a per-channel 3x3 filter followed by a 1x1 channel mixer) cost roughly an eighth of an ordinary 3x3 convolution at 128 channels (computed below), so mobile-family backbones are built from them. I would pick a pretrained backbone from that family, check which operators the phone's inference runtime accelerates (unsupported operators fall back to slow paths), and prefer plain convolutions, ReLU-style activations and few memory-bound operations (operations that spend their time moving data rather than computing, such as resizing or concatenating large maps). Two or three candidate widths are benchmarked on the device.

2. Resolution before depth. The cheapest way to lose small objects is to shrink the picture. The table below shows a 40-pixel object in a 1920-wide frame: at a 320 input it is about 7 pixels, which is less than one cell even at stride 8. Resolution costs compute roughly in proportion to pixel count, so I would test 416, 512 and 640 on device and take the smallest that holds recall on the small-object slice of the validation set (the validation boxes below a size cut-off; for example the COCO benchmark's convention calls an object small under 32x32 pixels, and your cut-off should come from the smallest object your app must find). A gap between recall on that slice and recall overall is the signal to raise the resolution.

3. Feature pyramid. Three levels (strides 8, 16, 32), narrow (64 channels), merged with a top-down path (deep features are upsampled and added to shallower ones) and one light bottom-up pass (a second sweep that sends the detailed shallow features back toward the deep levels) if the budget allows. The stride-8 level is the most expensive part of the head (76 percent of it, computed below) and also the only level that can localize small objects, so I narrow its head instead of removing it. Removing the stride-32 level saves only about 5 percent of head MACs, so it is the wrong place to economize.

4. Head. One shared head across levels, depthwise-separable, two layers deep, anchor-free (each cell predicts class scores and the distances to the four box sides) so there is no anchor tuning and one candidate box per cell: 5376 at a 512 input, where an anchor-based head with 3 anchors per cell would have 16128 (both counted below). Training uses a loss that down-weights easy background (focal-style: it shrinks the penalty on cells the model already gets right, so the many empty background cells do not drown out the few object cells) and multi-scale and mosaic augmentation (mosaic stitches four training images into one, so objects appear at varied sizes and positions) so small objects appear often.

5. Deployment. Int8 post-training quantization (convert the finished model to 8-bit integers) with a calibration set (a few hundred real phone frames run through the model once to measure the typical range of each layer's values); if accuracy drops, quantization-aware training (training with the rounding simulated, so the model learns to tolerate it). Cap candidates before NMS (keep only the top few hundred by score), and do NMS in the runtime if it is supported, because a slow decode step can eat the budget.

6. Temporal tricks, if still short. Run the detector on every frame but let a light tracker carry boxes, or run the full model every second frame and the tracker in between. Tiling a high-resolution frame multiplies the cost by the tile count, so I would reserve it for a region of interest.

If only a few choices can be made first, they are the input resolution, keeping the stride-8 level, int8 quantization and an on-device measurement. The tracker, mosaic augmentation, quantization-aware training and tiling are what you reach for when that measurement shows a gap.

The arithmetic behind the choices

The program counts MACs from the shapes (and checks the formula against forward hooks on a built module). It counts the head only; the backbone and pyramid merge are extra. The input is treated as a square R x R (stretched or padded), and the head is 2 layers with 64 channels per level; "std" means ordinary 3x3 convolutions, "sep" means depthwise 3x3 plus pointwise 1x1.

import torch, torch.nn as nn
torch.set_num_threads(1)
torch.manual_seed(0)

def macs_conv(cin, cout, k, hout, wout, groups=1):
    return (cin // groups) * cout * k * k * hout * wout

# 1) Why depthwise-separable blocks: MACs of one 3x3 block, 128 -> 128 channels on a 40x40 map
std = macs_conv(128, 128, 3, 40, 40)
dw = macs_conv(128, 128, 3, 40, 40, groups=128) + macs_conv(128, 128, 1, 40, 40)
print(f"3x3 conv {std/1e6:.1f} M MACs vs depthwise+pointwise {dw/1e6:.1f} M MACs ({std/dw:.1f}x fewer)")

# the same numbers measured from built modules with forward hooks
def hooked_macs(module, x):
    total = [0]
    def hook(m, i, o):
        if isinstance(m, nn.Conv2d):
            total[0] += macs_conv(m.in_channels, m.out_channels, m.kernel_size[0], o.shape[2], o.shape[3], m.groups)
    hs = [m.register_forward_hook(hook) for m in module.modules() if isinstance(m, nn.Conv2d)]
    with torch.no_grad(): module(x)
    for h in hs: h.remove()
    return total[0]
blk = nn.Sequential(nn.Conv2d(128, 128, 3, padding=1, groups=128), nn.Conv2d(128, 128, 1))
hooked = hooked_macs(blk, torch.zeros(1, 128, 40, 40))
print("hooked depthwise+pointwise MACs:", hooked, "matches the formula:", hooked == dw)

# 2) Input resolution vs small objects: a 1920x1080 frame, a 40 px wide object, resized so the long side is R
print("\ninput long side R | object px after resize | cells it spans at stride 8 / 16 / 32")
for R in (320, 512, 640):
    obj = 40 * R / 1920
    print(f"{R:>17} | {obj:>22.1f} | {obj/8:.2f} / {obj/16:.2f} / {obj/32:.2f}")

# 3) Head cost per pyramid level on a square R x R input: 2 conv layers per level with C=64 channels.
#    "std" = ordinary 3x3 convs, "sep" = depthwise 3x3 + pointwise 1x1 per layer.
#    Anchor-free: each cell gives one candidate box. An anchor-based head with A = 3 anchors per cell is printed for comparison.
R, C, A = 512, 64, 3                                   # A is used only for the anchor-based comparison
def sep(c, n): return macs_conv(c, c, 3, n, n, groups=c) + macs_conv(c, c, 1, n, n)
print("\nlevel stride | map size | std head MACs (M) | sep head MACs (M) | anchor-free boxes | with 3 anchors")
tot_std = tot_sep = tot_cells = 0
for s in (8, 16, 32):
    n = R // s
    h_std = 2 * macs_conv(C, C, 3, n, n); h_sep = 2 * sep(C, n)
    tot_std += h_std; tot_sep += h_sep; tot_cells += n * n
    print(f"{s:>12} | {n:>3}x{n:<4} | {h_std/1e6:>17.1f} | {h_sep/1e6:>17.1f} | {n*n:>17} | {n*n*A:>12}")
print(f"totals: std {tot_std/1e6:.1f} M, sep {tot_sep/1e6:.1f} M MACs, {tot_cells} anchor-free candidate boxes ({tot_cells*A} with {A} anchors per cell)")
print(f"stride-8 level share of head MACs: {2*sep(C,R//8)/tot_sep*100:.0f}% (sep) of {tot_sep/1e6:.1f} M")

# 4) Frame budget at 20 FPS
print("\nframe budget at 20 FPS:", 1000 / 20, "ms per frame, shared by camera copy, resize, model, decode + NMS, drawing")

# 5) Turning MACs into milliseconds needs the phone's sustained throughput, which only a device run gives.
#    Illustrative throughputs (not measurements of any phone): how many MACs fit in 30 ms of the 50 ms budget?
for gmacs in (20, 50, 100):
    print(f"if the phone sustained {gmacs} G MACs/s, 30 ms allows {gmacs * 1e9 * 0.030 / 1e9:.1f} G MACs for backbone + neck + head")

Output:

3x3 conv 235.9 M MACs vs depthwise+pointwise 28.1 M MACs (8.4x fewer)
hooked depthwise+pointwise MACs: 28057600 matches the formula: True

input long side R | object px after resize | cells it spans at stride 8 / 16 / 32
              320 |                    6.7 | 0.83 / 0.42 / 0.21
              512 |                   10.7 | 1.33 / 0.67 / 0.33
              640 |                   13.3 | 1.67 / 0.83 / 0.42

level stride | map size | std head MACs (M) | sep head MACs (M) | anchor-free boxes | with 3 anchors
           8 |  64x64   |             302.0 |              38.3 |              4096 |        12288
          16 |  32x32   |              75.5 |               9.6 |              1024 |         3072
          32 |  16x16   |              18.9 |               2.4 |               256 |          768
totals: std 396.4 M, sep 50.2 M MACs, 5376 anchor-free candidate boxes (16128 with 3 anchors per cell)
stride-8 level share of head MACs: 76% (sep) of 50.2 M

frame budget at 20 FPS: 50.0 ms per frame, shared by camera copy, resize, model, decode + NMS, drawing
if the phone sustained 20 G MACs/s, 30 ms allows 0.6 G MACs for backbone + neck + head
if the phone sustained 50 G MACs/s, 30 ms allows 1.5 G MACs for backbone + neck + head
if the phone sustained 100 G MACs/s, 30 ms allows 3.0 G MACs for backbone + neck + head

What it shows. A separable block does the same shape of work in about an eighth of the MACs (8.4x fewer here), and the hooked count equals the formula. At a 512 input a 40-pixel object spans 1.33 cells at stride 8 but only 0.33 at stride 32, so only the stride-8 map can localize it. The stride-8 level is 76 percent of the separable head's 50.2 M MACs because it has 16 times as many cells as stride 32, so on a tight budget you narrow that level's head rather than remove the level. Anchor-free, the three levels give 5376 candidate boxes, a third of the 16128 an anchor-based head with 3 anchors per cell would give, which also shortens the NMS work. The 50 ms per frame is shared by every stage, so the network gets only part of it. MACs turn into milliseconds only through the phone's sustained throughput, which a device run measures. The last three lines use made-up round throughputs (not any phone's figures) to show the conversion: at 50 G MACs per second, 30 ms of the 50 ms allows 1.5 G MACs for the whole network, and the 50.2 M MACs of this head are a small slice of that, so the choice of backbone width is what the on-device benchmark has to settle.

Trade-offs and what would flip the choice

• MACs are a proxy. Latency also depends on memory traffic, operator support and thermal throttling (the phone slowing its processor as it heats up). Measure sustained frame rate after the phone has warmed up, on the lowest device in the target tier, not a burst on a flagship.
• Accuracy vs speed is one curve to plot, not a single point. Plot small-object recall (or average precision on small boxes) against measured latency for each input size and backbone width, then choose the point under 50 ms with margin.
• A two-stage detector (propose regions, then classify each) is more accurate per scale but runs a second stage per region, so it only returns if the budget is relaxed or accuracy matters more than speed.
• If objects are tiny and rare, the flip is toward a higher input resolution with a narrower network, or a region-of-interest crop, instead of a wider backbone.
• Common mistakes. Judging speed from parameter count (parameters and MACs diverge: early high-resolution layers cost the most compute with few parameters), benchmarking on a desktop GPU, ignoring pre- and post-processing, and quantizing without a representative calibration set.