Purplelink
← Why it matters

Manuscript red-team reference · companion to Paper Review

hyperparameter asymmetry

Tuning the proposed method, not the baselines.

What it is

Your method might genuinely be better, OR you might have spent 100 GPU-days on hyperparameter search for your method and 5 minutes on default settings for the baselines. The empirical gap shrinks when both sides get matched search budgets — sometimes it disappears entirely.

Why a reviewer cares

Reviewers ask: what search budget did each method get? Were the same search ranges used? Did baselines have access to the same data augmentation, regularisation, and training recipe? An empirical comparison where only the proposed method's hyperparameters are tuned is not a fair comparison.

How to fix it

Match the hyperparameter search budget across all methods. If a baseline has fewer hyperparameters, give it the same number of search iterations. Report the search protocol explicitly in the methods. Where you can't match search budget, run an ablation showing what the gap looks like with matched effort.


This is one of ~15 canonical methodology explainers Paper Review's red-team report links to. To get a full review of your manuscript, start a Paper Review — $5.