Which method should sort your list, and what will it cost?

Never start with pointwise (“rate each 0–100”).

• It is the cheapest per item and the one that fails first: scores collapse into tie blocks, and a top-k cut through a block drops arbitrary members — up to 70% of the reference top-10 in the measured cells. → E12, E14.

n ≤ ~30 and you want the gaps (“how much better”), with a frontier judge?

• Pairwise ratio, default budget 4·n comparisons, both presentation orders, active planner. You get a cardinal scaling with σ per item, order flips reported, and a certified stop. At n = 8 that is 32 comparisons — measured at ≈ $0.11 on the default judge with the README's items; the calculator below gives $0.086 at 600-character items. → llmsort sort list.txt --by "…"

n from ~30 to a few hundred, and the order is what you need?

• Setwise — windows of 6–8 items in lettered slots, one full order per call, anchored ring so the graph connects in one round. Matches pairwise's own test–retest band at ~⅓ the cost; a 24-item sort cost $0.0054 live. → --setwise, E6.
• Past n = 24 (three times the window size) the engine adds a second round by default. Per-item budget does not transfer across n; graph density does (at n = 150, agreement 0.81–0.91 → 0.55–0.71 with one round, back within 0.02–0.07 of the pairwise ceiling with two, at ~⅙ its cost). → --rounds 2, E13.

You only need the best k?

• The funnel: setwise screen of the whole pool, then a certified pairwise refine of the top ~3k. Brackets the reproducibility ceiling in every measured cell at 0.3–0.6× the cost of pairwise alone. → --setwise --top-k K, E14.
• Read the stability number it prints. Subjective top-k reproduces itself across seeds at only 0.3–0.7 for every method; if yours is at the low end, the honest move is a bigger k or a sharper criterion, not a different tool.

Your judge is a small or local model (under ~30B parameters)?

• Ask for direction, not magnitude. Seven of nine small judges could not read the ratio ladder (four of them answer one half of the alphabet 99–100% of the time, one answers parity 99% of the time) yet agree with the reference at +0.40 to +0.80 on direction; gemma-4-12b-it on the ordinal instrument is as good as the 31b. In the crate the ordinal path is setwise — a full order per window, lowered to direction evidence — pointed at your model. The pairwise ordinal-letter instrument from the bakeoff (ordinal_letter_v1) is in the crate as the default read for the local gemma judges on the logprob route, but is not yet a documented --template value. → --setwise --model <slug>, bakeoff.

The list is in the thousands?

• Screen first. Retrieval or a cheap setwise pass bounds the pool to the few hundred that could plausibly matter, then the funnel. This is the frontier of the program, not a finished recipe; the n = 150 lesson (structure over budget) is the guide up the exponents. → agenda.

You are not sure the criterion means anything?

• Probe it before you spend: sort by a paraphrase and by the negation. A criterion the judge groks survives both (paraphrase ρ ≥ 0.8; polarity should sit near −1, read as a diagnostic — E2 demoted it from a gate). → --also-by "…", --two-sided, below.