What the Prompt A/B Tester does
This page puts two prompts in two editors and reports how they compare once you press Compare. It counts characters, words, sentences and non-empty lines in each one, then sets the two columns beside a difference column that shows both the absolute change and the percentage change from A to B. A single line under the table names whichever prompt came out using fewer estimated tokens.
The token figure is calculated inside the page rather than by a vendor tokenizer: it takes the character count times 0.25 and adds 0.8 for every Chinese, Japanese or Korean character it finds. That makes it a rough size signal for comparing drafts, not a billable count. Both editors load with a sample prompt, one written as a general assistant instruction and one written as a software engineer instruction, so the table has something to show before you paste your own text.
How to use it
- Paste your first draftClick into the left editor, clear the sample text, and paste the prompt you already use. The summary line under the box refreshes the next time you run a comparison, not as you type.
- Paste the variantPut the rewritten version in the right editor. Change one thing at a time, such as a role line, a length limit or an output format, so any difference you see is attributable to that change.
- Press CompareOne click fills both summary lines and opens the table with the difference column populated. Nothing is measured until you press it, which leaves you free to edit in either editor without the numbers jumping around.
- Read the diff columnStudy the A to B column rather than the raw totals. A shorter rewrite that dropped a constraint shows up as fewer words and fewer sentences in the same row, while a longer one that only added examples shows its growth plainly.
Options and formats
Things to watch out for
Frequently asked questions
Why do the numbers stay still while I type?
The counters and the table only update when Compare is pressed. That keeps the measurement stable while you are mid-sentence, and it also means whatever table you are looking at may describe an older version of the text now sitting in the editors.
Can I compare more than two prompts at once?
Only two at a time. For a set of three or more, keep the strongest candidate in slot A and test each challenger against it in turn, noting the word and token figures for each pairing before you move on.
Does a lower token estimate always mean a better prompt?
No. Deleting an example, a format rule or an explicit constraint lowers the count and can also lower the quality of the output. Treat the smaller figure as one input to the decision rather than as the verdict itself, and decide on quality by reading both drafts.
What counts as a line in the line row?
Only lines that still contain something after the surrounding whitespace is trimmed away. Blank lines used for visual separation are skipped entirely, so a prompt padded with empty lines will not look longer than the same prompt written without them.
Can I save a comparison and come back to it?
There is no stored history anywhere on the page. Copy the table by hand or keep your notes in another document, because refreshing the browser or pressing Clear discards both prompts and the results, including the sample text that loads by default.
How close is the estimate to what my API reports?
It lands in the right range for long English prose and starts to drift on code, JSON, heavy punctuation and mixed scripts. If you need an exact figure, measure one real request with the tokenizer your provider documents rather than relying on this rate.