🔒 Processed locally in your browser — no data uploaded
← Back to Home

Prompt A/B Tester

Prompt A
Prompt B
Comparison

What the Prompt A/B Tester does

This page puts two prompts in two editors and reports how they compare once you press Compare. It counts characters, words, sentences and non-empty lines in each one, then sets the two columns beside a difference column that shows both the absolute change and the percentage change from A to B. A single line under the table names whichever prompt came out using fewer estimated tokens.

The token figure is calculated inside the page rather than by a vendor tokenizer: it takes the character count times 0.25 and adds 0.8 for every Chinese, Japanese or Korean character it finds. That makes it a rough size signal for comparing drafts, not a billable count. Both editors load with a sample prompt, one written as a general assistant instruction and one written as a software engineer instruction, so the table has something to show before you paste your own text.

How to use it

  1. Paste your first draftClick into the left editor, clear the sample text, and paste the prompt you already use. The summary line under the box refreshes the next time you run a comparison, not as you type.
  2. Paste the variantPut the rewritten version in the right editor. Change one thing at a time, such as a role line, a length limit or an output format, so any difference you see is attributable to that change.
  3. Press CompareOne click fills both summary lines and opens the table with the difference column populated. Nothing is measured until you press it, which leaves you free to edit in either editor without the numbers jumping around.
  4. Read the diff columnStudy the A to B column rather than the raw totals. A shorter rewrite that dropped a constraint shows up as fewer words and fewer sentences in the same row, while a longer one that only added examples shows its growth plainly.

Options and formats

Two side-by-side editorsPrompt A and Prompt B sit in separate text areas with generous height, so both remain visible on a laptop screen and you can compare wording without scrolling between them.
Per-prompt summary lineEach editor prints its own characters, words, estimated tokens and line count underneath, giving a quick read of either draft before you look at the table at all.
Five-row metric tableCharacters, words, sentences, lines and estimated tokens are reported together, with the difference expressed both as a signed number and as a percentage of the first prompt.
Token-efficiency calloutAfter a run, one line states which prompt used fewer estimated tokens, or that the two came out level. It weighs size alone and says nothing about which prompt works better.

Things to watch out for

The tool does not run promptsNo model is called anywhere in this page. Two prompts of nearly equal length can produce very different answers, and this tool cannot tell you which answer is better because it only measures size and shape.
Word counts split on spacesText is divided on whitespace, so a Chinese or Japanese prompt written without spaces registers as one very long word, even though the token estimate does count those ideographic characters separately.
Sentences break on punctuationEach period, exclamation mark and question mark ends a sentence, so abbreviations such as e.g. or version strings like v1.2 add phantom sentences. Read that row as a structural hint rather than a count of instructions.
Token figures are approximationsThe estimate applies a fixed rate per character, while real tokenizers split on subwords and treat code, punctuation and uncommon words differently. Use the figures to compare two drafts, not to predict an invoice.

Frequently asked questions

Why do the numbers stay still while I type?

The counters and the table only update when Compare is pressed. That keeps the measurement stable while you are mid-sentence, and it also means whatever table you are looking at may describe an older version of the text now sitting in the editors.

Can I compare more than two prompts at once?

Only two at a time. For a set of three or more, keep the strongest candidate in slot A and test each challenger against it in turn, noting the word and token figures for each pairing before you move on.

Does a lower token estimate always mean a better prompt?

No. Deleting an example, a format rule or an explicit constraint lowers the count and can also lower the quality of the output. Treat the smaller figure as one input to the decision rather than as the verdict itself, and decide on quality by reading both drafts.

What counts as a line in the line row?

Only lines that still contain something after the surrounding whitespace is trimmed away. Blank lines used for visual separation are skipped entirely, so a prompt padded with empty lines will not look longer than the same prompt written without them.

Can I save a comparison and come back to it?

There is no stored history anywhere on the page. Copy the table by hand or keep your notes in another document, because refreshing the browser or pressing Clear discards both prompts and the results, including the sample text that loads by default.

How close is the estimate to what my API reports?

It lands in the right range for long English prose and starts to drift on code, JSON, heavy punctuation and mixed scripts. If you need an exact figure, measure one real request with the tokenizer your provider documents rather than relying on this rate.