How Do I A/B Test Two QR Code Designs?
To A/B test QR code designs, give each design its own dynamic code, mix both into the same print stacks, and fix the sample size before you look.
Nookesh Karri · 7 min read 
To A/B test QR code designs, make one dynamic QR code per design, print both versions in the same run, and shuffle them into the same stacks so each design goes to the same places at the same times. Change only one thing between the two, decide how many scans you need before you start, and compare scans per piece handed out. Then reprint the winner.
The setup is easy. The hard part is not fooling yourself with a small or lopsided sample.
What you need to A/B test QR code designs
An A/B test of QR code designs needs two printed versions that differ in one thing, a separate trackable code for each, and a way to count how many of each you handed out.
- Two dynamic codes. A static code can’t count scans, so it can’t tell you which design won.
- One variable. Pick color, frame, logo, or the call-to-action line. If you change two, you won’t know which one did the work.
- The same landing page behind both codes, so the only difference is what’s printed.
- A print count per version. Scans mean nothing without the number of pieces that went out.
Which QR code design elements are worth testing
The design elements worth testing are the ones a person sees before deciding to scan: the call-to-action text, the frame around the code, the colors, and a logo in the middle.
I’d start with the call to action. A line like “Scan for today’s menu” gives people a reason, and my guide to QR code calls to action has examples you can turn into variants. Color and frame come next. A logo in the middle is worth testing last, because it’s the change most likely to hurt scanning.
How to set up the test, step by step
Setting up a QR design test takes six steps, and most of the work happens before anything is printed:
- Create two dynamic codes and name them clearly:
menu-cta-aandmenu-cta-b. - Point both at the same page. If you use Google Analytics, keep the campaign, source and medium identical and set
utm_contenttodesign-aordesign-b. Google’s URL builder help describesutm_contentas the field for telling creatives apart, and the Terminus UTM cheat sheet gives the printed flyer exampleflyer-version-avsflyer-version-b. - Keep the margin. DENSO WAVE, which invented the QR code, says on QRcode.com that a QR code needs “a four-module wide margin” on all sides. A fancy frame that eats that margin breaks the test.
- Print an equal number of each in the same run, on the same paper.
- Interleave them. Stack A, B, A, B before they leave the print shop.
- Scan one printed copy of each with an iPhone and an Android phone before anything goes out.
Why you should interleave A and B in the same stacks
Interleaving means every stack contains both designs in alternating order, so placement, day and audience are the same for A and B. This is the step most QR testing guides skip, and it’s the one that makes the result believable.
Here’s the trap. You put design A at the gym and design B at the coffee shop. A wins. But the gym crowd might scan anything, so you’ve measured the gym, not the design. The same thing happens with time: A goes out in week one, B in week two, and a rainy week two sinks B.
Alternating in every stack fixes both. For a per-location flyer setup, you can still give each location its own code pair if you want both answers.
How many scans you need before you call a winner
The number of scans you need depends on your scan rate and the smallest difference you care about, and you have to fix that number before the test starts. Stopping when the numbers look good is the most common way to get a fake winner.
Evan Miller, whose free calculators many testers use, explains in How Not To Run an A/B Test that significance math assumes “the sample size was fixed in advance”. If you run until you see a significant difference, he says, the reported significance levels “become meaningless”. Peeking is fine. Stopping early isn’t.
His sample size calculator asks for four inputs: your baseline conversion rate, the minimum detectable effect, statistical power and significance level. Its default example comes out at 2,545 per variation.
For a print test, the “conversion rate” is scans divided by pieces handed out. That’s why step 4 matters. If you can’t hand out enough pieces to reach the number, test a bigger change: two very different calls to action, not two shades of blue.
What to measure and how to read it
The main metric is scans per piece handed out for each design, and unique scans matter more than totals. One curious person scanning a flyer four times inflates the count without adding anyone.
Check both in your dashboard, and see my post on unique vs total scans if your tool shows both. If your page has a sign-up or booking, compare those too. A design that gets more scans from people who bounce isn’t better.
Then write the result down with the dates, the print count and the one thing you changed. The next test starts from the winner.
Common mistakes when testing QR code designs
Most failed QR design tests break one of the rules above:
- Two changes at once, like a new color and a new call to action together.
- Different spots for each version, so the location wins instead of the design.
- Stopping early because B is ahead after the first weekend.
- A variant that doesn’t scan. If a low-contrast design fails on some phones, it loses for a reason that isn’t appeal. Run each version through a QR code validator for contrast before printing.
- Different landing pages, which turns a design test into a page test. For UTM naming, my UTM parameters guide has a full scheme.
How QRAnalytica handles design tests
QRAnalytica gives each dynamic code its own scan counts with time of day, city and device, so two codes become two columns you can compare. UTM injection adds the UTM tags to each redirect for you, and the styling options cover colors, gradients, eye patterns, body shapes and logo overlays for building the variants.
The limitation: QRAnalytica has no built-in split-test report or significance calculator. You compare the two codes yourself and run the numbers through Evan Miller’s calculator or one like it.
My take
Most small businesses should test the call-to-action line and nothing else. Color tests need a lot of printed pieces to show anything. A better reason to scan can move the numbers in a single run.
FAQ
Can I A/B test with one QR code?
Not for design tests. Both designs need their own code so each scan is counted against the right version. A single code that rotates destinations tests landing pages, not the printed design.
What should I test first on a QR code?
Test the call-to-action line next to the code before colors or shapes. It tells people why to scan, and it is the easiest thing to change on the next print run.
How many scans do I need to pick a winner?
It depends on how big a difference you want to detect. Put your expected scan rate and the smallest lift you care about into a sample size calculator, then commit to that number before you start.
Can a fancy QR code design stop it from scanning?
Yes. Low contrast, a logo that covers too much of the pattern, or a cropped margin can break scanning. Test every printed variant on several phones before the run goes out.