Sorta true but my focus here is to explain the difference between the two types of testing and show applications of each type of testing within the realm of basic email marketing.
Split, or A/B, testing pitches two versions of one variable against each other. Subject line, personalization, pre-header, image, choice of descriptive words, CTA’s, sent from, etc. The folks in optimization land call this level of testing “global”. Multivariate testing pitches multiple variables against each other and is described as “local”. Typically the methodology incorporates various landing pages and requires some very thoughtful planning and analysis, branch and bound decision making, etc
.
In the real world we do global and local testing all the time. Recently I decided to start the purchase process for a truck, my first! What prompted this? I recently moved to North Carolina and everybody but me has a truck. I set-up appointments with the local Ford and Chevy dealers because I had decided that the smaller, mid-size models would be easier for my wife (who is relatively short) to get into and these models stood out as the top two in my search. We test drove a Ford Ranger and a Chevy Colorado. In both cases we would have to add running boards, but generally we preferred the Colorado. Split testing two trucks of similar characteristics would qualify at “global” level.
Having decided to focus on the Chevy Colorado, we now needed to move to the “local” testing level and consider the different colors, engines, transmissions, accessories, etc. In other words, multiple variables had to come into play to complete our testing process.
We chose to stop and re-group because the split-test did not actually measure our original test question: “Which one was easier to get into before you even turned the key?”
Jump to the end: Ordered a Ford Maverick! If you’re not familiar with the new Ford Maverick, it’s based on the Ford Escape body style and size. My wife’s car is a Ford Escape and she likes it’s size; its easy to get into.
Back to email. A/B tests are best used at the “global” level. I noted earlier a few of the obvious ones for consideration. Deciding which variable should be guided by a strategic macro-level question or issue. Low open rates. Poor click through. Unsubscribes. Segmentation. At the “local” level, multivariate testing could be as simple as testing more than 2 versions of a subject line if the email platform provider has that capability.
A more sophisticated test could present multiple versions of a proven, or control, email containing a link to a lead magnet download sign up form. Create several versions of the download sign up form: text only, text with graphic, one with additional demographic question, one with check boxes, etc. Each form would have a unique link coded into the various versions of the control email. Each submitted form would trigger a unique tag associated with each form via an automation. After the fact analysis would be simple and within the skill range of a moderately experience email marketer.