DESIGN AND ANALYZE AN A/B TEST USING A SURVEY TOOLSTEP 1: 5–10 minutesForm a hypothesis (must be measurable) about an email campaign for your favorite new or existing product. For example, a hypothesis might be: “An email campaign for a firm that sells outdoor gear will convert (i.e., cause a customer to finalize a purchase) more with a subject line with a promotional message plus call to action (e.g., ‘50% Off All Jackets. Buy Now!’) than one with a promotional message alone (e.g., ‘50% Off All Jackets.’).”STEP 2: 15–30 minutesCreate two versions of the email in the survey, each an email mockup that reflects a test of your hypothesis (e.g., one with “50% Off All Jackets. Buy Now!” as the subject line and the other with “50% Off All Jackets” as the subject line; and all else (e.g., the email body) equal. Note that in Qualtrics this would need to be two separate BLOCKS and in SurveyMonkey it would need to be two separate PAGES. Similarly, Google Forms will let you create separate SECTIONS.STEP 3: 5–10 minutesAsk in BOTH SECTIONS, BLOCKS or PAGES the identical outcome question such as likelihood of purchase: “How likely would you be to click-through to finalize the purchase of an appealing jacket?” Measure this on a scale from 1 to 10, where 1 = very unlikely to 10 = very likely.STEP 4: 5–10 minutesSetup the survey tool so that participants are randomly assigned to one of the two conditions. (Google Forms offers SECTIONS in answer to questions and Qualtrics offers BLOCK randomization by pressing its SurveyFlow button at the top left,and SurveyMonkey allows PAGE randomization; be careful that it is set up to assign onlyONE conditional as the default is sometimes to assign multiple conditions to the same personbut in random order).STEP 5: 10–15 minutesStudents may publish the survey to one another in our General Student Discussion Board (Share Articles and Ask Each Other Questions), or to friends or colleagues through social media like LinkedIn. You’ll want at least 20 responses.STEP 6: 5–10 minutesAfter the survey has been completed by respondents, export the results to a spreadsheet tool such as Excel.STEP 7: 5–15 minutesCalculate the mean response for A and the mean response for B, using the “=average()” function of Excel.(Optional) Predict purchasing intention using ACNielsen (Links to an external site.) BASES Predicted Purchase Intent.STEP 8: 5–10 minutesIllustrate the results by creating a simple bar chart.STEP 9: 5–10 minutesIn a brief write up (300-500 words), write your hypothesis, provide the link to your survey, include your illustration of the results and discuss the marketing implications in relation to the hypotheses drawn. In addition, attach your Excel file as a separate document.
