This post originally appeared on ContentVerve.com – which was one of the best CRO blogs around until it disappeared. This was one of my favourite posts by Michael Aagaard which i’ve brought back to keep this great content alive.
I recently tested four different privacy policies on a sign-up form on the home page of a betting community. The results were quite surprising as the variations had drastically different impact on sign-ups – from an 18.70% drop in sign-ups to an increase of 19.47%.
In this article I’ll show all four variations, run you through the test data, and give you concrete takeaways.
Privacy Policy Experiment 1:
Here’s a screen dump of the BettingExpert.com home page. As you can see, the control variant of the sign-up form does not feature a privacy policy, so I decided to test the impact that adding a privacy policy would have on conversions – in this case sign-ups.
In the first experiment I went for an informal, slightly cute privacy policy:
100% privacy – we will never spam you
I totally assumed that the treatment with the privacy policy would perform significantly better than the control. In fact, I only ran this test in order to document how much of a lift it would generate. So I was pretty taken aback by the test results.
The treatment with the privacy policy actually hurt conversions and reduced the number of sign-ups by a staggering 18.70%!
Test data: I ran the test for 9 days, reached a sample size of 16152 visits and 297 conversions, and the control variant outperformed the treatment throughout the entire test period. The statistical confidence level was 96%, and the standard error was 0%.
The development of experiment one over the 9-day test period.
Takeaways from experiment 1:
However counterintuitive it may seem, adding a privacy policy does not guarantee more sign-ups. In fact, it can seriously hurt conversions.
My hypothesis is that – although the messaging revolves around assuring prospects that they won’t be spammed – the word spam itself give rise to anxiety in the mind of the prospects. Therefore, the word should be avoided in close proximity to the form.
Privacy Policy Experiment 2:
In the second privacy policy test, I stayed clear of the word spam but stuck with the short and sweet 100% privacy from the first treatment:
100% privacy. We keep all your personal information secret
Test data: I ran this test for 12 days, and after 15675 visitors and 279 conversions, there really was no significant difference between the control variant and the treatment. The treatment started out performing better, but tanked as the sample size grew over the 12 day test period.
The treatment started out performing better, but tanked as the sample size grew over the 12-day test period.
Takeaways from experiment 2:
The treatment in experiment 2 performed way better than the treatment from experiment 1, but it still didn’t improve conversion.
It would seem that removing the word spam and focusing on the aspect of keeping information secret had a positive effect. Nevertheless, this is still a vague policy that doesn’t say a whole lot.
Privacy Policy Experiment 3:
Ok, so experiment 1 and 2 gave me some valuable insights and helped me develop an idea for yet another treatment. I was still pretty surprised by the results of the first two tests.
For the third treatment I decided to go for a more authoritative and solid policy with no “cuteness factor”. Moreover, I wanted this treatment to be much clearer than the two previous ones. With all this in mind, I came up with:
We guarantee 100% privacy. Your information will not be shared
Bingo – this treatment was a home run! It increased conversions significantly and gave us 19.47% more sign ups.
Test data: I ran the test for 12 days and reached a sample size of 20257 visitors and 380 conversions. The statistical confidence level was 96% and the standard error was 0%. From very early on in the test, the treatment outperformed the control variant.
The development of experiment 3 over the 12-day test period.
Takeaways from experiment 3:
The most noticeable change in treatment 3 is the guarantee. In experiment 1 and 2, the policy only said, 100% privacy, whereas the policy in experiment 3 said, We guarantee 100% privacy.
Personally I find this wording much more credible. Moreover, the second part, Your information will not be shared, is way more clear and authoritative.
Credibility, clarity, and authority is really what you want in a privacy policy, and I believe that the combination of those three factors is what made this treatment perform so well.
Privacy Policy Experiment 4:
OK, so the first three experiments taught me a lot and helped me develop a few hypotheses – mainly that you need a guarantee on your privacy policy and that using the word spam can seriously backfire.
Now I was curious to see what would happen if I combined the “best” and “worst” from the previous experiments into one variant – which resulted in treatment 4:
We guarantee 100% privacy. We will never spam you!
Test data: I ran this test for 15 days and after 18959 visitors and 370 conversions, there was no significant difference between the control variant and the treatment policy.
Takeaways from experiment 4:
Treatment 4 was a combination of the best performing variant, We guarantee 100% privacy, and the worst performing variant, We will never spam you.
As the test data revealed that there was no significant difference between the control and the treatment, it would seem that the “good” and the “bad” part parts cancelled each other out and therefore had no real impact on sign-ups.
Main Take-aways From All 4 Experiments
The privacy policy you use on your signup form can have major impact on your sign-up rate. However, just sticking one on there doesn’t guarantee more conversions – in fact, if your not careful about choosing the right wording, you could seriously hurt your conversion rate.
But If you’re willing to put some time into researching what works on your specific form, a privacy policy can lead to a serious lift in conversions – talk a bout a major low-hanging fruit!
My research points to the fact that a credible, clear policy with a guarantee effectively assures prospects that it’s safe to fill out the form. Moreover, my data suggests that you should be careful with using the word spam – even if the intention is to guarantee against it – as it can backfire and create a higher level of anxiety.
The policy that did best in this series of tests was:
We guarantee 100% privacy. Your information will not be shared.
I’m currently testing it on my newsletter form, and at the moment the treatment with the policy is performing 6% better than the control – however, the data isn’t conclusive yet.
