I’ve seen a few examples lately where the word “test” was bandied about when no such thing took place. One of my recent favorites was this article by the MD of a digital trading desk who claimed to be testing when, in fact, he was creating spurious data flotsam for a clickbait headline.

Takeaway number 1: Read the blog posts by the leaders of the digital marketing service providers you rely on for marketing, testing, and analytical advice. You just may decide to switch agencies.

Takeaway number 2: If you can’t design the gold standard marketing test–the univariate test–you have no business doing more complicated tests.

For a quick refresher course in statistics for marketing test design, check out Evan Miller’s excellent blog*. He’s written several articles on testing that statistics that will make your tests better today.

• How Not To Run An A/B Test – The hazards of repeated significance testing.
• Lazy Assignment and A/B Testing – Increase your statistical power with this weird tip.
• A/B Testing Duration Data – When you don’t know the standard deviation.
• The Low Base Rate Problem – Why A/B testing is often a waste of time.

A lot of the growth hacker babble I read–at least as much as I can stomach–leads me to believe they are advocating running Bayesian A/B tests, except without the correct math. Making them non-tests, but I digress.

To help solve that problem, Evan has also written an easy-to-understand article on setting up Bayesian A/B testing. It is worthwhile reading for those doing online testing such as home page design, path testing, cart tests and the like.

* A last plug for Evan. He creates and sells the wonderful Wizard statistical analysis software for Mac OS X. It is worth every penny. I highly recommend it!