The process of A/B split testing is a fairly new subject which relates to the old scientific method that was called a controlled experiment. While testing the efficiency of newer drugs, researchers split the volunteers into two groups, that is a treatment group and control group. In the scientific world, mostly all research experiments could be considered as a ‘split test,’ as they come complete with a control and variation, a hypothesis, and even a result that is statistically and carefully calculated. Controlled experiments are known to exemplify the finest scientific design for creating a causal relationship between some changes and also their influences on users’ behaviors which can be observed.
So what exactly is A/B testing? As depicted in Figure 1, a randomly chosen set of users are shown in variation A and other half is shown in variation B.
Figure 1 Simple A/B Testing
As per the figure, we can see that variation B is much more effective as its conversion rate is a bit better than variation A. This method of testing is only useful when we have a huge number of users. Moreover, the ratio of users does not always need to be 50-50. The A/B testing improves product development by using the data available. On sites like Amazon, Google or other technology giants, they conduct many such experiments all the time. Many features are launched using the A/B testing technique.
The experts in the fields of computer sciences or even machine learning might not be very acquainted with controlled experiments and might probably be curious about A/B testing having some unique benefits. For instance, a marketer detected that “if he advertised more in December, he would sell more in December”, but the better question was what is the causal impact of amount spent on advertising and sales? There are plenty of samples about unauthentic correlation that could help one to understand this point, like the correlation between the spending on science by the US and the amount of suicides by hanging.
