Google Includes Split Testing Inside Analytics

Kills Website Optimizer

On Friday Google announced that it would now be including Split Testing inside of Google Analytics and doing away with Google Website Optimizer by the end of July.

The new solution is called Content Experiments and on first glance it seems a good move but a quick inspection reveals some worrying flaws in Google’s move.

See video below for Google's announcement of the new service.

No Multivariate Testing

The first, and highly worrying one, is that there is no option to carry out multivariate testing in the new process.

In other words, if you wanted to test specific elements on a page or site wide, such as a “Buy Button” you can no longer do this in Content Experiments.

This seems like an incredible omission on Google’s part, but perhaps it will be added back later.

In any event, it makes the new service less than useful.

No Results for Two Weeks

The other worrying change is that Content Experiments won’t show you any results for two weeks.

This is supposedly to encourage greater statistical significance. However, on high traffic sites, it is often possible to get a result in a matter of hours, and make business decisions accordingly in real time. A two week delay in getting data seems very unhelpful.

Also, as anyone who’s ever looked deeper into testing results will know, after you’ve tested your changes on your high value visitors in the first few days, what you end up testing after that is the effect of the changes on your low value visitors.

If you’ve ever seen your results cross over during a test, you’ll know what I mean.

Again, waiting for weeks seems more likely to encourage false conclusions and wrong business decisions.

No Ecommerce as Goal Tracking

Given that the intention of many tests is to improve revenue, the inability to use ecommerce results as goals is another mystifying omission.

{{experiments}}

Easier to Use

On the plus side the new Content Experiments is easier to use and set up. Having it inside Google Analytics obviously makes it easier to access and see results. Google’s hope is that this will encourage more users to start testing.

Also the old Google Website Optimizer never used to play nice with Google Analytics, so this integration should remove all of the integration traps previously inflicted on unsuspecting users.

The new service lives under the Content menu in the Google Analytics side menu.

Conclusion

Google Website Optimizer was never the best tool to use for split testing or multivariate testing, and this new option with all its omissions and faults will probably do more to push serious testers towards using alternative solutions such as Unbounce, Monetate, Liveball, Visual Website Optimizer and Optimizely to name a few.

That may not have been Google’s intent, but I think it is the most likely outcome.

Google's announcement.