January is the time when all the data science experiments have been ready for the last two months but have been delayed because of the code freeze of Christmas and maybe even Cyber weekend. As a data scientist, this is a great time because you are probably allowed to test more risky things in production. But remember that you are allowed to do it because it is a slow period.

And be diligent. There are challenges with this month because your historical data is all over the place due to the changed habits of Christmas shoppers (see previous post), which could mean that models will probably not perform well in production.

Another reason to be extra on your toes when setting up the A/B tests is that the KPIs of most e-commerce sites are plummeting compared to last month’s capitalist spree of spending.

Are you sure you have data to prove that it is not somehow your experiment’s fault?