When running A/B tests - usually the best thing you can do is to partition your users randomly into subsets A and B, and for a large enough sample the populations will be similar enough by the randomness of the the partition.

However - if we can't partition on a user level, and we must use a geography based partition - the randomness assumption takes a hard hit. For example - if we were to place Tel Aviv in group A and the negev in group B - clearly a hugh bias is inserted into our partition. To avoid this bias we have to carefully design our partition to try and minimize this bias. We also need to analyze our results carefully.

In this talk I present how we design smart geographic partitions, to minimize the inherent bias in the partition. I also discuss how we analyze the results using Google's TBR library.