Have Your Cake and Eat It Too with A/B Testing

Hub One 1:00pm

Nobody wants to waste time building features that won’t be successful. Unfortunately, it’s often difficult to anticipate users’ reactions and once your feature is out in the wild, it’s hard to iterate and gauge the right direction to take a failing feature — are users finding it difficult to use? Maybe they just can’t find it within your app? Or maybe the UI doesn’t invoke engagement? Enter A/B testing: try out variations on the same feature with different segments of your user base, make changes on-the-fly without a new release, and have the data to back up what works best for your users. Enable your team to make confident, data-driven decisions to bring about the success of your features. The Capital One Wallet teams have actively been integrating A/B tests into new features and we’ve had successes even with seemly-insignificant A/B tests -- for example, slightly tweaking push notification text through A/B tests has led to a 5-10% increase in our notification clickthru rates! In this talk, you’ll learn the different types of A/B tests which can be conducted and some best practices around A/B testing -- how should you be testing? And when is it appropriate? You’ll learn some of the considerations of A/B testing that come specifically with Android and you’ll also see some examples of using third party libraries -- including Firebase Remote Config and Swrve. Get the best tips on how to choose a third party library to eliminate "gotchas" and even gain insight towards building an in-house solution. Embrace your indecisiveness, try all the possibilities, and maximize the potential of your future features.

Nick Capurso, Capital One

Nick’s experience with Android spans back to Ice Cream Sandwich, initially working heavily with location, inertial sensors, and Bluetooth. He has published peer-reviewed papers, including one on modifications to the AOSP, which was accepted into the IEEE IoT Journal. Aside from Android, Nick's interests also include IoT, operating systems, and teaching undergraduates at The George Washington University.