How To Run AB Tests In React

6 min read

I was recently faced with a situation where I needed to run an AB test from within my React app. The platform that we use to run AB tests is Google Optimize, a free Google product that helps you run experiments on your website by varying parts of it and analysing the resulting metrics you care about.

Up until this point, the experiments we ran involved changes that were so wide-ranging and numerous that it made sense to just set up an entirely new route and split our visitors on the page prior to entering the React app. i.e. 50% sent to mywebsite.com/enteringreactnow/v1 and 50% sent to mywebsite.com/enteringreactnow/v2.

Setting up entirely new routes is super easy to set up, both on React and on Optimize, but isn't an efficient option if you only need to, say, test one component within your app. So what to do?

Option 1: Use your own feature flags and break free from Optimize

This option is more of a hack. I came across a very specific situation where the experiment was to either show / hide a particular screen route. I had set up my app so that the sequence of screens a user goes through is pre-set. I then use a custom hook to determine any conditional flows for navigation (useNavigator) e.g. whether to show or hide a particular screen as the user is clicking through this linear sequence of screens.

The essence of what I was trying to do was to show / hide a string (the path name) in an array of strings (the end to end flow), rather than showing or hiding a component. There didn't seem to be an obvious or easy way of doing this with Optimize. I therefore decided to implement the function of breaking users into random groups myself, then writing the logic that sends them down one experiment path vs. the other.

To create random groups,

const ALL_GROUPS = ['A', 'B']
type Group = typeof ALL_GROUPS[number]
const randomGroup = (): Group => ALL_GROUPS[Math.floor(Math.random() * ALL_GROUPS.length)]

The group allocation of each visitor is stored in the Redux state and persists to local storage. This ensures that visitors that revisit the app retain their original group allocation and thus have the same app experience. This is similar to how Google Optimize does this, where they use cookies instead.

From here, it's easy to write the logic of whether to show or hide a particular screen depending on the group someone belongs to. The harder thing is to have to monitor and calculate the experiment results yourself. You'd need to use something like Google Tag Manager + Google Analytics to set up event tracking, then be comfortable with calculating how statistically significant the variation in results are. In all likelihood, if you're already using Google Optimize, you'd likely also have set up Google Analytics, so it would just be the actual analysis of results you'd have to deal with.

Pros

Easy to implement in React.

Cons

Not scalable.
To run more than 1 text at the time, you'd need to store multiple "groups" in the Redux store so that each test is truly randomised.
You'd need to set up analytics collection and analyse results yourself.

Option 2: Use the Google Optimize API

In essence, what you need to do is:

Paste the Optimize script within the <head> tags in your index.html file. This script will include your Google Analytics ID and Optimize ID.
Set up your experiment in the Optimize dashboard. While you're here, be sure to Run Diagnostics to check that the script has been installed properly and is running on your app.
You can either choose to set up the control and variants for your experiment using the dashboard, or alternatively, code in the variants directly in your React app. If you just want to test simple variants like copy change, or the colour of a button, then the dashboard view will probably suffice. For more complex tests, you'd probably want the ability to set this out from within React.
To do this:
- Ensure you've changed the activation event to "custom" and make a note of the event name. The default is optimize.activate but feel free to rename this.
- Make a note of the experiment ID.
In your code base, within the component to be tested, and upon the component mounting, you'll want to launch the activation event (i.e. launch Optimize), check when Optimize is ready to roll, then communicate to Optimize what experiment ID this component relates to.
From here, you just have to tell Optimize what you want to render for the various variants and Optimize + Google Analytics will take care of the rest (from the segmenting of users to different groups, to storing cookies, all the way through to reporting).
Check out this article for a step by step on how to implement if you fancy giving it a go.

Option 3: Similar to option 2 but use a pre-built package

The great thing about how popular React is, is that someone, somewhere has probably already come across the problem you face and has created a package.
One such example of a package that solves this problem is the aptly named react-optimize. Check its GitHub repo for more info.
It basically does all of what's in Option 2 for you, and provides nice wrapper components for you to interface with, like <Experiment id="<experiment-id>"> and <Variant id="0">. Looking under the hood, it's pretty lightweight and does all the management of loading Optimize at the right time, for you. You will however, need to stick to the default custom activation event name of optimize.activate for it to work.
The way it pulls everything together is through React's context hook.

Option 4: Use a pre-built package, but send data elsewhere i.e. not Google Analytics

If you don't fancy using Google Optimize + Google Analytics, check out the react-ab-test package from Marvel App.
The package has helpers available for Segment and Mixpanel, but it allows for flexibility on where you want to store tracking data through the use of emitters.
As you won't have a Google Optimize dashboard to set up things like experiments, weightings and number of variants, all of these can also be set up from within your React code. It's kinda nice that everything is kept in one place. 😎

For my experiments, I ended up going with Option 1 and Option 3. They both sufficed for what I needed, without needing too much set up time. Not knowing whether it was something I'd be wanting to do continuously, over the long term, also meant I needed something quick, and easy to understand and review (and potentially kill if not needed).

Jambo, I'm Julia.