How to set up A/B testing with Microsoft Clarity and GA4

As a marketer, you want to improve your website performance and generate more conversions. In many cases, A/B testing can be the answer.

By comparing two versions of a web page, you can determine which one is more effective in achieving your goals.

In this article, we will guide you through the steps of setting up an A/B test using Microsoft Clarity and GA4.

Microsoft Clarity is the free heatmap analytics tool that has all the segmentation functionality needed for A/B testing preparation, especially when Google Optimize ends up being a marketer; You need alternative ways to run your tests.

The easiest and most direct way to set up A/B testing is to set up two versions of a web page and direct traffic to both of them.

With Microsoft Clarity’s URL filter, you can segment and analyze data for different versions of your web page.

But what if you want to test different page layouts on direct traffic without different URLs?

Fortunately, Clarity has custom tags (and custom dimensions for GA4), so you can tag and filter users in reports.

What are Microsoft Clarity custom tags?

Clarity custom tags are custom alphanumeric labels that you can assign to a visitor and use later to segment data, analyze recordings, and heatmaps for different test suites.

Are there limits to custom tags in Microsoft Clarity?

There are no limits. You can add many tags to your project without any limitation or limitation.

How to tag a visitor with Microsoft Clarity

Tagging is as simple as running a small snippet of JavaScript code:

clarity("set", "experiment", "group_name");

But I would like to guide you through a specific real-world example of how to use this from our experience.

At SEJ, we run various tests on different types of ads and web page layouts to gain insights into how user behavior is affected by factors such as banner ad type or web page layout.

Examples of A/B tests we conduct:

• Static banner ads vs. animated banner ads.
• Left sidebar vs. right sidebar.
• Change menu labels.

The goal is to understand when users are scrolling deeper into the article and thus engaging in reading – or whether changing menu labels can help drive more clicks.

1. A/B testing static banner ads vs. Animated banner ads

We are using Google Ad Manager to load ads on our web page, thus we can use Google Publisher Tag API to pass key values ​​to our ad server.

We distribute the traffic evenly by using the Math.random() function in JavaScript, which returns 1 or 2.

To run the experiment, copy and paste below.

We are using ‘ad_type’ with predefined values ​​’static_ads’ and ‘animated_ads’ in GAM so that we can run ad reports on the GAM side as well, such as CTR for each group.

Next, in section to your webpage, copy and paste the JS code, or you can use the GTM custom HTML tag on each pageview where there are ads.

<script>
   window.group_name = "animated_ads";
   let randomNumber = Math.floor(Math.random() * 2) + 1; // either 1 or 2
   if( randomNumber == 2 )
      group_name = "static_ads";
      
document.addEventListener('DOMContentLoaded', function()    
   //make sure publisher tag has loaded   
   if( typeof googletag != 'undefined' )   
      googletag.pubads().setTargeting("ads_type", group_name );
   
   //check if clarity has loaded and set tag "experiment" with values "static_ads" or "animated_ads"
   if( typeof window.clarity != 'undefined' )
      window.clarity("set", "experiment", window.group_name );
   
);
</script>

When the ‘DOMContentLoaded’ event is fired, the publisher and clarity tag are normally loaded. If not, you can consider wrapping JS inside in setTimeout() function with a little delay.

Using the ads_type switch in GAM it is possible to configure different banner types to be displayed for each group. Since this key is set up as a flag value for the “experience” key in clarity, we can analyze the data for each group and run your reports.

If you need a specific setup that requires advanced coding, you can try to use ChatGPT to write code for you.

If you want to track how users’ conversion rates change in GA4, you can add a custom dimension using the main “experience” in GA4 and populate it on load the initialization tag using the datalayer.push method.

dataLayer.push( 'experiment': group_name );

Alternatively, you can use GTM JavaScript variable to get the value of the window.group_name global variable that we defined above as a test parameter.

And in the config tag, prepare a custom dimension to pass this variable value as shown below:

Populate the “experience” custom dimension from the JS global variable window with the collection name, and voila!

Now the custom dimension of your experience is passed to GA4, and you can filter reports using the custom component “Experience”.

(Quick tip: when naming your custom dimensions, make sure you don’t use any of the reserved parameter names to get it to work properly.)

2. Left sidebar vs. Right sidebar

The principle is the same. Use the Math.random() function in JavaScript to split the test.

<style>
/*when adding this class to the content div it swaps sidebar position */
.main_content_right
flex-direction: row-reverse;

</style>
<script>
   // since we have no any css under .main_content_left class it will do nothing i.e. sidebar will be the default right;   
   window.group_name = "main_content_left" 
   let randomNumber = Math.floor(Math.random() * 2) + 1; // either 1 or 2. 
   //let randomNumber = Math.floor(Math.random() * 5) + 1; // random number from 1-5. use this if you want to run test on the sample of your traffic e.g. on the 25%.
   if( randomNumber == 2 )
      group_name = "main_content_right" // we will use group_name as a class name and a custom tag at the same time;
   
//If DOMContentLoaded has loaded run the code, otherwise attach an event listener   
if (document.readyState === 'complete') 
     move_sidebar( group_name )      
    else 
   // DOMContentLoaded event has not yet fired
   document.addEventListener('DOMContentLoaded', function() 
       move_sidebar( group_name );
   );
   
function move_sidebar( class_name )   
   document.querySelector('.sej-sect>div').classList.add(class_name);// add class 
   //check if clarity has loaded and set tag "experiment" with values "right_sidebar" or "left_sidebar"
   if( typeof window.clarity != 'undefined' )
      window.clarity("set", "experiment", class_name );
   
   console.log('sidebar position', class_name );

</script>

In this case, we are dealing with the DOM to change the layout.

In your particular case, you may need to implement a different CSS for layout adjustments. You can use ChatGPT as a useful tool to help you with custom coding.

After a certain period of time, when you have enough sample data for segmented testing, you can use the Microsoft Clarity tag filter “experiment=main_content_left” or “experiment=main_content_right” to segment your data.

3. A/B Test List Labels

Once again we will use the Math.random() function and manipulate the DOM via JavaScript.

We want to test the “Recent” vs. “News” menu label in the navigation bar on our site.

So, let’s get the JS path using DevTools Browser as shown below.

We will use the JS path to access elements in the DOM and change the naming.

<script>
   //We want to test the menu label for the Latest section in our website's navigation bar
   window.group_name = "Latest" 
   let randomNumber = Math.floor(Math.random() * 2) + 1; // either 1 or 2. 
   //let randomNumber = Math.floor(Math.random() * 5) + 1; // random number from 1-5. use this if you want to run test on the sample of your traffic e.g. on the 25%.
   if( randomNumber == 2 )
      group_name = "News" // we will use group_name as a label and a custom tag at the same time;
   
//If DOMContentLoaded has loaded run the code, otherwise attach an event listener   
if (document.readyState === 'complete') 
     change_label( menu_label )      
    else 
   // DOMContentLoaded event has not yet fired
   document.addEventListener('DOMContentLoaded', function() 
       change_label( menu_label );
   );
   
function change_label( menu_label )   
   document.querySelector("#main-navigation > li:nth-child(1) > a").textContent=menu_label;//set label, in your case it will be different
   //check if clarity has loaded and set tag "experiment" with values "right_sidebar" or "left_sidebar"
   if( typeof window.clarity != 'undefined' )
      window.clarity("set", "experiment", menu_label );
   
   console.log('menu label', menu_label );

</script>

To add your code, you can either enter it in a section from your webpage or using GTM, as described earlier.

Note that if you’re tracking with GA4, you’ll also need to use a custom dimension.

To pull reports in GA4, you’ll need to use Explorer Reports and filter the metric by the “Experience” custom component.

Conclusion

A/B testing tools can be expensive to buy, and they may not always offer the specific features you need to achieve your goals.

For example, none of the A/B testing tools we tried can provide an intuitive interface for testing different types of ads without custom coding.

However, the methods described here may require some effort to learn custom coding.

With ChatGPT available to help you write code, however, the process shouldn’t be too difficult.

More resources:

Featured image: Burdun Iliya/Shutterstock