The company I work for is a data-driven company. It is necessary for us to make the best decisions and aim for our core value: being customer-centric.
Naturally, when we iterate on a feature, we study the data to define how to improve it. And because we can’t improve what we don’t measure, we measure the results of our new version with the previous version.
To do this, we almost systematically perform AB tests that we couple with our events database.
AB tests are not for experimenting things, but to measure them.
And since it's HacktoberFest, we published one of our tools in open source here: Travaux.com VariantRetriever
Next part of this article is to start your first AB test, like changing the subject of an email when a new user registers.
The experiment we are doing will be called welcome-email-experiment
and it will have a control
version (the actual version) and a participant
(or variant) version (the new version we want to try).
When we will send this email, we’re going to require the VariantRetriever and ask it: for this experiment and this user identifier, tell me which variant it should have.
Lets run a new AB test with Symfony 5
First step is of course to install the AB test package in your project:
composer require travaux-com/variantretriever
Because it’s not a Symfony bundle, you don’t need to declare it in your AppKernel file, but you still need (recommanded in fact) to declare it as a service.
We’re going to declare it as a service with the Symfony container factory system to have our “variant retriever” with all information about our running AB tests. (welcome-email-experiment
and display-cta
).
services:
App\FeatureFlag\VariantRetriever:
factory: ['@Travaux\VariantRetriever\Factory\VariantRetrieverFactory', 'createVariantRetriever']
public: true
arguments:
- welcome-email-experiment:
- control-email: 50
- participant-email: 50
- display-cta:
- control-cta: 75
- participant-cta: 25
The service name is actually App\FeatureFlag\VariantRetriever
but I strongly encourage you to customize it.
Now you’re able to use the Symfony autowiring to retrieve the VariantRetrieverInterface and use it in your command handler.
And so on, how to use it ?
$affectedVariant = $variantRetriever
->getVariantForExperiment(
new Experiment('my-ab-test'),
(string) $user->getId()
);
VariantRetrieverInterface
actually came with the getVariantForExperiment method where you will have to define the experiment you want to test (by just filling the experiment name) and the “user identifier” who just need to be a string, so it can be a UUID v4 or an auto-incremented integer.
It will return you a Variant
value object that contains the name of the affected variant.
Now you can imagine select the right translation key to use depending on the affected variant for your email:
$experimentTranslationsKeys = [
'control' => 'email.welcome.subject_control',
'variant' => 'email.welcome.subject_variant',
];
$userVariant = $variantRetriever
->getVariantForExperiment(
new Experiment('welcome-email-experiment'),
(string) $user->getId()
);
$emailSubject = $experimentTranslationsKeys[(string) $userVariant];
This package will ensure you to always retrieve the same variant for your user, without any database or cache.
You’re now able to run your first AB in your Symfony project 🎉
Go beyond this example
As you can see, we’ve use the Symfony container factory system to instantiate our VariantRetriever, but it can also be declared with Symfony method call like this:
Travaux\VariantRetriever\Retriever\VariantRetrieverInterface:
class: Travaux\VariantRetriever\Retriever\VariantRetriever
calls:
- addExperiment:
- !service
class: Travaux\VariantRetriever\ValueObject\Experiment
arguments:
- 'welcome-email-experiment'
- !service
class: Travaux\VariantRetriever\ValueObject\Variant
arguments: ['control', 100]
- !service
class: Travaux\VariantRetriever\ValueObject\Variant
arguments: ['variant', 0]
How does it work internally ?
Thanks to php crc32 function, we’re able to process a string that return everytime the same integer value. So on, even if it’s a big number, we’re able to reduce it under a range of 0 to 99 that match the rollout percentage given by one of our variant.
Also, the Travaux\VariantRetriever\Retriever\VariantRetriever
class is not final to allow you to extend it. Like injecting a PsrLogger and/or the event dispatcher to keep a trace of which experiment have been affected to which user. You can also add you own method to retrieve a variant based on your User entity object like this:
public function getUserVariantForExperiment(
Experiment $experiment,
User $user
);
Feel Free to contribute to this project or give feedback.
Top comments (0)