How To Perform A/b Testing In Polymer Web Components?

I'm encountering a situation at a customer: they want to do A/B Testing. As far as I know, this most of the time happens at LoadBalancer level (Kubernetes) redirecting users to a c

Solution 1:

not sure if this covers your full question - but within a Microservice architecture you'd test within each service on it's own.

Speaking of Kubernetes as your plattform to host your services you could have one LoadBalancer service which selects Pods from different Deployments. Each Deployment could provide different container/application versions or it could provide the same container but with different settings.

Here's a small example - the single services has a selector (app: testme) which matches pods from both deployments. The Deployments define containers from the same image (yourcontainerimage:version) but with different environment variables. Also the different amount of replicas would allow you to have different proportions of the traffic routed to one or the other option.

apiVersion:v1kind:Servicemetadata:name:appspec:ports:-name:httpport:8080selector:app:testme---apiVersion:extensions/v1beta1kind:Deploymentmetadata:name:app-deployment-aspec:replicas:2template:metadata:labels:app:testmeab:onspec:containers:-name:appimage:yourcontainerimage:versionenv:-name:FEATURE_TOGGLEvalue:trueports:-containerPort:8080---apiVersion:extensions/v1beta1kind:Deploymentmetadata:name:app-deployment-bspec:replicas:1template:metadata:labels:app:testmeab:offspec:containers:-name:appimage:yourcontainerimage:versionenv:-name:FEATURE_TOGGLEvalue:falseports:-containerPort:8080

Depending on your application and the type of feature you test, you might want to adjust the services and e.g. enable or disable SessionAffinity of the services. You'll find details in the official docs.

Solution 2:

The case of the distributed server side (e.g. micro services) is cleanly addressed in Variant. (Disclaimer: I work there). Whichever component touches the experiment first, creates a Variant session (independent of any notion of the session that the host application may have, e.g. HTTP Session) and then passes the session handle to the next component, which will be able to retrieve it and all the experiment related data from Variant server. The only catch is that we only support Java components at this time.

As well, using deployment infrastructure, like load balancer, for application concerns like A/B testing is a bad idea on many levels and should be abandoned.