HN Explorer

Ask HN: Is there an A/B test service for developers?

1 points by hurrymaplelad ↗ HN
I'm looking for something one level more abstract than Optimizely or SendWithUs that makes it easy for non-developers to start a test, add variants, and view results, but has a developer focused api for integrating a test with webpages, emails, and mobile apps, and reporting results.

As a developer, I'd like more control over the performance impact of running a test on web and mobile apps, and the opportunity to validate emails and webpages after pulling in test variants to guard against misconfiguration.

Does anything like this exist as either a paid service or an open source project?

2 comments

[ 3.8 ms ] story [ 13.6 ms ] thread
shostack
Were there gaps/issues with the Optimizely or SendWithUs APIs such that they didn't suit your needs?
hurrymaplelad
I want variants defined as data, like JSON.

From the Optimizely API docs: "Variations define the code that should be applied on a page to change the experience". Their Variants are JS and CSS, which is great for web pages, not useful for mobile apps or emails.

I'm imagining variants defined as JSON, perhaps a markdown string, perhaps a boolean, perhaps an array describing section order. As a developer I'd get an experiment variant while rendering a page, and could use the variant data to make make changes to the rendered response.

A similar flow would work for emails and mobile apps.