Ask HN: What a/b testing framework do you use?

3 points by dpix 9y ago ↗ HN

I am currently looking into using an ab testing framework for our company. We want something that is framework agnostic, preferably with a REST API but also with good language integrations already. It would be great to have tools to target tests at specific user demographics based on our user data.

We don't really want to build something from scratch.

I have been checking out Optimizely so far and it seems like it will tick most of the boxes. What other alternatives should I be looking into? What do you love and or hate about what you are currently using?

Interested in both paid subscription services and OSS

4 comments

[ 5.3 ms ] story [ 19.3 ms ] thread

JB_alarcon 9y ago ↗

Hey dpix, so full disclosure I work there but we (Kameleoon) tick your boxes too ;)

And we do have an awesome customer success team that can do either classic support or handle your testing for you.

But I'm obviously biaised so you'll have to check it out yourself :)

Usual suspects are: VWO, Optimizely, Adobe target, Oracle Maxymiser.

mauerbac 9y ago ↗

Hello!

I work at Optimizely so happy to answer any questions about our tools. We have a solution for the Web, as well as developer SDKs to A/B test in code.

You could also look into open source tools like Facebook's Planout: https://github.com/facebook/planout.

dpix 9y ago ↗

Thanks! One thing that is unclear is that you seem to have to create a project based on a language/toolset. Can you use that same project using different client sdks?

If I want to use the same project across both front-end and back-end, should I be choosing javascript or C#? Does it matter?

mauerbac 9y ago ↗

Good question -- If you create a single SDK project, you can use any of the SDKs. Use the C# SDK on the backend and JS SDK to track client-side events. Feel free to reach out with more questions.