Yes! This is very important to us in dealing with transactional email. Our Z-Test Method in Confidence.js pinpoints when results become statistically significant by calculating the required sample size at any given time…
Thanks for your question! I'll pass it on to Emily, our stats expert.
Hey! I'm the author of Confidence.js. Emily Malcolm and I have been working hard on this new approach for the past few weeks and we're super excited to share it! We're both here to answer any questions :)
Hey martian, thanks for the feedback. Your methodology is pretty sound - it looks like we use a similar technique. If you supply a small sample size here, you'll get the message, "There is not enough data to determine a…
The recommendation of 95% significance level is quite common across A/B testing resources. The main goal here is to be able to say which variant will consistently perform best (in this case, 95% of the time). I agree…
Hey Noel, I was in charge of the math here - I really appreciate your feedback. You seem really knowledgeable in this area and I would love to improve this A/B testing library. If you have some time to chat, email me at…
This is a feature that I would love to add to Confidence.js sometime in the near future.
Hey, I'm the author of Confidence.js which was used to power this A/B testing calculator. That sounds really interesting and worth looking into - thanks for the feedback!
You're right about revenue being a very important metric, and I think optimizing for conversions has its place too. I would be interested to hear the results of your repurposing!
Glad you like it, happy to have you contribute :)
Stats class R sleepy... Real world applications like A/B testing are definitely more interesting. We did a lot of digging and there aren't a lot of resources that are easy to digest - but this one really stood out to…
Hi sharnett, Thanks for bringing that to my attention - I agree that our explanation is a little more convoluted than it needs to be. I'll update that later today to be more "readable" ;)
Thanks for the feedback! Confidence.js comes from how we operate on A/B tests at sendwithus - we think of them as variants associated with results. While the code could definitely boil down to a few simple math…
Hey! I'm the author and I'd be happy to answer any questions you might have. Confidence.js is based on the A/B testing code that we use at sendwithus. A/B testing math is hard and we've worked really hard on making this…
The layout generator is so cool! Yoink.
Yes! This is very important to us in dealing with transactional email. Our Z-Test Method in Confidence.js pinpoints when results become statistically significant by calculating the required sample size at any given time…
Thanks for your question! I'll pass it on to Emily, our stats expert.
Hey! I'm the author of Confidence.js. Emily Malcolm and I have been working hard on this new approach for the past few weeks and we're super excited to share it! We're both here to answer any questions :)
Hey martian, thanks for the feedback. Your methodology is pretty sound - it looks like we use a similar technique. If you supply a small sample size here, you'll get the message, "There is not enough data to determine a…
The recommendation of 95% significance level is quite common across A/B testing resources. The main goal here is to be able to say which variant will consistently perform best (in this case, 95% of the time). I agree…
Hey Noel, I was in charge of the math here - I really appreciate your feedback. You seem really knowledgeable in this area and I would love to improve this A/B testing library. If you have some time to chat, email me at…
This is a feature that I would love to add to Confidence.js sometime in the near future.
Hey, I'm the author of Confidence.js which was used to power this A/B testing calculator. That sounds really interesting and worth looking into - thanks for the feedback!
You're right about revenue being a very important metric, and I think optimizing for conversions has its place too. I would be interested to hear the results of your repurposing!
Glad you like it, happy to have you contribute :)
Stats class R sleepy... Real world applications like A/B testing are definitely more interesting. We did a lot of digging and there aren't a lot of resources that are easy to digest - but this one really stood out to…
Hi sharnett, Thanks for bringing that to my attention - I agree that our explanation is a little more convoluted than it needs to be. I'll update that later today to be more "readable" ;)
Thanks for the feedback! Confidence.js comes from how we operate on A/B tests at sendwithus - we think of them as variants associated with results. While the code could definitely boil down to a few simple math…
Hey! I'm the author and I'd be happy to answer any questions you might have. Confidence.js is based on the A/B testing code that we use at sendwithus. A/B testing math is hard and we've worked really hard on making this…
The layout generator is so cool! Yoink.