Disclaimer: I have watched the video but not tried the product, so some of these observations may not apply to the the product itself.

Overall, this looks like a great product. I know a number of people who find Google Website Optimizer complicated to use, and I would definitely recommend this to them as a simpler option. I love how slick the browser interface to edit pages is, and I think having the default 'engagement' metric so people can see results without having to set up a goal page is a brilliant idea.

There are a few things you have done which I would consider doing differently.

1. It looks like your mission is to make A/B testing really easy, but your pricing page at the moment doesn't really reflect that. Number of visitors tested is an easy metric, but one that it is hard for me to interpret without lots of knowledge of A/B testing. How many tests does this mean I can run how quickly?

I would also reconsider the additional features you offer in premium packages. Cross-browser testing sounds complex and makes me worry that your site edits will fail in IE6. I don't want to have to test it, I just want it to work. With uptime monitoring, what does this have to do with A/B testing? Bigger sites probably already have some form of monitoring already anyway, so it looks like they are going to pay for something they don't need. I think your core product is strong enough that you don't need to offer these.

2. Showing me the percentage significance level appeals to my inner stats nerd, but I suspect the sort of people I think will benefit most from Optimizely will have difficulty interpreting this number. What level is 'ok'? Having a rank out of 5 below doesn't really address this, is 4/5 ok or do I need 5/5? Google deal with this very well with their bars which turn red or green when they reach significance.

3. The 'select container' option to expand the selection seems non-obvious, and isn't how multi-select works in any other interface I've seen. Maybe allow people to select multiple components and then take their deepest common parent?

There are also some additional features I personally would like to see

1. It would be great if you gave an estimate for how long until my experiment will reach an appropriate significance level (obviously based on % change and traffic seen so far).

2. I would like to be able to choose my conversion action by clicking on a form button or link in your page editor.

3. It would be amazingly useful to have some automatic suggestions for how a page could be changed - on many occasions I've seen people resist A/B testing because the options are so wide and they don't know what to do. Doing this for some simple suggestions sounds possible - e.g. making key links bigger and moving them up the page. Doing anything more sophisticated could be a good challenge though :-)