It's funny, the only use I see for this is IE testing, since everything else I can install the latest version of on my Mac, and the differences between versions are pretty small (and if you can support IE's lack of HTML5 functionality you can certainly do the same for FF3).
Still, solid UI and product, and will be handy for the quick IE spot-checks that I should be doing more of :).
I've run into the situation more times than I can count where there was some little display difference between, for example, FF on OSX and FF on Windows. Always pays to spend the short amount of extra time needed to test across operating systems.
One thing I never quite understood about online testing tools like browserstack and saucelabs is how to reset server state.
Most webapps have some kind of datastore. Poking around a web ui will CRUD records in that datastore, possibly corrupting the next test. Another problem with this is coupling between tests, making them hard to modify without breaking a bunch of others.
Most automated test setups I have seen run on a local machine, and all server state is reset between each test.
The way you'd usually do it is either having a multi-user application and just create a new user for a new test (also helps testing concurrently) or you'll have to reset the app by:
- redeploying it on a new e.g. EC2 instance
- clearing out the database and inserting your original blank state from an SQL dump
- clearing out the filesystem by e.g. using a git reset
p.s. you should try to have an automated deployment for your product. That way you could also just spin up a new EC2 instance -> deploy -> test -> kill instance
Thats entirely a different problem, as of now we are completely focused in providing browsers across OSes and Mobile platforms with a very easy to use UI!
By the way, just realized you are based out of India. Great job! Very happy to see a quality web product that is made in India. Who did your design? Love it!
I thought this was going to be another cross-browsing testing thing that wouldn't work for me, but the local access is a really good idea. It really does change everything.
34 comments
[ 2.3 ms ] story [ 67.3 ms ] threadhttp://twitter.com/#!/browserstack/status/110011514911592449
Still, solid UI and product, and will be handy for the quick IE spot-checks that I should be doing more of :).
Most webapps have some kind of datastore. Poking around a web ui will CRUD records in that datastore, possibly corrupting the next test. Another problem with this is coupling between tests, making them hard to modify without breaking a bunch of others.
Most automated test setups I have seen run on a local machine, and all server state is reset between each test.
How is this done on browserstack/saucelabs?
(not 100% sure about that, but I think Selenium mentioned that in the logging output)
- redeploying it on a new e.g. EC2 instance
- clearing out the database and inserting your original blank state from an SQL dump
- clearing out the filesystem by e.g. using a git reset
p.s. you should try to have an automated deployment for your product. That way you could also just spin up a new EC2 instance -> deploy -> test -> kill instance
99c to test in 5 different browser/OS combinations would be cool.
Can it handle httpauth?
This does appear to be cheaper though. The local testing thing may be cool. Not sure how that will work.
Also, if you're testing emails in different clients (worse than IE6 compatibility testing IMHO), I highly recommend http://litmus.com/
Obviously Android and iOS are webkit based... but what about the horror that is BlackBerry rendering? <shudder>
All I get is "Oops! We have run into a small problem. Please try again." over and over again.
Overloaded?