Ask HN: Web Dev Testing Best Practices

For testing the front-end, take a look at Selenium: http://seleniumhq.org/

It's a bit of a pig to get used to initially, but I guarantee the first time you run a test and find a newly-introduced bug, it'll make your day - and your team's, for that matter.

Individuals and interactions over processes and tools. Working software over comprehensive documentation. Customer collaboration over contract negotiation. Responding to change over following a plan.

Drupal has its own version of SimpleTest for testing code:

http://drupal.org/simpletest/

Core ships with a lot of tests and most of the larger contributed modules do too. They're pretty helpful to run frequently to make sure you haven't broken anything but they won't test your configuration or any of your front end stuff.

For JavaScript unit testing I have been pleased with Siesta so far (http://bryntum.com/products/siesta/). The standard version can be automated using Selenium or PhantomJS.

I would recommend using the Selenium WebDriver browser automation framework (there are C# .NET bindings). This can be used in combination with a BDD approach using SpecFlow, which is also for .NET, or Cucumber for Ruby.

Here are two useful resources where you can learn more about these and successful automation approaches.

* The Secret Ninja Cucumber Scrolls (http://cuke4ninja.com/toc.html)

* Ideas for efficient BDD with SpecFlow through examples (http://ndcoslo.oktaset.com/t-4861) a very recent presentation, from NDC 2012, where the SpecFlow author goes through a couple of testing approaches.

Learn about the "Page Object" model approach to automated web testing. It's important for creating reusable, encapsulated tests that can evolve with the system under test. This is vital for advancing past the initial, small automated tests to much larger suites whilst still keeping them maintainable.

Learn to use the dev tools in Chrome.

Here's a great start: http://jtaby.com/2012/04/23/modern-web-development-part-1.ht...

I like to think of test coverage as a grid.

On the horizontal layers you have unit testing, integration testing, acceptance testing - which are tests at various levels of abstraction throughout your code base.

I then like to test the vertical slices tiers of the application - usually views, controllers, models, database etc. I like to see unit tests, integration tests, and 'acceptance tests' focusing in on each of those tiers to build confidence in the coverage of your testing.

Automation is absolutely key, and getting those automated tests into your continuous build and release process is definitely worthwhile.

I tend to believe that QA should focus equally on function and experience. Consequently, I'd:

1. Unit test both front and back end code. This thread is full of great examples for tools that will make that process easier.

2. Use a tool (I like jmeter) to frequently test how your project performs. You would be surprised how often small changes result in massive slowdowns, especially under load.

3. Bring basic usability testing into the QA fold. Web dev is extraordinarily client facing - projects live and die according to the whims of clients. Consequently, if you bring things like simple hallway testing into the development flow, your whole team will recognize how important it is to make things elegant and simple to use.

Most agencies that size do very little testing. Typically any testing is manual and done by a mix of developers and account managers. Not saying that you have to operate that way, but be aware that automated testing is a relatively foreign concept for most agencies.

I try to write unit tests for internally developed libraries. For some projects, Selenium tests will be written as well.

One of the problems I really wanted to tackle was ensure consistency in web rendering across browsers and over time. Basically, automating the process of "does this page look right?" That culminated in my startup (Mogotest) and the concept of Web Consistency Testing. Neither are right for everyone, but check them out. Feedback is always appreciated, too, even if it's about how I use Bootstrap :-)

A friend of mine introduced me to CasperJS and I love it. It's built on top of PhantomJS.

If you'll forgive the self-promotion, you might like to check out my web app, Testpad (https://ontestpad.com), as a lightweight tool for organising manual testing. It's basically structured checklists rather than formal test case management, and has been doing well at agencies both big and small. Because it's lightweight and flexible (think cross between Excel, Notepad and OmniOutliner) you can start out as simple as you like and iterate the complexity as and when it's appropriate.

And to answer your question: I am of course using it to test itself! Plus some selenium for basic automation, but without going overboard as the app is very javascript-heavy and interactive, which limits how much can be usefully/cost-effectively achieved with automation.

For backend testing, I like to write tests using nose in python, using the awesome Requests library to drive the web interaction. Because Requests handles cookies well (for whole sessions), and allows all http methods, and has great header, file, post params, etc support, it really makes driving webapps simple. A couple of functions to get a session into appropriate state (logged in with user or whatnot) and you have easy setup fixtures -- so you can try things in isolation quite well.

Strive to test on three levels:

1 - Unit tests, low-level method testing w/ mocked or no external dependencies (such as db)

2 - Integration tests, test against external dependencies, such as db, apis

3 - System tests, front-end testing that covers all parts of the system using a tool such as Selenium

Terminology varies and differentiation of levels, as well, but this can serve as a good guideline.

You'll need to have everything automated and integrated into a build system where you can reliably kick off a build and see the results.

A note on Selenium: any non-trivial setup is going to require a fair number of resources, both in time and environments. Also, you will have to dedicate time to maintaining tests and your UI changes. However, the pay-off is quite large when you can run and end-to-end set of tests on your application and know that you didn't break something.

edit: formatting

Take a look at Adobe Shadow (good writeup here: http://www.webmonkey.com/2012/03/adobe-shadow-simplifies-mob...)

It'll let you refresh changes on all your devices at once.

Interesting article about unit testing as it relates to API development: http://blog.seatsync.com/2012/06/unit-testing-as-path-of-lea...

Unit tests of lower level back-end functionality is always a good idea, but I find it actually makes API development simpler because it frees you from the problems and complications inherent in testing via your target client.

Disclosure: I wrote the article.

I work at a web design agency about that size. Good luck finding any budget for testing. Clients don't care, and if you budget for it most will go with a lower bid somewhere else.

As someone else said, it's basically account people clicking around, you fix stuff, they click more until they feel comfortable presenting it to the client. Then you find out that some bigwig at the client has a 5yr old blackberry where the layout doesn't work right, and that becomes your focus of your debugging for a week.

am I hell banned?

Just wanted to thank everyone for their input.

So from what I gather, eveyrone is a fan of selenium. I do recall it being brought up during my interviews.

Does anyone have a favorite resource they like? i.e: eBook, blog etc

When you type the words "web design agency", it makes me think that the right answer is "do not test". The projects more often than not are just not big enough to justify it.