How should I create a process for front end testing?

Question

We have a product similar to Photoshop and we have a lot of trouble in the testing period.

we started to use Agile Framework, in middle of project.

We have to use human manual testing to test visual effects (e.g. the opacity feature of background image)

Unit testing is not sufficient and is not applicable for the visual side (front end) of the application.

now we are using selenium for UI testing, but it is not enough, because testers have to check screenshots which tooken by selenium. And there are more than 500 screenshoots for every release test.

we tried manuel tests but we missed a lot of cases (ex : if z-index value of button greater than other elements, background opacity of button does not working). So i had a question "how do they test Photoshop's features in Adobe ?".

How should I test a very complicated user interface?

Answer 1

As Marv mentions in his comments, this is a very broad question, but I will try to float a few ideas that might be helpful (given the breadth of the question, I'm certain they will not be comprehensive).

First and foremost, test as you go. Do not wait until a test period if you can in any way help it. This is really true of almost any project. The test period gets crunched and not enough time is there for testing.

Challenge your developers and testers to identify lower-level tests to validate a lot of your diverse functionality - especially at the integration test level. For example, if I want to be able to lighten or darken a region, I should be able to create a number of automated test cases to validate that the right calculations are done and changes made to the color of the pixels. I could use something like Fitnesse to create a large set of tests that thoroughly test this functionality below the UI layer.

Of course, this work has to be validated on the UI too, but if you've verified that all the calculations work under the hood, you just need to test that the data makes its way into the UI properly, requiring far fewer tests. Again, challenge your team to find automated ways to test this. For example, a team I worked with recently used image matching to make sure that given a certain starting image and putting in certain parameters, you would get the expected resulting image.

This sort of work does take time, but it also adds a lot of stability. If you put in time throughout the project to invest in these automated regression tests, a substantial amount of what you now test manually will run in a weekend in an automated test and your manual testing can focus on the extremely complex and dynamic scenarios with plenty of time to spend on it.

Answer 2

As noted above, test as you go:

Are you using an Agile framework that promotes iterative testing?

What's your current testing process? Delivery process? Do your processes promote testing as soon as the code is checked in? Before?

Do you invest in test automation to cover common scenarios?

Have you looked at what must, should, and could be tested to determine which tests are most valuable in terms of business value?

Do you release an experimental or alpha version of your product to your user base frequently? If you are resource constrained its sometimes possible to get "free" testing from the community of end users.

If you answer "No" to any of the above, you have plenty of opportunities to improve your testing capability.