Unusually for me, my current project I'm doing in the traditional Web 1.0 style. It's a forms based app, and the server is rendering HTML direct to the browser. And it's really efficient to work this way.
Until you get to automated testing. With REST APIs, or GraphQL, or anything like that, you've got very obvious test points. For example, you have an API to get a user profile, and it just returns the exact JSON for the user. That's easy to test.
With forms and views, you have a controller to render the user profile page, and this will render the entire screen of which a part is the user details. But there's a lot of other things going on too - page headers, menus, etc.
Right now I'm doing snapshot testing. This is in Java, so I'm using jsoup to normalize the HTML - that ensures that irrelevant changes to the HTML, such as whitespace, don't affect the tests - and then asserting that the full rendered string is exactly correct.
It works, but I don't like it. It feels clunky, and - well - wrong. It also doesn't ensure that it renders correctly, just that the string is as expected. If the CSS changes then this will make the page look wrong but the tests won't catch it! But I'm not sure yet what to do instead.
So - what do other people do for this kind of testing? Is there a better way to do this?
Top comments (2)
If you don't have to deal with JavaScript, then I would actually parse the page and use Xpath or CSS selectors to locate the data of interest. There likely exists some nice tooling around this for Java (just b/c Java has so much momentum behind it). For Ruby, we use a tool called Capybara, so looking at that will likely give you a sense of what I'd expect the tool to look like.
I would avoid testing the aesthetics of it. Aesthetics are volatile and you don't want your test suite failing due to a change in presentation. Besides, it will likely render differently in different browsers, or on devices of different sizes. I've seen tools that will render your page on lots of different devices and then save screenshots of them, diffing screenshots with each release. But that struck me as fragile and painful, and I haven't felt much of a need for it. Probably a better solution is to keep your CSS modular so that you don't have to worry much about changing something for one page rippling out to all the other pages.
Also, in general, at this level of abstraction, I just do a single test for each path (eg happy path, sad path). And then the code that implements the endpoint quickly delegates out to simple objects that don't know anything about HTML, and know as little about HTTP as possible. Then you can test these objects much more easily. I probably unit test those objects in a much more traditional sense, but when I'm testing HTTP requests / responses, they don't look like unit tests, just 1 big request, assert as much as seems valuable (gives you confidence without becoming fragile), and move on. These tests tend to be much bigger, much more expensive, and I tend to have a lot less of them.
So, I'm actually doing three levels of testing:
I'm very happy with the Unit and End-to-End tests, but less so on the Integration tests.
I fully get where you're coming from on the selectors to pick out appropriate pieces of the page. However, I'm concerned that might miss things. For example, my "User Profile" page I could easily pick out the "Username" and "Email" fields and ensure they are populated correctly. But should I also pick out the "User menu" in the header bar and make sure that's correct? If so, do I do that on every page since it's rendered on every view? And if now then when do I test that, since it might feasibly be broken on only one view.
Conversely, my current approach of snapshot testing ensures that the header is tested on every view, and implicitly tests that every part of the page is as expected. But it's brittle to page-wide changes. If I change the header then every single snapshot is not invalid.