Automated Accessibility Part 3: Regression Tests

Automated libraries such as axe-core and pA11y have been a very seamless way to bring accessibility testing into development teams UI testing. It can get development teams to begin to learn and grow accessibility in their teams. However, one big problem has appeared since the rise in popularity of these libraries.

Using these UI libraries has lead to teams thinking this is "all the automated testing we can do for accessibility" or that this is a "great start, but there isn't anymore". This thought process, has made development teams not go beyond just using those libraries to test.

Did you know that there is more automated testing you can do to help ensure the accessible functionality of the web content you create is accessible? It's true! Lets dive in!

Why A11y Regression Tests?

Automated testing libraries generically test your page or component for about 1/3 of accessibility issues. Yes, it is very simple to setup and make a test case like you can with Cypress-Axe:

  beforeEach(() => {
    cy.visit('https://www.spacejam.com/1996');
    cy.injectAxe();
  })

  it('is an accessible homepage', () => {
    cy.waitFor(500);
    cy.checkA11y();
  })

We can go beyond just checking generically and write specific regression tests that ensure the accessible functionality of the component or page we created.

When we say accessible functionality, we are not talking replacing screen reader testing or a manual audit. It is quite the opposite actually. We are talking about using the results from a manual audit to help build automated tests that ensure that the HTML and JavaScript created are behaving as expected so when assistive technology is used they can be accessible.

Let's take at a couple of example scenarios where we could test the accessible functionality of the content we created!

Writing A11y Regression Tests

Let's say we are a development team who is using Cypress for our UI automated testing. We have two new components that we just wrote up, an expand collapse section and a more information modal.

When building the expand/collapse button, we include the state of the button using aria-expanded. We can test to ensure that there is expanded state before click(with keyboard included), after opening and then closing too!

  it('A11y - FAQ proper expanded state', () => {
      cy.get('#toggle').invoke('attr', 'aria-expanded')
        .should('eq', 'false');

      cy.get('#toggle').trigger('click');
      cy.wait(150);
      cy.get('#toggle').invoke('attr', 'aria-expanded')
        .should('eq', 'true');

      cy.get('#toggle').trigger('click');
      cy.wait(150);
      cy.get('#toggle').invoke('attr', 'aria-expanded')
        .should('eq', 'false');
  });

As simple as that! Now we have tested that ARIA has been properly set for the component!

For our modal component, we need to ensure the focus order is proper. Focus order for modals is one of the most common accessible issues, and the great part is we can write regression test to ensure it properly works for all users!

The focus order on click of the button should go into the modal, in this case on the heading, and then also when it closes back to the button that triggered it.

  it('A11y - Focus order on modal', () => {
    cy.waitFor(1000); 
    cy.get('#modalButton').trigger('click');
    cy.focused().should('have.attr','id').and('eq', 'modalTitle');

    cy.waitFor(500);
    cy.get('#modalClose').trigger('click');
    cy.focused().should('have.attr','id').and('eq', 'modalButton');
  })  

Just like that we have written two accessible automated tests that ensure both the HTML and JavaScript we have written are functioning accessibly!

These are only a couple of examples that are possible, there are many more including:

• Focus indicator being set on actionable items
• Custom buttons are keyboard acccessible
• Disabled button state
• Testing ARIA states
  • Hidden
  • Pressed
  • Selected
• Link purpose (ensuring links have different text)
• Focus Order
  • Skip to main links
  • Modals
  • Page change

The Benefits

One thing to make clear, writing these does NOT end manual testing with screen reader or keyboard. The idea is to be an enhancement for those that:

• Makes debugging for accessibility issues easier
• Ensures new code is accessible
• Speeds up validation
• Makes A11y apart of acceptance criteria.
• Builds developers that think a11y first.

In Summary

It is time to take automated accessibility farther than where it's at in the field now. Automated libraries are a great starting point for generically catching issues, but we can build so much more!

Accessibility regression tests can help build a culture where tests MUST include accessibility as part of functional UI testing, and encourage developers to know and understand what it takes to make their components accessible.

The question is, why haven't you started yet?!?

For more information and examples on automated accessibility check out my new library on GitHub that will be updated weekly with new examples: https://github.com/Steady5063/Automated-Accessibility-Example-Lib