Skip to content

Commit a9d1caa

Browse files
marktnoonantbiethmanjennifer-shehane
authored
Add Cypress Rules documentation (#6215)
* update supported providers for the Results API * wip docs for Cypress rules * add detail, update changelog * changelog updates * improve alt text * update wording for website crawling * tweaks * lint fix * fix typo in rule id * remove temp file * Update docs/accessibility/core-concepts/cypress-rules.mdx Co-authored-by: Tyler Biethman <tbiethman@gmail.com> * Update docs/accessibility/core-concepts/cypress-rules.mdx Co-authored-by: Tyler Biethman <tbiethman@gmail.com> * fix a typo * update rule text * add more detail * add more detail * lint fix * add more detail * Update docs/accessibility/core-concepts/cypress-rules.mdx Co-authored-by: Tyler Biethman <tbiethman@gmail.com> * Update docs/accessibility/core-concepts/cypress-rules.mdx Co-authored-by: Tyler Biethman <tbiethman@gmail.com> * remove unfinished example * changelog * Update docs/accessibility/core-concepts/cypress-rules.mdx Co-authored-by: Jennifer Shehane <jennifer@cypress.io> * Update docs/accessibility/core-concepts/cypress-rules.mdx Co-authored-by: Tyler Biethman <tbiethman@gmail.com> * Update docs/accessibility/core-concepts/cypress-rules.mdx Co-authored-by: Jennifer Shehane <jennifer@cypress.io> * add overall explainer at the top * Update docs/accessibility/core-concepts/cypress-rules.mdx Co-authored-by: Tyler Biethman <tbiethman@gmail.com> --------- Co-authored-by: Tyler Biethman <tbiethman@gmail.com> Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
1 parent e41ed2e commit a9d1caa

File tree

10 files changed

+249
-8
lines changed

10 files changed

+249
-8
lines changed

docs/accessibility/changelog.mdx

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,24 @@ sidebar_position: 200
99

1010
# Changelog
1111

12+
## Week of 7/7/2025
13+
14+
- A new custom accessibility rule, ["Interactive elements should be semantically correct"](/accessibility/core-concepts/cypress-rules#Interactive-elements-should-be-semantically-correct), is now available. This rule is based on the interactions that take place during your test execution, and helps detect accessibility issues that are not a part of typical DOM-based accessibility scans.
15+
16+
## Week of 6/2/2025
17+
18+
- We now support fetching Accessibility results from Drone CI with the [Results API](/accessibility/results-api).
19+
20+
## Week of 5/26/2025
21+
22+
- A new [element identification](/accessibility/core-concepts/element-identification) algorithm is now available for Cypress Accessibility. This produces improved element deduplication across different snapshots and runs, and incorporates testing-specific attributes like Test IDs into the element recognition and deduplication process. This replaces the default Axe-Core® element identification behavior.
23+
- Two new configuration options are available in Cypress Accessibility - [`significantAttributes`](/accessibility/configuration/significantattributes) and [`attributeFilters`](/accessibility/configuration/attributefilters). These allow you to fine-tune Cypress's element identification behavior to ensure stable and relevant locators are used.
24+
- Branch Review for Cypress Accessibility is now generally available, with an updated interface for displaying new and resolved violations, based on feedback during the Beta period. Thank you all for hitting the feedback button! [Learn more about comparing reports](/accessibility/core-concepts/compare-reports).
25+
26+
## Week of 5/12/2025
27+
28+
- We now support fetching Accessibility results from AWS CodeBuild with the [Results API](/accessibility/results-api).
29+
1230
## Week of 3/24/2025
1331

1432
- Cypress Accessibility results are now included in the [Data Extract API](/cloud/integrations/data-extract-api) so that you can retrieve data over time.
Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
title: 'Cypress Rules | Cypress Accessibility Documentation'
3+
description: 'Review the main areas to pay attention to when first reviewing an accessibility report for a Cypress run.'
4+
sidebar_position: 70
5+
sidebar_label: Cypress rules
6+
sidebar_custom_props: { 'new_label': true }
7+
---
8+
9+
<ProductHeading product="accessibility" />
10+
11+
# Cypress rules
12+
13+
## What are Cypress rules?
14+
15+
In addition to running the default ruleset of the Axe-Core® library, Cypress develops custom rules that take advantage of the additional layer of information available in a test automation context. These rules are identified by a "Cypress Rule" badge, to distinguish them from the main Axe-Core® rule set.
16+
17+
The first custom rule that we have implemented is called "Interactive elements should be semantically correct." At launch, it is a non-blocking **manual-review** rule that will either pass if no issues are found, or return an inconclusive state.
18+
19+
## Why do we need custom accessibility rules?
20+
21+
Conventional accessibility checks work by scanning the DOM of a web page to discover errors and inconsistencies that would affect the accessibility of the content. These checks use signals based on the nature of the content itself to determine what the correct accessible experience should be. Buttons, links, and form fields without labels can all be detected this way - if they are written correctly.
22+
23+
This form of "passive" accessibility scan is limited to what can be learned from the page itself, because there is no other source of input about the purpose of the elements on the screen. The tools used are typically general-purpose scanners that can be called in many different contexts, not always in a test automation pipeline.
24+
25+
In Cypress Accessibility, we can build rules that move beyond a passive scan and offer more insights about the accessibility of what you are testing based on your existing test activities, which already describe how a user is supposed to move through the application and what elements they use.
26+
27+
This helps provide early detection of issues that might later appear in manual testing or in bug reports from users, increasing the total range of issues detectable in an automated scan without custom assertions or setup.
28+
29+
## Interactive elements should be semantically correct
30+
31+
- **ID**: `cy-semantic-html-warning`
32+
- **Tag**: `cypress-rule`
33+
- **Severity**: `serious`
34+
35+
:::caution
36+
This rule requires that runs be recorded using Cypress version 13.14.0 or above.
37+
:::
38+
39+
<DocsImage
40+
src="/img/accessibility/core-concepts/cypress-rule.png"
41+
width="60%"
42+
alt="A custom Cypress accessibility rule called 'Interactive elements should be semantically correct' has two failed elements displayed. One element is open and message reads 'This element should use semantic HTML or ARIA to support the interactions that take place during tests. Detected interactions: mousedown, mouseup, click."
43+
/>
44+
45+
This custom rule is intended to surface potential usability problems with the kinds of elements that Axe-Core® traditionally does not evaluate as interactive, such as div, span, SVG, and image elements.
46+
47+
This works based on the interactions that take place during testing. Often an inaccessible custom component will have Cypress tests that perform interactions the way a user does, and that allows us to detect a mismatch between the underlying purpose of the element in code, and the usage of the element in the context of a test.
48+
49+
### How to pass this rule
50+
51+
You should look at the purpose and nature of the specific elements flagged by this rule and decide what interactive element is appropriate.
52+
53+
For example:
54+
55+
- If the element that gets flagged is a span that has been styled to look like a button and submits a form, the correct solution would be to instead use a standard HTML button element. This communicates the expected action and is activated with the space bar or enter key.
56+
57+
- If the same element is styled with underlined text, and when clicked it resets the URL to navigate to a new page, the correct solution would be to instead use a link element with a `href` attribute.
58+
59+
In each case, the solution is to choose the element that provides the best information to the user, so they can independently understand what the action does, and activate it.
60+
61+
After you have updated to a suitable HTML solution, elements will then be able to be correctly parsed and analyzed by the standard Axe-Core® checks.
62+
63+
:::info
64+
**Tip**: Follow the [first rule of ARIA use](https://www.w3.org/TR/using-aria/#firstrule) and implement interactive controls with standard semantic HTML where possible, relying on ARIA `role` attributes only when no suitable HTML element is available.
65+
:::
66+
67+
### Why this matters
68+
69+
In the Web Content Accessibility Guidelines (WCAG), interactive elements are required to meet a wide range of Success Criteria (SCs) in order to follow the POUR principles, which stands for Perceivable, Operable, Understandable, and Robust.
70+
71+
Interactive elements that are not semantically correct often don't appear in automated scans, but do appear in manual testing, because they are usually not operable with a keyboard or correctly understandable when described by a screen reader.
72+
73+
Custom components implemented this way have a direct impact on the independence of disabled users.
74+
75+
### Related WCAG Success Criteria
76+
77+
An incorrectly implemented interactive element may be failing multiple Success Criteria from WCAG, for example:
78+
79+
- [SC 1.3.1 Info and Relationships](https://www.w3.org/TR/WCAG21/#info-and-relationships)
80+
- Elements may incorrectly appear to be non-interactive generic elements
81+
- [SC 2.1.1 Keyboard](https://www.w3.org/TR/WCAG21/#keyboard)
82+
- Non-interactive elements require custom implementations for keyboard accessibility and focus states
83+
- [SC 4.1.2 Name, Role, Value](https://www.w3.org/TR/WCAG21/#name-role-value)
84+
- Non-interactive elements require explicit ARIA roles to have their function and name correctly described by assistive technology such as screen readers
85+
86+
### Status
87+
88+
This rule is available in all Cypress Accessibility projects, though it is not enabled by default. Reach out to Cypress if you would like to have this rule enabled for you.
89+
90+
### Possible outcomes
91+
92+
For each run, this rule can be in one of four states. Most importantly, it will never fail, which means it will not affect your accessibility score.
93+
94+
| Outcome | Description |
95+
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
96+
| **Pass** | No interactive elements with semantic issues were detected during the test run. |
97+
| **Inconclusive** | Interactive elements with potential semantic issues were detected. You should review each element and determine the correct solution. |
98+
| **Not Applicable** | The test run did not interact with any elements, or the version of Cypress was below 13.14.0, so interaction data was not gathered. |
99+
| **Ignored by configuration** | The rule was not executed during this test run because it was intentionally turned off. |
100+
101+
### False positives
102+
103+
Please use the "Submit Feedback" button in the product to let us know if you see something that you think is a false positive. Our goal with any new rules is to follow the general Axe-Core® philosophy that minimizes false positives, so we will make adjustments for specific situations that are not already known as we learn about them.

docs/accessibility/get-started/introduction.mdx

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ sidebar_label: Introduction
33
title: 'Accessibility Testing | Cypress Documentation'
44
description: 'Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems.'
55
sidebar_position: 10
6-
sidebar_custom_props: { 'new_label': true }
76
---
87

98
<ProductHeading product="accessibility" />

docs/accessibility/guides/production-monitoring.mdx

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,19 @@
11
---
2-
sidebar_label: Production monitoring
2+
sidebar_label: Production monitoring and website crawling
33
title: 'Production monitoring | Cypress Accessibility Documentation'
44
description: 'Monitor accessibility issues in production by running scheduled Cypress tests against live environments, capturing dynamic content changes, and generating automated reports for a comprehensive accessibility overview.'
55
sidebar_position: 60
66
---
77

88
<ProductHeading product="accessibility" />
99

10-
# Production monitoring
10+
# Production monitoring and website crawling
1111

12-
Cypress is commonly used in CI and local development workflows. It can also be leveraged to monitor production or staging environments through scheduled tests. This approach is particularly useful for environments involving dynamic content, such as those managed by Content Management Systems (CMS), where content changes can impact accessibility.
12+
Cypress is commonly used in CI and local development workflows. It can also be used to monitor production or staging environments through scheduled tests. This approach is particularly useful for environments involving dynamic content, such as those managed by Content Management Systems (CMS), where content changes can impact accessibility.
1313

1414
Dynamic or externally controlled content, such as A/B tests or user-generated content, often makes it challenging to write explicit assertions. For these cases, production smoke tests can serve as high-level health checks, ensuring the accessibility of dynamic UI variations without requiring full test coverage.
1515

16-
## Using Cypress for Production Monitoring
16+
## Using Cypress for production monitoring
1717

1818
Cypress Accessibility enables you to test dynamic content seamlessly. By visiting a production URL within your Cypress tests and performing minimal UI interactions, you can capture the page's accessibility state in reports. This allows teams to detect accessibility issues introduced outside the regular development lifecycle.
1919

@@ -34,6 +34,9 @@ describe('Accessibility Scan', () => {
3434
Cypress._.each(URLs, (URL) => {
3535
cy.visit(URL)
3636
cy.contains('<YOUR_FOOTER_CONTENT>').scrollIntoView()
37+
// perform some repeatable actions here that influence accessibility, for example:
38+
// - change from light mode to dark mode
39+
// - modify the viewport size to simulate mobile screens and higher zoom levels
3740
})
3841
})
3942
})

docs/accessibility/results-api.mdx

Lines changed: 49 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ Fetching Accessibility results for a run supports fetching results for the follo
3434
- [GitLab](/app/continuous-integration/gitlab-ci)
3535
- [Jenkins](/app/continuous-integration/overview#Jenkins)
3636
- [AWS CodeBuild](/app/continuous-integration/aws-codebuild)
37+
- Drone
3738

3839
Please reach out to Cypress Support to request support for a different provider.
3940

@@ -59,6 +60,25 @@ If you check this in as a dependency, your installation will fail when we update
5960

6061
Write a script using the `getAccessibilityResults` utility to retrieve the results and perform one or more assertions to verify if the changes are acceptable. This script will be executed in CI.
6162

63+
#### Basic example
64+
65+
This snippet uses the `getAccessibilityResults()` helper to log out the results. It assumes your Project ID and Record Key variable are set. The following should work in any of the supported CI Providers out of the box:
66+
67+
```javascript title="scripts/verifyAccessibilityResults.js"
68+
// Assuming these environment variables are set:
69+
// CYPRESS_PROJECT_ID=your-id
70+
// CYPRESS_RECORD_KEY=your-record-key
71+
72+
const { getAccessibilityResults } = require('@cypress/extract-cloud-results')
73+
74+
getAccessibilityResults().then((results) => {
75+
// use `console.dir` instead of `console.log` because the data is nested
76+
console.dir(results, { depth: Infinity })
77+
})
78+
```
79+
80+
#### How to assert that only known rules are failing in the run
81+
6282
The Cypress App [repository](https://github.com/cypress-io/cypress) uses the Results API to ensure no new violations have been introduced. You can reference [this script](https://github.com/cypress-io/cypress/blob/develop/scripts/verify-accessibility-results.js) as a real example.
6383

6484
```javascript title="scripts/verifyAccessibilityResults.js"
@@ -133,7 +153,7 @@ getAccessibilityResults({
133153
})
134154
```
135155

136-
#### `getAccessibilityResults` Arguments
156+
### `getAccessibilityResults` Arguments
137157

138158
`getAccessibilityResults` uses the following attributes to identify the Cypress run and return the Accessibility results:
139159

@@ -444,5 +464,33 @@ phases:
444464
+ - CYPRESS_INTERNAL_ENV=staging CYPRESS_PROJECT_ID=[slug] CYPRESS_RECORD_KEY=[KEY] node ./scripts/verifyAccessibilityResults.js
445465
```
446466

467+
</TabItem>
468+
469+
<TabItem value="Drone" >
470+
```diff title=".drone.yaml"
471+
kind: pipeline
472+
name: default
473+
474+
environment:
475+
CYPRESS_PROJECT_ID: example_project_slug
476+
CYPRESS_RECORD_KEY:
477+
from_secret: example_record_key_secret
478+
479+
steps:
480+
481+
- name: test
482+
image: node:latest
483+
commands:
484+
- npm install
485+
- npx cypress run --record
486+
487+
* - name: validate
488+
* image: node:latest
489+
* commands:
490+
* - npm install --force https://cdn.cypress.io/extract-cloud-results/v1/extract-cloud-results.tgz
491+
* - node ./scripts/verifyAccessibilityResults.js
492+
493+
```
447494
</TabItem>
448495
</Tabs>
496+
```

docs/cloud/account-management/users.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ The permissions for each user role for Cypress Cloud:
144144

145145
Cypress App users can "request" access to a given organization. If a developer on
146146
your team has access to Cypress and your project's source code - they can request
147-
to be given access to your organization from within the Cypres App. This means
147+
to be given access to your organization from within the Cypress App. This means
148148
instead of you having to invite team members up front, they can request access and
149149
you can choose to accept or deny them access.
150150

docs/ui-coverage/changelog.mdx

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,18 @@ UI Coverage now supports defining custom commands that will count towards covera
1616
- [`additionalInteractionCommands`](/ui-coverage/configuration/additionalinteractioncommands)
1717
- [`allowedInteractionCommands`](/ui-coverage/configuration/allowedinteractioncommands)
1818

19+
## Week of 6/2/2025
20+
21+
- We now support fetching UI Coverage results from Drone CI with the [Results API](/ui-coverage/results-api).
22+
1923
## Week of 5/26/2025
2024

2125
- Launched AI-powered Test Generation in Cypress Cloud, to help you quickly add tests for the untested elements detected in UI Coverage reports, in a way that follows your existing practices and conventions. For more details, read [our blog post](https://www.cypress.io/blog/add-your-missing-tests-faster-with-test-generation-in-ui-coverage).
2226

27+
## Week of 5/12/2025
28+
29+
- We now support fetching UI Coverage results from AWS CodeBuild with the [Results API](/ui-coverage/results-api).
30+
2331
## Week of 4/1/2024
2432

2533
- **Shadow DOM and iFrame Support:** UI Coverage now supports Shadow DOM and iFrames for reporting on interactions. Both of these settings are off by default. Please contact your Cypress representative to opt in.

docs/ui-coverage/get-started/introduction.mdx

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,6 @@ title: 'Test coverage with UI Coverage | Cypress Documentation'
33
description: 'Increase your test coverage with UI Coverage, a visual test coverage report based on the interactive elements that make up your application.'
44
sidebar_label: Introduction
55
sidebar_position: 10
6-
sidebar_custom_props: { 'new_label': true }
76
---
87

98
<ProductHeading product="ui-coverage" />

docs/ui-coverage/results-api.mdx

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ The utility supports the following CI providers. Refer to the linked guides for
2020
- [GitHub Actions](/app/continuous-integration/github-actions)
2121
- [GitLab](/app/continuous-integration/gitlab-ci)
2222
- [Jenkins](/app/continuous-integration/overview#Jenkins)
23+
- [AWS CodeBuild](/app/continuous-integration/aws-codebuild)
24+
- Drone
2325

2426
For other CI providers, contact Cypress Support to request support.
2527

@@ -324,5 +326,66 @@ workflows:
324326
- run-cypress
325327
```
326328

329+
</TabItem>
330+
331+
<TabItem value="AWS CodeBuild" >
332+
333+
```diff title="buildspec.yaml"
334+
phases:
335+
install:
336+
runtime-versions:
337+
nodejs: latest
338+
commands:
339+
# Set COMMIT_INFO variables to send Git specifics to Cypress Cloud when recording
340+
# https://docs.cypress.io/app/continuous-integration/overview#Git-information
341+
- export COMMIT_INFO_BRANCH="$(git rev-parse HEAD | xargs git name-rev |
342+
cut -d' ' -f2 | sed 's/remotes\/origin\///g')"
343+
- export COMMIT_INFO_MESSAGE="$(git log -1 --pretty=%B)"
344+
- export COMMIT_INFO_EMAIL="$(git log -1 --pretty=%ae)"
345+
- export COMMIT_INFO_AUTHOR="$(git log -1 --pretty=%an)"
346+
- export COMMIT_INFO_SHA="$(git log -1 --pretty=%H)"
347+
- export COMMIT_INFO_REMOTE="$(git config --get remote.origin.url)"
348+
- npm ci
349+
pre_build:
350+
commands:
351+
- npm run cypress:verify
352+
- npm run cypress:info
353+
build:
354+
commands:
355+
- CYPRESS_INTERNAL_ENV=staging CYPRESS_PROJECT_ID=[slug] npx cypress run --record --key [KEY]
356+
+ post_build:
357+
+ commands:
358+
+ - npm install --force https://cdn.cypress.io/extract-cloud-results/v1/extract-cloud-results.tgz
359+
+ - CYPRESS_INTERNAL_ENV=staging CYPRESS_PROJECT_ID=[slug] CYPRESS_RECORD_KEY=[KEY] node ./scripts/verifyUICoverageResults.js
360+
```
361+
362+
</TabItem>
363+
364+
<TabItem value="Drone" >
365+
```diff title=".drone.yaml"
366+
kind: pipeline
367+
name: default
368+
369+
environment:
370+
CYPRESS_PROJECT_ID: example_project_slug
371+
CYPRESS_RECORD_KEY:
372+
from_secret: example_record_key_secret
373+
374+
steps:
375+
376+
- name: test
377+
image: node:latest
378+
commands:
379+
- npm install
380+
- npx cypress run --record
381+
382+
* - name: validate
383+
* image: node:latest
384+
* commands:
385+
* - npm install --force https://cdn.cypress.io/extract-cloud-results/v1/extract-cloud-results.tgz
386+
* - node ./scripts/verifyUICoverageResults.js
387+
388+
```
327389
</TabItem>
328390
</Tabs>
391+
```
Loading

0 commit comments

Comments
 (0)