-
Notifications
You must be signed in to change notification settings - Fork 18
Open
Labels
wontfixThis will not be worked onThis will not be worked on
Description
Before opening, please confirm:
- I have searched for duplicate or closed issues and discussions.
- I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
- I acknowledge that I will attach a full debug log, otherwise the issue will be closed with no response.
Environment information
# Put output below this line
System:
OS: Linux 6.1 Debian GNU/Linux 12 (bookworm) 12 (bookworm)
CPU: (8) x64 Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz
Memory: 18.94 GB / 31.16 GB
Container: Yes
Shell: 5.2.15 - /bin/bash
Binaries:
Node: 18.16.0 - /usr/bin/node
Yarn: 1.22.19 - /usr/bin/yarn
npm: 9.5.1 - /usr/bin/npm
Browsers:
Chrome: 114.0.5735.90
Chromium: 116.0.5845.140
npmPackages:
eslint-config-custom: latest => 0.0.0
prettier: latest => 3.0.2
turbo: ^1.10.13 => 1.10.13
npmGlobalPackages:
azurite: 3.23.0
corepack: 0.17.0
gulp-cli: 2.3.0
npm: 9.5.1
samfirm: 0.2.0
Describe the bug
Due to this cypress commit that introduces some breaking API changes in cypress 13.x, I noticed two issues when using cypress-cloud with it:
- Executed cypress tests are always reported as failing whatever their real statuses are.
See sample cypress-cloud output put below:
$ yarn run cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress36
yarn run v1.22.19
$ /home/anlambert/swh/swh-environment/swh-web/node_modules/.bin/cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress36
WARNING Failed to load config file, falling back to the default config. Attempted locations: [
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.js',
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.cjs',
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.mjs'
]
Copyright (C) 2023 Currents Software Inc https://currents.dev
This is free software, and you are welcome to redistribute it under certain
conditions. This program comes with no warranty. Parts of this program are MIT
licensed. Refer to the license for details
https://github.com/currents-dev/cypress-cloud/blob/main/LICENSE.md
Discovered 1 spec files
Tags: false; Group: false; Parallel: true; Batch Size: 3
Connecting to cloud orchestration service...
🎥 Run URL: http://localhost:8080/run/aff6e1ec70f400b22c4deeb5b83332e7
====================================================================================================
Running: cypress/e2e/home.cy.js (1/1)
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 13.1.0 │
│ Browser: Electron 106 (headless) │
│ Node Version: v18.16.0 (/usr/bin/node) │
│ Specs: 1 found (home.cy.js) │
│ Searched: cypress/e2e/home.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: home.cy.js (1 of 1)
Home Page Tests
✓ should have focus on search form after page load
✓ should display positive stats for each category
✓ should display null counters and hide history graphs when storage is empty
✓ should hide counters when data is missing
✓ should redirect to search page when submitting search form
5 passing (6s)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 5 │
│ Passing: 5 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 6 seconds │
│ Spec Ran: home.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ home.cy.js 00:06 5 5 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 00:06 5 5 - - -
Reporting results and artifacts in background...
====================================================================================================
Cloud Run Finished
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Spec Tests Passing Failing Pending Skipped │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ cypress/e2e/home.cy.js 0ms 1 - 1 - - │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✖ 1 of 1 failed 0ms 1 - 1 - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
🏁 Recorded Run: http://localhost:8080/run/aff6e1ec70f400b22c4deeb5b83332e7
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
After reading cypress-cloud code, it seems that issue can be fixed using the following diff:
diff --git a/packages/cypress-cloud/lib/results/results.ts b/packages/cypress-cloud/lib/results/results.ts
index f9380de..635f0b4 100644
--- a/packages/cypress-cloud/lib/results/results.ts
+++ b/packages/cypress-cloud/lib/results/results.ts
@@ -12,11 +12,6 @@ import { ConfigState } from "../state";
const debug = Debug("currents:results");
-export const isSuccessResult = (
- result: CypressResult
-): result is CypressCommandLine.CypressRunResult => {
- return result.status === "finished";
-};
export const getScreenshotsSummary = (
tests: CypressCommandLine.TestResult[] = []
@@ -282,9 +277,6 @@ export function getCypressRunResultForSpec(
spec: string,
cypressResult: CypressResult
): CypressCommandLine.CypressRunResult | undefined {
- if (!isSuccessResult(cypressResult)) {
- return;
- }
const run = cypressResult.runs.find((r) => r.spec.relative === spec);
if (!run) {
- Screenshots fail to be processed.
Once the fix above applied, there is an error when cypress-cloud attempts to process test screenshots, see output below:
$ yarn run cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress38
yarn run v1.22.19
$ /home/anlambert/swh/swh-environment/swh-web/node_modules/.bin/cypress-cloud run --parallel --record --key somekey --cloud-debug true --ci-build-id hello-cypress38
WARNING Failed to load config file, falling back to the default config. Attempted locations: [
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.js',
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.cjs',
'file:///home/anlambert/swh/swh-environment/swh-web/currents.config.mjs'
]
Copyright (C) 2023 Currents Software Inc https://currents.dev
This is free software, and you are welcome to redistribute it under certain
conditions. This program comes with no warranty. Parts of this program are MIT
licensed. Refer to the license for details
https://github.com/currents-dev/cypress-cloud/blob/main/LICENSE.md
Discovered 1 spec files
Tags: false; Group: false; Parallel: true; Batch Size: 3
Connecting to cloud orchestration service...
🎥 Run URL: http://localhost:8080/run/8daacfecbd761c3d663e7e4bd2646eae
====================================================================================================
Running: cypress/e2e/home.cy.js (1/1)
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 13.1.0 │
│ Browser: Electron 106 (headless) │
│ Node Version: v18.16.0 (/usr/bin/node) │
│ Specs: 1 found (home.cy.js) │
│ Searched: cypress/e2e/home.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: home.cy.js (1 of 1)
Home Page Tests
✓ should have focus on search form after page load
✓ should display positive stats for each category
✓ should display null counters and hide history graphs when storage is empty
✓ should hide counters when data is missing
✓ should redirect to search page when submitting search form
5 passing (6s)
(Results)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Tests: 5 │
│ Passing: 5 │
│ Failing: 0 │
│ Pending: 0 │
│ Skipped: 0 │
│ Screenshots: 0 │
│ Video: false │
│ Duration: 6 seconds │
│ Spec Ran: home.cy.js │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ home.cy.js 00:06 5 5 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
✔ All specs passed! 00:06 5 5 - - -
Reporting results and artifacts in background...
ERROR TypeError: Cannot read properties of undefined (reading 'flatMap')
at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:26:21
at Array.flatMap (<anonymous>)
at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:25:19
at Array.flatMap (<anonymous>)
at getScreenshotsSummary (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:24:16)
at getInstanceResultPayload (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/results.ts:62:18)
at getReportResultsTask (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/results/uploadResults.ts:22:13)
at createReportTask (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/reportTask.ts:26:3)
at /home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/runner.ts:55:7
at Array.forEach (<anonymous>)
at runTillDone (/home/anlambert/swh/swh-environment/swh-web/node_modules/lib/runner/runner.ts:54:7)
====================================================================================================
Cloud Run Finished
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Spec Tests Passing Failing Pending Skipped │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ undefined 6.2s 5 5 - - - │
├────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ✔ All specs passed! 6.2s 5 5 - - - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
🏁 Recorded Run: http://localhost:8080/run/8daacfecbd761c3d663e7e4bd2646eae
Done in 13.82s.
The fix for this one is not straightforward to me.
Expected behavior
cypress 13.x should be supported.
Command and Setup
Just run any cypress-cloud session with cypress 13.x and the reported errors will show up.
Full log and debug output
marthayaputra, adelegard, zvickery and mausch
Metadata
Metadata
Assignees
Labels
wontfixThis will not be worked onThis will not be worked on