My Testrail Results are being sent as Failed and Passed at the same time

[GregJacobs82]

NOTE: I did just recently update to latest cypress-testrail v2.8.1

Notice the identical timestamps in the Testrail report. Any thoughts as to why it is sending both a passing and failing status? It SHOULD just be sending the failed status (in this specific case).

---

I think it might be due to my test scenario is a Scenario Outline: with more than 1 example to run. In this test case scenario outline, the First test example for USA fails, but the second test example for CANADA passes. If any example fails in the scenario outline, it should fail the test:

────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  PublicUser/Homepage.feature                                                     (2 of 2)
  Estimated: 1 minute, 1 second


  Public User > Homepage
    (Attempt 1 of 2) C305363: My test scenario outline [USA] (example #1)
    1) C305363: My test scenario outline [USA] (example #1)
    ✓ C305363: My test scenario outline [Canada] (example #2) (6040ms)


  1 passing (1m)
  1 failing

  1) Public User > Homepage
       C305363: My test scenario outline [USA] (example #1):
     AssertionError: Timed out retrying after 16000ms: expected 'https://xxxxx.com' to include 'xxxx.expected'
      at Context.eval (cypress/integration/cucumber/PublicUser/common/common.spec.js:392:21)
      at Context.resolveAndRunStepDefinition (node_modules/cypress-cucumber-preprocessor/lib/resolveStepDefinition.js:193:0)
      at Context.eval (node_modules/cypress-cucumber-preprocessor/lib/createTestFromScenario.js:27:0)



  TestRail >> Sending case results to run R651: C305363,C305363 
  Results sent to TestRail for: C305363,C305363 

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        2                                                                                │
  │ Passing:      1                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  2                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     1 minute, 11 seconds                                                             │
  │ Estimated:    1 minute, 1 second                                                               │
  │ Spec Ran:     PublicUser/Homepage.feature                                                      │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

Any help is greatly appreciated. Thank you!

[boxblinkracer]

Hi

thank you for this
ok wow, so i'm not aware of that, that those tests come in multiple times
usually a cypress result only has that test once
according to the debug line from the integration i can confirm its sent both times

so i could try to do a distinct on the found test cases...but some people have indeed multiple tests linked to the same testrail id i think

let me reproduce it, what cypress version are you using?

[GregJacobs82]

let me reproduce it, what cypress version are you using?

Cypress version 13.3.0
(To verify, you can see version in the screenshot provided in the testrail result)