Skip to content

Occasional flaky tests with grouped output. #2349

@trulede

Description

@trulede

Description

There is an occasional flaky test result. It seems like the output of task is captured with split lines, that is to say, a partial line is written as one line, and then the rest of that line is written as a subsequent line. This output, when sorted, the causes the goldie check to fail.

For instance, in the following example, the single line "d3\n" seems to be "d3" and "\n". As a result the check fails.

--- FAIL: TestDeps (0.14s)
      executor_test.go:401: Result did not match the golden fixture. Diff is below:
          
          --- Expected
          +++ Actual
          @@ -1,2 +1,2 @@
          -d1
          +
           d11
          @@ -8,3 +8,2 @@
           d23
          -d3
           d31
          @@ -12,2 +11,3 @@
           d33
          +d3d1
           
          
  FAIL
  FAIL	github.com/go-task/task/v3	6.995s

Its possible that the function

func PPSortedLines(t *testing.T, b []byte) []byte {
is not processing the output correctly. It may first need to join lines without a trailing '\n' to the following line, and then apply the sorting algorithm. However, this is not proven, and the output "d3d1\n" does seem odd. In theory, with grouped output, such a combination should not be possible, right? Therefore I wonder if the post processing (PPSortedLines()) causes the issue.

I observe this problem most often with the "deps" test:

version: '3'

Version

v3.44.1

Operating system

Ubuntu

Experiments Enabled

No response

Example Taskfile

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions