Skip to content

BenchmarkRunner cannot Run(Assembly) if the filter eliminates all benchmark cases in any type #2860

@mstefarov

Description

@mstefarov

I am using BenchmarkRunner.Run(Assembly, config) with a custom filter. I wanted to run all benchmarks in assembly except a few filtered-out cases. Unfortunately I am finding that this does not work if my filter happens to eliminate all benchmark cases within any one type. When this happens, a validation error prevents running the benchmarks:

// Validating benchmarks:
// * No [Benchmark] attribute found on 'MyTypeWithFilteredOutCases' benchmark case.

I found a workaround: discover types-with-benchmarks manually, use BenchmarkConverter.TypeToBenchmarks to generate runInfos, and skip runInfos where config's filter removes all cases.

That works, but it seems like BenchmarkRunner would be better with this behavior by default.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions