[Feature]: Model Drivel Application - Health Check Test Coverage Report

[Grant-Archibald-MS]

Is your feature request related to a problem? Please describe.

As build a set of tests it is helpful to be able to share a test coverage metric with stakeholders so that they can continue to evaluate how well the tests provide coverage for the solution.

Specifically looking at health checks that can be used as part of test and monitoring phase of lifecycle

Describe the solution you'd like

The solution should provide a detailed Health Check Test Coverage Report specifically for Power Apps Model Driven Apps. This report will help stakeholders evaluate how well the tests cover the solution by providing specific metrics and examples. The health check will focus on the following types of pages within the application: entity list pages, entity record pages, and custom pages.

The health check will be expressed as a ratio, where the denominator is the total number of possible pages included in the application (entity list, entity record, and custom pages), and the numerator is the total number of pages for which all tests pass. This metric will be crucial during the test and monitoring phase of the lifecycle to ensure comprehensive coverage and reliability of the application.

Denominator: Total Possible Pages

The denominator in the health check metric represents all the possible pages included in the application. This includes:

• Entity List Pages: These are the pages that display lists of records for a specific entity.
• Entity Record Pages: These are the pages that display detailed information for a specific record of an entity.
• Custom Pages: These are the pages that are custom-built to provide specific functionalities or views within the application.

For example, if the application includes 10 entity list pages, 15 entity record pages, and 5 custom pages, the total possible pages would be 30.

Numerator: Pages with All Tests Passing

The numerator in the health check metric represents the total number of pages for which all tests pass. This means that for each page, all the defined tests (such as opening the page, performing actions, and verifying data) have been executed successfully without any failures.

For example, if out of the 30 possible pages, 8 entity list pages, 12 entity record pages, and 4 custom pages have all tests passing, the numerator would be 24.

Example Calculation

Let's consider an example to illustrate the calculation of the health check metric:

Total Possible Pages (Denominator):

Entity List Pages: 10
Entity Record Pages: 15
Custom Pages: 5
Total: 30
Pages with All Tests Passing (Numerator):

Entity List Pages: 8
Entity Record Pages: 12
Custom Pages: 4
Total: 24

The health check metric would be calculated as follows:

Health Check Percentage Pass

= ( Numerator / Denominator ) * 100
= ( 24 / 30 ) * 100
= 80%

Describe alternatives you've considered

No response

Additional context?

No response

[Grant-Archibald-MS]

Example health check to from Copilot Studio Kit sample to consider as feature to generate test summary report from a set of test results using RunTests.ps1.

Key points of the sample:

1. Ability to aggregate test results across a set of different test suites.
2. Aggregate Test / Pass fail and Health metric

3. Generate a Health Check Metric that looks at pages of a Model Driven Application (Entity List, Entity Record and Custom Pages)
4. Calculation workings. Explanation of how health check metric is calculated

5. Graphical summary of pass fail

6. Summary of test result files with test summary

[Grant-Archibald-MS]

Expand the scope of the report to include the following:

1. All possible tests
2. Ability to sort and filter test results

[Grant-Archibald-MS]

This feature is implemented as part of copilotstudiokit/RunTests.ps1 in #560 as a sample.

Future version of the pac cli could include this functionality as a pac cli action.

For example pac test summary --start "datetime" --output summary.html

[Grant-Archibald-MS]

Extend the sample used at Build 2025 so that rather than using PowerShell script that process trx file available as a new command.