DSC config does discovery on every resource instance

[ThomasNieto]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest version
• Search the existing issues.

Summary

dsc config does discovery for each resource instance. This causes dsc runs to be unnecessarily long. The appx-discover.ps1 script takes on average ~4 seconds.

Why does DSC not do discovery for all resource types at the beginning of a run instead of foreach resource instance?

Steps to reproduce

1. Create a PowerShell resource instances
2. dsc config get

Expected behavior

Removed appx and speed improved by ~35 seconds for 7 PowerShell resources.

metadata:
  Microsoft.DSC:
    version: 3.1.0
    operation: get
    executionType: actual
    startDatetime: 2025-06-11T14:03:52.458297900-05:00
    endDatetime: 2025-06-11T14:04:13.214544700-05:00
    duration: PT20.7562468S
    securityContext: restricted

Config is ran without extra time being devoted to multiple resource discoveries.

Actual behavior

metadata:
  Microsoft.DSC:
    version: 3.1.0
    operation: get
    executionType: actual
    startDatetime: 2025-06-11T14:02:26.864581200-05:00
    endDatetime: 2025-06-11T14:03:22.309614-05:00
    duration: PT55.4450328S
    securityContext: restricted

2025-06-11T19:03:16.270404Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\assertion.dsc.resource.json
2025-06-11T19:03:16.284597Z TRACE dsc_lib::discovery::command_discovery: 284: Resource 'Microsoft.DSC/Assertion' found
2025-06-11T19:03:16.285374Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\echo.dsc.resource.json
2025-06-11T19:03:16.297578Z TRACE dsc_lib::discovery::command_discovery: 284: Resource 'Microsoft.DSC.Debug/Echo' found
2025-06-11T19:03:16.298195Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\group.dsc.resource.json
2025-06-11T19:03:16.305508Z TRACE dsc_lib::discovery::command_discovery: 284: Resource 'Microsoft.DSC/Group' found
2025-06-11T19:03:16.306295Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\include.dsc.resource.json
2025-06-11T19:03:16.315133Z TRACE dsc_lib::discovery::command_discovery: 284: Resource 'Microsoft.DSC/Include' found
2025-06-11T19:03:16.315906Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\osinfo.dsc.resource.json
2025-06-11T19:03:16.324911Z TRACE dsc_lib::discovery::command_discovery: 284: Resource 'Microsoft/OSInfo' found
2025-06-11T19:03:16.325860Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\powershell.dsc.resource.json
2025-06-11T19:03:16.328407Z TRACE dsc_lib::discovery::command_discovery: 281: Resource adapter 'Microsoft.DSC/PowerShell' found
2025-06-11T19:03:16.329218Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\reboot_pending.dsc.resource.json
2025-06-11T19:03:16.330882Z TRACE dsc_lib::discovery::command_discovery: 284: Resource 'Microsoft.Windows/RebootPending' found
2025-06-11T19:03:16.331393Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\registry.dsc.resource.json
2025-06-11T19:03:16.346360Z TRACE dsc_lib::discovery::command_discovery: 284: Resource 'Microsoft.Windows/Registry' found
2025-06-11T19:03:16.346899Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\RunCommandOnSet.dsc.resource.json
2025-06-11T19:03:16.351945Z TRACE dsc_lib::discovery::command_discovery: 284: Resource 'Microsoft.DSC.Transitional/RunCommandOnSet' found
2025-06-11T19:03:16.353172Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\windowspowershell.dsc.resource.json
2025-06-11T19:03:16.358613Z TRACE dsc_lib::discovery::command_discovery: 281: Resource adapter 'Microsoft.Windows/WindowsPowerShell' found
2025-06-11T19:03:16.359387Z TRACE dsc_lib::discovery::command_discovery: 242: Found resource manifest: C:\dsc\wmi.dsc.resource.json
2025-06-11T19:03:16.361539Z TRACE dsc_lib::discovery::command_discovery: 281: Resource adapter 'Microsoft.Windows/WMI' found
2025-06-11T19:03:16.362928Z DEBUG dsc_lib::discovery::command_discovery: 305: Calling extension 'Microsoft.Windows.Appx/Discover' to discover resources
2025-06-11T19:03:16.364114Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-11T19:03:20.284642Z DEBUG dsc_lib::dscresources::command_resource: 660: Process 'powershell' id 19236 exited with code 0
2025-06-11T19:03:20.288204Z  INFO dsc_lib::extensions::dscextension: 99: No results returned for discovery extension 'Microsoft.Windows.Appx/Discover'
2025-06-11T19:03:20.289070Z DEBUG dsc_lib::discovery::command_discovery: 307: Extension 'Microsoft.Windows.Appx/Discover' found 0 resources
2025-06-11T19:03:20.290259Z DEBUG dsc_lib::discovery::command_discovery: 500: Found 0 matching non-adapter-based resources

Error details

Environment data

7.6-preview6

Version

3.1.-rc1

Visuals

No response

[ThomasNieto]

@SteveL-MSFT is the slowness penalty for appx resource discovery able to be addressed before 3.1 goes out or would that be after?

[SteveL-MSFT]

@SteveL-MSFT is the slowness penalty for appx resource discovery able to be addressed before 3.1 goes out or would that be after?

It's likely not a trivial change, so after 3.1

[ThomasNieto]

@SteveL-MSFT I think there is two potential enhancements here.

1. Implementing discovery cache inside appx-discover.ps1 so if it does run it only need to do discovery once unless a new/updated appx package is installed
2. Update dsc engine to not discover on every resource instance

[SteveL-MSFT]

@ThomasNieto the engine is supposed to already be caching, however, it also has to rely on adapters/extensions to perform their own caching since their discovery mechanism is unknown to dsc engine. So in this case, it seems the best case is to have caching in appx-discover.ps1. Simple case might be to just check if the WindowsApps folder modified date has changed otherwise use the cached version.

[Gijsreyn]

@SteveL-MSFT, the problem with the WindowsApps folder is that it is protected. I can't imagine users setting the securityContext to always run elevated, meaning it isn't that of a simple case. There are registry keys to search for, which might be cheaper, but still...

[SteveL-MSFT]

@ThomasNieto can you provide a simple configuration example showing discovery happening multiple times? The only case I can think of where this would happen is when using a Group resource which is just a nested dsc instance. From using the examples, discovery only happens at the start which is what I expected. Just want to make sure there isn't a fundamental issue in the engine where discovery happens multiple times. The appx caching I can work on.

Also, before I add caching, can you give me some data from your system since it's not that slow on mine:

get-appxpackage | measure-object
# from within the dsc home folder
measure-command { .\appx-discover.ps1 }

[ThomasNieto]

@SteveL-MSFT here is the config yaml I used.

$schema: https://aka.ms/dsc/schemas/v3/bundled/config/document.json
variables: 
  regKeyNuance: HKLM\Software\Wow6432Node\Nuance\Dragon Medical One 2022\InstallProperties
  regKeySoftware: HKLM\Software\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\083B6735F7E34244D991001CA1F4E222\InstallProperties
resources:
- name: dmo logLevel
  type: Microsoft.Windows/Registry
  properties:
    keyPath: "[variables('regKeyNuance')]"
    valueName: logLevel
- name: dmo installDir
  type: Microsoft.Windows/Registry
  properties:
    keyPath: "[variables('regKeyNuance')]"
    valueName: installDir
- name: dmo powerMicMobile
  type: Microsoft.Windows/Registry
  properties:
    keyPath: "[variables('regKeyNuance')]"
    valueName: powerMicMobile
- name: dmo organizationToken
  type: Microsoft.Windows/Registry
  properties:
    keyPath: "[variables('regKeyNuance')]"
    valueName: organizationToken
- name: dmo supportedLanguages
  type: Microsoft.Windows/Registry
  properties:
    keyPath: "[variables('regKeyNuance')]"
    valueName: supportedLanguages
- name: dmo supportedTopics
  type: Microsoft.Windows/Registry
  properties:
    keyPath: "[variables('regKeyNuance')]"
    valueName: supportedTopics
- name: dmo localPackage
  type: Microsoft.Windows/Registry
  properties:
    keyPath: "[variables('regKeySoftware')]"
    valueName: localPackage
- name: dmo flavor
  type: XmlDsc/XmlElementValue
  properties:
    Path: C:\temp\SoD.exe.config
    XPath: /configuration/applicationSettings/SoD.Settings/setting[@name='Flavor']/value/text()
- name: dmo domain
  type: XmlDsc/XmlElementValue
  properties:
    Path: C:\temp\SoD.exe.config
    XPath: /configuration/applicationSettings/SoD.Settings/setting[@name='Domain']/value/text()
- name: dmo authentication
  type: XmlDsc/XmlElementValue
  properties:
    Path: C:\temp\SoD.exe.config
    XPath: /configuration/applicationSettings/SoD.Settings/setting[@name='Authentication']/value/text()
- name: dmo clientID
  type: XmlDsc/XmlElementValue
  properties:
    Path: C:\temp\SoD.exe.config
    XPath: /configuration/applicationSettings/SoD.Settings/setting[@name='ClientID']/value/text()
- name: dmo audience
  type: XmlDsc/XmlElementValue
  properties:
    Path: C:\temp\SoD.exe.config
    XPath: /configuration/applicationSettings/SoD.Settings/setting[@name='Audience']/value/text()
- name: dmo redirectUri
  type: XmlDsc/XmlElementValue
  properties:
    Path: C:\temp\SoD.exe.config
    XPath: /configuration/applicationSettings/SoD.Settings/setting[@name='RedirectUri']/value/text()
- name: dmo systemGUID
  type: XmlDsc/XmlElementValue
  properties:
    Path: C:\temp\SoD.exe.config
    XPath: /configuration/applicationSettings/SoD.Settings/setting[@name='SystemGUID']/value/text()

Here is the excerpt from the trace log where it says its executing the appx discovery.

3 lines of when it called discovery and on average how long it took.

2025-06-26T20:33:52.755954Z DEBUG dsc_lib::discovery::command_discovery: 305: Calling extension 'Microsoft.Windows.Appx/Discover' to discover resources
2025-06-26T20:33:52.756502Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo", "-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-26T20:33:56.683335Z DEBUG dsc_lib::dscresources::command_resource: 660: Process 'powershell' id 51336 exited with code 0

The number of times appx discovery was ran for that configuration file.

2025-06-26T20:33:44.104990Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo",
"-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-26T20:33:52.756502Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo",
"-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-26T20:33:59.095027Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo",
"-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-26T20:34:05.293951Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo",
"-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-26T20:34:11.967733Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo",
"-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-26T20:34:18.612973Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo",
"-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-26T20:34:25.593589Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo",
"-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])
2025-06-26T20:34:32.208066Z DEBUG dsc_lib::dscresources::command_resource: 699: Invoking command 'powershell' with args Some(["-NoLogo",
"-NonInteractive","-ExecutionPolicy", "Bypass", "-NoProfile", "-Command", "./appx-discover.ps1"])

PS C:\dsc> get-appxpackage | measure-object

Count             : 146
Average           :
Sum               :
Maximum           :
Minimum           :
StandardDeviation :
Property          :

PS C:\dsc> measure-command { .\appx-discover.ps1 }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 1
Milliseconds      : 371
Ticks             : 13713640
TotalDays         : 1.58722685185185E-05
TotalHours        : 0.000380934444444444
TotalMinutes      : 0.0228560666666667
TotalSeconds      : 1.371364
TotalMilliseconds : 1371.364

[SteveL-MSFT]

Thanks. This helps.

Ok, so I think I know what the problem is. Basically, you are using PS-based DSC resources without specifying the adapter. This results in the engine generating a nested config that specifies the adapter and runs that. When that nested config runs, it will itself perform discovery again. So I think what needs to happen is a way to pass a parameter to the generated config to skip discovery since it already happened initially.

[SteveL-MSFT]

I looked at how I could fix this and it's not possible without some hacks. Basically, a nested configuration has no awareness of the parent configuration and at the point of invocation of the resource, it has no context of it's parent configuration. This means there's no way to pass the internal cache of discovered resources to a nested config.

Using implicit adapted resources already incurs the cost of a separate PowerShell process per invocation. For perf reasons, it would be better to wrap all of your PS resources invocations within an explicit adapter resource call (whether WinPS or PS7).

I can experiment with appx-discover.ps1 to have it check if well-known Appx folders have been modified, but I suspect the time saved will be small compared to creating PowerShell process and initializing the runspace.

Actually, I think caching won't give much benefit since the cache needs to be stored on disk and read and processed as a structured file. Instead, I'll see if there's a big perf improvement if I write a small app to call the Appx APIs instead of using PowerShell.