Changing clk_sys divisor causes long delay

[zshivers]

Changing the system clock divisor, which the datasheet says can be changed on-the-fly, causes a ~350 ms delay for some divisors.

#include "hardware/clocks.h"
#include "hardware/gpio.h"
#include "pico/stdlib.h"

int kDebugPin = 4;

int main() {
  gpio_set_function(kDebugPin, GPIO_FUNC_SIO);
  gpio_set_dir(kDebugPin, GPIO_OUT);
  gpio_put(kDebugPin, false);

  // Switch clk_sys to source from the USB PLL.
  clock_configure(
      clk_sys,                                           // Clock to configure.
      CLOCKS_CLK_SYS_CTRL_SRC_VALUE_CLKSRC_CLK_SYS_AUX,  // The main clock
                                                         // source.
      CLOCKS_CLK_SYS_CTRL_AUXSRC_VALUE_CLKSRC_PLL_USB,   // The auxilary clock
                                                         // source.
      48 * MHZ,  // Frequency of the input clock source.
      48 * MHZ   // Requested clock frequency.
  );

  clock_hw_t *clk_sys_hw = &clocks_hw->clk[clk_sys];

  while (true) {
    gpio_put(kDebugPin, true);
    // Set clk_sys to 1 MHz by setting the divider to 48.
    clk_sys_hw->div = 48 << CLOCKS_CLK_GPOUT0_DIV_INT_LSB;

    gpio_put(kDebugPin, false);
    // Set clk_sys to 48 MHz by setting the divider to 1.
    clk_sys_hw->div = 1 << CLOCKS_CLK_GPOUT0_DIV_INT_LSB;
  }
  return 0;
}

This results in the following waveform on the debug pin:

Changing the divisor to 7 with clk_sys_hw->div = 7 << CLOCKS_CLK_GPOUT0_DIV_INT_LSB; results in no long delay:

Any divisor 1 through 7 results in no delay; anything >= 8 results in the 350 ms delay.

What could be causing this delay?

[lurch]

Is this on an RP2040 or an RP2350?

[zshivers]

RP2040