Expected performance of ctrl-sim behavior model

[itsbman]

Hello,

Thank you for releasing the code for training the behavior model recently. I'm currently trying to reproduce the performance of ctrl-sim and I was wondering if you can share some training results (such as the validation loss value) as reference to let me know if the training went well. In your paper, you showed the results in table 5 but scanning through your code, it doesn't look like there's any way to reproduce it.

Any guidance would be greatly appreciated. Thank you!

[RLuke22]

Hello,
I trained a CtRL-Sim model for about 150k steps using the public repository code shortly before releasing it publicly. I've attached a screenshot of the training/validation curves. Please ignore the fragmented nature of the training curves: my run is segmented into 3-hour chunks due to compute restrictions.

I've also verified that this model (after 150k steps) drives reasonably well in simulation. I will release the code for simulating CtRL-Sim agents (and computing metrics) within the next week, as well as a CtRL-Sim checkpoint trained to 1M steps.

Hope this helps!
Best,
Luke

[itsbman]

Thank you for the quick response! Since this is trained with waymo v1.1.0, do you expect any change if I train it with v1.2.1?

[RLuke22]

My understanding is that the main differences between v1.1.0 and v1.2.1 are relatively minor for CtRL-Sim training and mostly relate to minor updates in the roadgraph (eg, see here: https://github.com/waymo-research/waymo-open-dataset/tree/5f8a1cd42491210e7de629b6f8fc09b65e0cbe99?tab=readme-ov-file#march-2023-update). My guess is that any impact would be marginal, though I haven’t run experiments to confirm this.

[jiangchaokang]

I'm trying to train Ctrl-Sim using the data preprocessing script you provided, but the loss function keeps failing to converge. Have you encountered a similar problem? What could be causing this?

Should I use self.predict_action(output[:, 1]) or self.predict_action(output[:, 2]) here? I'm a bit confused:

scenario-dreamer/nn_modules/ctrl_sim.py

Line 320 in a1cc6d0

action_preds = self.predict_action(output[:, 1])

Below is a loss trend graph of my Ctrl-Sim model, roughly like this (I can't upload screenshots, sorry, I'm using an ASCII image instead). I found that it completely fails to converge. Do you know the possible reasons?

Chart 1: grad_2.0_norm_total

5.5e+7|                                      *
      |                                    * 
5.0e+7|                                  *   *
      |                                 *
4.5e+7|                               *
      |                             *
4.0e+7|                           *
      |                         *
3.5e+7|                       *
      |                     *
3.0e+7|                   *
      |                 *
2.5e+7|               *
      |             *
2.0e+7|           *
      |         *
1.5e+7|       *
      |      *
1.0e+7|    *
      |   *
5.0e+6|  *
      | *
     0|*___
      +-------|-------|-------|-------|-------|-------|-------|-------|
      0      10k     20k     30k     40k     50k     60k     70k     80k

Chart 2: loss

3.75  |              * * *   * *     * *   *
      |             *     * *   * * *   * * *
3.70  |           *                         
      |          *
3.65  |        *
      |       *
3.60  |      *
      |     *
3.55  |    *
      |   *
3.50  |  *
      | *
3.45  |*
      +-------|-------|-------|-------|-------|-------|-------|-------|
      0      10k     20k     30k     40k     50k     60k     70k     80k

[RLuke22]

Hi Chaokang,

This is indeed not expected behaviour. Have you tried training using the uploaded CtRL-Sim preprocessed data? (https://drive.google.com/drive/folders/13DSHf2UhrvguD7i7iYL5SfSDhgLcW_ja)

Also, what is your training setup? Are you training with 4 GPUs?

It would be nice to isolate whether this is a data problem or a training setup/code problem.

I will launch another training run tonight to make sure everything works on my end, as my more recent changes may have broken something.

Best,
Luke

[RLuke22]

self.predict_action(output[:, 1]) is correct there. The second dimension is length 3, with tokens ordered as [s_t, G_t, a_t]. The output[:,1] extracts the G_t tokens to predict a distribution over actions a_t, as in Figure 2 of CtRL-Sim here: https://arxiv.org/pdf/2403.19918.

[RLuke22]

Hi Chaokang,

I trained a CtRL-Sim model on 4 GPUs last night using the public repo and the preprocessed data that I uploaded to Google Drive. The training seems to work as expected.

Please let me know if you have issues reproducing this on your end.

[jiangchaokang]

Hi Chaokang,

I trained a CtRL-Sim model on 4 GPUs last night using the public repo and the preprocessed data that I uploaded to Google Drive. The training seems to work as expected.

Please let me know if you have issues reproducing this on your end.

Thank you very much for your reply and the valuable information you provided.

I will pull the latest CtRL-Sim training code from the Scenario-Dreamer repository you shared and use the processed data from your Google Drive (https://drive.google.com/drive/folders/13DSHf2UhrvguD7i7iYL5SfSDhgLcW_ja) to retry the training process.

I also look forward to your future contributions, particularly regarding:

• Evaluating planners in Scenario-Dreamer environments
• Training Scenario-Dreamer compatible agents in GPUDrive

These resources are invaluable to the research community. Thank you again for your excellent work!