Should we perform parallel execution for parameter sweeps in mesa-frames

[suryanshgargbpgc]

Hi @EwoutH @adamamer20, I was wondering if we can implement parallel execution for parameter sweep in mesa-frames.

Since mesa frames is good for optimizing single model run using vectorixed operations, however as far as I understood in the mesa frames codebase there is no proper class or function which runs multiple model instances in parallel. currently all the load for implementing this task is on the user which they implement by writing their own code.

this feature is crucial for models involving randomness, since it can deliver different output even when we pass the same parameters, so the user might need to run this model multiple times to get a clear picture, ,moreover if there are many parameter combinations this would result in too many model runs.

I think the parallel execution should save a lot of time.

I want to add this feature as upcoming implementation for my GSOC proposal, can you guys provide me some insight on this
thank you

[EwoutH]

So basically batch_run from Mesa for Mesa-frames?

One thing we should watch out for is that we don’t build a second EMAworkbench. However, Mesa-frames support in the EMAworkbench could be super interesting (CC @quaquel).

[quaquel]

The tricky thing is that you get 2 levels of parallelization: within-run parallelization and across-run parallelization. To do this properly is complicated and will be very model-specific. That is, you would need to design the code such that you can control both the within-run and the across-run parallelization.

[adamamer20]

I like the idea! The only concern is that it could actually slow things down. You’d need plenty of memory because datasets usually become large quickly, and running vectorized operations simultaneously on CPU/GPU could bottleneck performance. However, for smaller models, it might still be worthwhile.

[suryanshgargbpgc]

Hi @adamamer20 @quaquel @EwoutH, I believe there should be a way of implementing this while I can address your concerns, I think we might not need to build entire EMAworkbench but the implementation should only be focused on parallel parameter sweeps and we might not need to duplicate the advanced features.

The tricky thing is that you get 2 levels of parallelization: within-run parallelization and across-run parallelization.

mesa-frames already handles within run parallelization and we can define a separate class for across- run and let the user control both independently, depending on the situation. I believe this approach should work.

You’d need plenty of memory because datasets usually become large quickly, and running vectorized operations simultaneously on CPU/GPU could bottleneck performance. However, for smaller models, it might still be worthwhile.

I think we should test this functionality and its implementation and try to find a sweet spot between no. of model runs and the memory usage.

I am excited about this functionality, this is something we must test out to see, if I am not wrong.

also, I apologize for the late response, I was not well for a past few days.