Skip to content

Passing arguments betwen modules #4

@simeonem

Description

@simeonem

I have inputs arguments that needs to be passed through 3 identical modules. The only difference between the modules it the job iterations/labels. Since the limit of jobs submission on Cranium is 3000 and we want to run 9000 jobs per experiment, each module runs 3000 jobs, with labels 1-3000, 3001-6000 and 6001-9000 for each module. I want the module 1-3000 to pass its input arguments (i.e., specs, R script, dataset to process, workspace directory,...) as inputs arguments for the 3001-6000 module, and so on.
Ultimately, I want to pass these specs across different experiments.
The final workflow will have a unique module for initialization, where we specify: i) the R script to use, ii) the dataset to process, iii) the arguments as specs for each experiment, iv) the workspace directory, v) the jobs labels to run, and vi) flags for the shell script. The only module-specific parameter will be the experiment label.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions