docs: `task` requirements 2

[casperdcl]

p2

• task feedback 2 #457
  • docs: misc updates #450
    • show examples for all the task resource arguments
    • Show/explain CI restart workflow lifecycle & environment propagation
    • mention future plans (distributed/multi-instance, security)
• banner/logo: TPI: narrower banner static#8
• simple flowcharts/diagrams docs: iteration 3 #492
• post-merge feedback from docs & readme: what/why/how #433
• docs: iteration 4 #513
• end-to-end tutorials/example repo(s) - e.g. getting credentials, using cloud = "{aws,az,gcp}", installing deps & running scripts
  • see e.g. https://github.com/kubeflow/examples/blob/master/named_entity_recognition/documentation/step-1-setup.md?
• improve formatting of table at docs/guides/generic-machine-types docs: iter 4 #532
• consider collapsing machine-type, machine-image and cloud-region sections in <details>
• add or link to ML-related examples
• what other common use-cases should be mentioned (e.g. install awscli & upload artefacts to a bucket)?
• maybe link to terraform help (https://learn.hashicorp.com/tutorials/terraform/aws-build?)
• README vs registry (à la dvc.org#2443)
  • readme says "terminate compute instances upon job completion/failure." Should say this in "Run Task" & also in registry docs: iter 4 #532
  • misc feedback readme intro suggestions #558

p1

• docs: task requirements 1 #397
• basic docs improvements #415
• what cloud permissions are required for docs/guides/getting-started#launching-tasks?
  • Provide reference roles and permissions for every cloud vendor #75

[DavidGOrtega]

what permissions are required (e.g. port 22 open)?

This is needed for runner. task is not enforced right now to use ssh, we use the cloud storage

[DavidGOrtega]

This are my pics @casperdcl

As far as I see you already have some

README

Automatically destroy unused cloud resources (never forget to turn your GPU off again)

This is not true at current status unless you use runner

TERRAFORM DOCS

• Why generic machine types are in Development section?
• Missing specific cloud permissions
• Missing complete spot instances survival lifecycle (which is the best effort in tgi)
• CI restart workflow lifecicle and explain also the environment propagation

Create a directory named shared to store input data and output artefacts.

I do not fully undestand this example. Why should the user modify the project structure? If someone is telling me to modify my project to use a tool I would say... F*#$@ U

The script argument can take any string, including a heredoc or the contents of a file returned by the file function.

Is better if we setup an example instead (terraform docs are not great).
Also is very nice the idea of using python scripting as an example

Status and events don't produce a stable output between cloud providers and are intended for human consumption only.

Right now the status is normalised

[0x2b3bfa0]

This is not true at current status unless you use runner^{[citation needed]}

As far as I can tell, iterative_task terminates all the instances as soon as the script finishes running. 🤔

[0x2b3bfa0]

Add to the list #258 (comment), “add a documentation footnote that after the task completes its instance is auto destroyed”

[casperdcl]

Added an item to the p1 checklist here #397 instead :)

[DavidGOrtega]

@casperdcl can you please add these two?

• Complete spot instances survival lifecycle (which is the best effort in tpi)
• CI restart workflow lifecycle and explain also the environment propagation