improve data sync features

[casperdcl]

Collection of data handling issues

new features

• handle task workdir file deletions #360
• task bucket usage vs "directory" within a bucket #299
  • prerequisite for migrating runner to task?
• Data sync: allow more advanced workdir and output specs #590
• Allow tasks to destroy themselves #289

improving existing features

• task: optionally skip downloading artefacts on destroy #303
• task: skip existing files on download #361
• task: optionally download data on refresh #301
• task: optionally skip data sync on error #697

user experience

• task rclone Config file not found #373

[0x2b3bfa0]

How does #289 relate to data synchronization? 🤔

[casperdcl]

Tasks would presumably have to sync with the storage bucket during the self-destruction process.

[0x2b3bfa0]

Tasks would presumably have to sync with the storage bucket during the self-destruction process.

Data synchronization is an event–driven process independent from (self–)destruction. Last edge cases were fixed with #377.