add --export option on esgpull update

[alaniwi]

Adds a --export option on esgpull update in order to extract lists of files that would be added to the download queue (before the point where it asks the question about whether to add them, or exists if there are no new files)

This gives the user an opportunity to properly inspect what URLs the query is matching, before committing to add the files to the queue.

Two lists of dictionaries are written, in JSON format (after prompting for the filenames, with reasonable defaults):

• all files (before filtering to remove existing SHAs)
• new files that will be added

(PR also simplifies a loop with a list comprehension, not directly related to the new option.)

[svenrdz]

FYI the new_files variable name is misleading, as it corresponds to files that are new, relative to the specific query they are added to, but those same files might already exist in the database, maybe even already downloaded from another query. And in this case they would still appear in the new_files.json with status New, which might not be helpful at all.

If your intention is to know which files are new relative to the database, you can use this snippet to filter the list:

            if export:
                hash = qf.query.name.strip("<>")
                files_not_in_db = [file for file in new_files if file not in esg.db]
                for descrip, file_list, dfl_fn in [("all files", qf.files, f"{hash}_all_files.json"),
                                                   ("new files", files_not_in_db, f"{hash}_new_files.json")]:
                    fn = esg.ui.prompt(f"Filename to export {descrip} list:", dfl_fn)
                    with open(fn, "w") as fout:
                        json.dump([file.asdict() for file in file_list],
                                  fout,
                                  indent=4)
                    print(f"{len(file_list)} files - written to {fn}")

I'm not sure about whether the all_files.json is useful, but I don't have a strong opinion against having it.

In any case, could you run ruff format on the update.py file ? I'm fine without, it's going to be formatted next time I commit in this file anyway, but that would attribute the changes to me instead.

I can merge this as it is now if you don't mind my comments, I'll let you decide and tell me so I can click the button.