-
Notifications
You must be signed in to change notification settings - Fork 39
Adding some SMAP Harmony examples #399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
fixed markdown cells that I couldn't due to wandering Jupyter notebook cell bug.
edited output format explanation in L4 section
added reference to SMAP examples in the intro to notebook.
@jules32 Would you mind giving this a review? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks really good @mikala-nsidc! I was able to run it all locally except for the s3 direct streaming, which was expected. Your explanations of all of the concepts are just at the right level so that people understand what's going on without getting bogged down in the weeds.
I have one question - I had to restart the notebook partway through, and when I submitted the request the second time I was a bit surprised to see Harmony processing the whole request again. Does it do any caching on s3 so it doesn't have to reprocess identical requests?
I don't think it's necessary for this notebook, but it would be interesting to demonstrate how to submit a request that puts the processed data in a user's own S3 bucket, does that exist anywhere? If so, it might be worth linking to?
@ateucher Those are good questions! You can access a previously processed request using the Harmony job id - it's staged in a Harmony S3 bucket for 30 days. But yes, if you run the request again from the notebook, you are making a new request and it will create a new Harmony job. |
Thanks Mikala! Yes, I think it's good to merge as-is, I'll do that now, and make an issue based on your comment :) |
Added some SMAP Harmony examples to the notebook and renamed the notebook to be more general.
This won't need to be merged until after October 21, when we publicly release SMAP data sets and Harmony services.