Skip to content

Fix sequence size and compute_validation_tests#264

Open
isaacmg wants to merge 25 commits into
masterrfrom
fixes_to_loss_tests
Open

Fix sequence size and compute_validation_tests#264
isaacmg wants to merge 25 commits into
masterrfrom
fixes_to_loss_tests

Conversation

@isaacmg
Copy link
Copy Markdown
Collaborator

@isaacmg isaacmg commented Feb 3, 2021

The goal of this PR is to do the following

  • Correct a bug where forecast_length was not being passed to `simple_dec
  • Fix bugs related to the unscaled_test_loss
  • Add unit tests to validate compute_validation logic explicitly.

@isaacmg isaacmg changed the title fix seuqnece size Fix sequence size and compute_validation_tests Feb 3, 2021
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 3, 2021

Codecov Report

❌ Patch coverage is 84.37500% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.70%. Comparing base (7343cfb) to head (f6d48be).
⚠️ Report is 1625 commits behind head on masterr.

Files with missing lines Patch % Lines
flood_forecast/pytorch_training.py 61.53% 5 Missing ⚠️
flood_forecast/basic/linear_regression.py 50.00% 2 Missing ⚠️
tests/validation_tests.py 95.65% 2 Missing ⚠️
flood_forecast/custom/dilate_loss.py 0.00% 1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (7343cfb) and HEAD (f6d48be). Click for more details.

HEAD has 5 uploads less than BASE
Flag BASE (7343cfb) HEAD (f6d48be)
python 18 13
Additional details and impacted files
@@             Coverage Diff              @@
##           masterr     #264       +/-   ##
============================================
- Coverage    71.06%   60.70%   -10.37%     
============================================
  Files           56       50        -6     
  Lines         3543     3275      -268     
============================================
- Hits          2518     1988      -530     
- Misses        1025     1287      +262     
Flag Coverage Δ
python 60.70% <84.37%> (-10.37%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@isaacmg
Copy link
Copy Markdown
Collaborator Author

isaacmg commented Feb 3, 2021

This #265 #253

@isaacmg
Copy link
Copy Markdown
Collaborator Author

isaacmg commented Feb 4, 2021

Update there is still an ongoing problem that I can't figure out where in the cases of unscaled_test_loss MSE seems to be double DilateLoss in pretty much every training session. This is try in every logging for some reason. This isn't the case for the scaled_test. At the moment I believe the scaled_test_loss is correct. Additionally when un-scaling the scaled_test_loss it often doesn't equal the scaled_test_loss.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Edge case where forecast_len>forecast_history causes error Double check test_loss

1 participant