Skip to content

nan raises pdb debugger with LLFF dataset #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
samhodge-aiml opened this issue Sep 6, 2023 · 2 comments
Open

nan raises pdb debugger with LLFF dataset #23

samhodge-aiml opened this issue Sep 6, 2023 · 2 comments

Comments

@samhodge-aiml
Copy link

see

[Epoch 3295] it=240500, loss=5.24928904, time=10.5139
  3295 ep: Train: ATE: 0.574 RPE_r: 27.093
  3295 ep: Train: PSNR: 17.051
> /media/sam/aimlwork/github/nope-nerf/model/losses.py(207)forward()
-> 'loss': loss,
(Pdb) exit
Traceback (most recent call last):
  File "/media/sam/aimlwork/github/nope-nerf/train.py", line 364, in <module>
    train(cfg=cfg)
  File "/media/sam/aimlwork/github/nope-nerf/train.py", line 210, in train
    loss_dict = trainer.train_step(batch, it, epoch_it, scheduling_start, render_path)
  File "/media/sam/aimlwork/github/nope-nerf/model/training.py", line 87, in train_step
    loss_dict = self.compute_loss(data, it=it, epoch=epoch, scheduling_start=scheduling_start, out_render_path=render_path)
  File "/media/sam/aimlwork/github/nope-nerf/model/training.py", line 371, in compute_loss
    loss_dict = self.loss(rendered_rgb, rgb_gt, rendered_depth, gt_depth, **kwargs)
  File "/media/sam/aimlwork/github/nope-nerf/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/media/sam/aimlwork/github/nope-nerf/model/losses.py", line 207, in forward
    'loss': loss,
  File "/media/sam/aimlwork/github/nope-nerf/model/losses.py", line 207, in forward
    'loss': loss,
  File "/media/sam/aimlwork/github/nope-nerf/env/lib/python3.10/bdb.py", line 90, in trace_dispatch
    return self.dispatch_line(frame)
  File "/media/sam/aimlwork/github/nope-nerf/env/lib/python3.10/bdb.py", line 115, in dispatch_line
    if self.quitting: raise BdbQuit

@samhodge-aiml
Copy link
Author

Restarting seems fine.

@samhodge-aiml samhodge-aiml changed the title nan raises gdb debugger with LLFF dataset nan raises pdb debugger with LLFF dataset Sep 6, 2023
@bianwenjing
Copy link
Collaborator

Hi, please let me know if this issue still exists

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants