Release Plan

[yjxiong]

To improve the ease of use for the community, let's plan on how we gonna organize the released codebase.

• Preliminary Tools
  • Frame/optical flow extraction: already released in our dense_flow. Need to add instructions
  • Caffe: released with full usage guides
  • Scripts for building initialization models for the temporal stream
• Training TSN
  • TSN training model spec in Caffe style based on BN-Inception
    • segment number: 3
    • aggregation func: avg
    • RGB
    • Flow
    • Warped flow
  • Initialization model weights based on BN-Inception
    • RGB
    • Flow
    • Warped flow
• Testing TSN
  • standard deploy model spec in Caffe style based on BN-Inception
  • Trained model weights to reproduce reported accuracies
  • Evaluation framework:
    • Python: adapted from the anet repo.
    • Matlab: partially completed, will add whole dataset evaluation soon.
• Guidance
  • Data preparation
  • Run evaluation with trained models
  • Train models

[yjxiong]

As of Aug. 8, we have released the basic functionalities. The MATLAB scripts and WARP-related stuff will be released very soon.

[double-vane]

@yjxiong flow modality not work now?

[yjxiong]

@double-vane why did you say that?

[double-vane]

@yjxiong Like #7 ,I don't know where I can download the correct flow caffemodel

[yjxiong]

The models downloaded with get_init_models.sh work well with our environments. This is also verified by multiple users. Please check your steps in reproducing the results.

And please do not spam this thread. It'd be better to open a new issue with more details of your problem.

[propaganda12]

Hi， Professor Xiong, in "...rgb_train_val.prototxt" file, new_segment and new_length are set to 3 and 1, respectively, that means a snippet has only 1 frame, based on these settings, I train the model on HMDB51 and test on the entire HMDB51 with the setting of evenly sampling 25 frames per video, but the accuracy are really low, however, the validation accuracy during training can reach 53% only with RGB frames for input. I wonder what may cause such difference. Thank you!

[yjxiong]

@propaganda12
Please use the provided training and testing scripts to see whether you can reproduce the results.