How to use pretrained Weights (Sensor-Agnostic)?

[hai4john]

Pretrained Weights (DOFABase16_Weights.DOFA_MAE/DOFALarge16_Weights.DOFA_MAE) have reported performance on m-eurosat. Does anyone know how this performance was measured? Can you provide a python script?

[adamjstewart]

Performance metrics were actually calculated without TorchGeo (it was added to TorchGeo after the paper was released). You can see most of the pretraining code here: https://github.com/zhu-xlab/DOFA. The paper also contains some details of how evaluation was performed: https://arxiv.org/abs/2403.15356. @xiong-zhitong may be able to add more details/context.

[hai4john]

You mean that GFMs (CROMA, DOFA) models cannot be used on torchgeo tasks (ClassificationTask, SemanticSegmentationTask, etc.), right? The performance reported by these models is not the actual running of torchgeo?

[adamjstewart]

Correct. Many of these models require custom metadata that is not present in all datasets and is not compatible with all other models. Each of these GFMs is unique and a bit more challenging to use than the average timm model. We would love to make these easier to use, but still have a long way to go before they are integrated into all TorchGeo tasks. TerraTorch (built on top of TorchGeo) is a bit better in this regard.