I have a question about the Matbench discovery model.

[hyeon4977]

Hello.
I'm trying to fine-tune the OpenLAM weights available at https://www.aissquare.com/models/detail?pageType=models&name=DPA-3.1-3M&id=343 .
Is there a way to fine-tune the trained weights based on all branches, rather than selecting a single branch? Currently, I understand that only one branch can be selected.

[iProzd]

Hi @hyeon4977, when selecting a model-branch, you're actually utilizing all the model's weights including the unified descriptor and fitting net. Each model-branch in this architecture shares identical weights except for a one-hot dataset-encoding vector, meaning your branch selection specifically determines which dataset-encoding is applied alongside the complete set of shared weights rather than selecting partial weight components. For technical details, please refer to the DPA-3 paper. Hope this clarifies the mechanism.

[hyeon4977]

First of all, thank you for your answer!
So, does this mean we simply add one-hot encoding for the material group to the vector created based on the trained weights and then input it into the MLP? Could you share any information on whether this process significantly improves accuracy?
Thank you.

[iProzd]

Hi @hyeon4977 — yes, that’s exactly right. In DPA-3.1-3M we use a multitask setup where each dataset (material group) is assigned a one-hot encoding that is concatenated with the descriptor produced by the shared descriptor network (which contains the vast majority of trainable parameters), and this combined vector is passed through a single, unified MLP (“fitting net”).

Conceptually, there are two common ways to do multitask learning for heterogeneous DFT datasets:

(1) the DPA2 style with a separate readout/fitting MLP per dataset (shared descriptor, multiple heads), and
(2) the DPA3 approach we use here with one shared fitting MLP plus a dataset one-hot. As the number of datasets grows,

approach (1) scales parameter count in fitting nets linearly with the number of heads, while approach (2) keeps parameters essentially fixed and is therefore more scalable. In our experiments (see below),

the two approaches deliver comparable accuracy overall; the main benefit of the dataset encoding + single-head scheme is parameter efficiency and cleaner scaling rather than a large accuracy boost.