Skip to content

[BUG] MultiSystems load fparam shape mismatch #831

Closed
@anyangml

Description

@anyangml

Bug summary

When constructing the data set, fparam was registered

DataType(
            "fparam",
            np.ndarray,
            shape=(Axis.NFRAMES, 2),
            required=False,
        ),

...

system = dpdata.MultiSystems()
fparam = np.array([atoms.info["charge"],atoms.info["spin"]]).reshape(1,2)
assert fparam.shape == (1, 2)
frame.data["fparam"] = fparam
system.append(frame)
system.to_deepmd_npy_mixed(temp_out_pth)

When loading the system using


DataType(
            "fparam",
            np.ndarray,
            shape=(Axis.NFRAMES, 2),
            required=False,
        ),

sys = dpdata.MultiSystems()
sys.load_systems_from_file(filepth, fmt="deepmd/npy/mixed")

The following error occurs

dpdata.data_type.DataError: Shape of fparam is (2361, 2), but expected (6, 2)

dpdata Version

dpdata v0.2.24

Input Files, Running Commands, Error Log, etc.

Systems: 0%| | 0/1 [00:00<?, ?it/s]/mnt/data_nas/public/public_training_data/OMol25_fparam/val_0/10
Systems: 0%| | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/mnt/data_nas/penganyang/LAMBench/lambench/models/ase_models.py", line 443, in
print(omol.run_ase_dptest(omol.calc, Path("/mnt/data_nas/public/public_training_data/OMol25_fparam/")))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/data_nas/penganyang/LAMBench/lambench/models/ase_models.py", line 288, in run_ase_dptest
sys.load_systems_from_file(filepth, fmt="deepmd/npy/mixed")
File "/mnt/data_nas/public/Miniconda/envs/omol25/lib/python3.11/site-packages/dpdata/system.py", line 1493, in load_systems_from_file
return self.from_fmt_obj(load_format(fmt), file_name, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/data_nas/public/Miniconda/envs/omol25/lib/python3.11/site-packages/dpdata/system.py", line 1396, in from_fmt_obj
system_list.append(LabeledSystem(data=data_item, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/data_nas/public/Miniconda/envs/omol25/lib/python3.11/site-packages/dpdata/system.py", line 193, in init
self.check_data()
File "/mnt/data_nas/public/Miniconda/envs/omol25/lib/python3.11/site-packages/dpdata/system.py", line 221, in check_data
dd.check(self)
File "/mnt/data_nas/public/Miniconda/envs/omol25/lib/python3.11/site-packages/dpdata/data_type.py", line 119, in check
raise DataError(
dpdata.data_type.DataError: Shape of fparam is (2361, 2), but expected (6, 2)
(omol25) root@dsw-140567-9888d4895-8xtqz:/mnt/data_nas/penganyang/LAMBench/lambench/models# ^C
(omol25) root@dsw-140567-9888d4895-8xtqz:/mnt/data_nas/penganyang/LAMBench/lambench/models# ls /mnt/data_nas/public/public_training_data/OMol25_fparam/val_0/10
nopbc set.000 set.001 type_map.raw type.raw
(omol25) root@dsw-140567-9888d4895-8xtqz:/mnt/data_nas/penganyang/LAMBench/lambench/models# python -i
Python 3.11.0 (main, Mar 1 2023, 18:26:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import numpy as np
exit()
(omol25) root@dsw-140567-9888d4895-8xtqz:/mnt/data_nas/penganyang/LAMBench/lambench/models# ls /mnt/data_nas/public/public_training_data/OMol25_fparam/val_0/10/set.000
box.npy coord.npy energy.npy force.npy fparam.npy homo_energy.npy homo_lumo_gap.npy real_atom_types.npy
(omol25) root@dsw-140567-9888d4895-8xtqz:/mnt/data_nas/penganyang/LAMBench/lambench/models# python -i
Python 3.11.0 (main, Mar 1 2023, 18:26:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import numpy as np
np.load("/mnt/data_nas/public/public_training_data/OMol25_fparam/val_0/10/set.000/fparam.npy").shape
(2000, 2)
np.load("/mnt/data_nas/public/public_training_data/OMol25_fparam/val_0/10/set.001/fparam.npy").shape
(361, 2)
np.load("/mnt/data_nas/public/public_training_data/OMol25_fparam/val_0/10/set.001/real_atom_types.npy").shape
(361, 10)
np.load("/mnt/data_nas/public/public_training_data/OMol25_fparam/val_0/10/set.000/real_atom_types.npy").shape
(2000, 10)

Steps to Reproduce

debug.zip

from dpdata.data_type import (
    Axis,
    DataType,
)
import dpdata

datatypes_ef=[
        DataType(
            "fparam",
            np.ndarray,
            shape=(Axis.NFRAMES, 2),
            required=False,
        ),
    ]
    
for datatype in datatypes_ef:
    dpdata.System.register_data_type(datatype)
    dpdata.LabeledSystem.register_data_type(datatype)


sys = dpdata.MultiSystems()
sys.load_systems_from_file("debug", fmt="deepmd/npy/mixed")

Further Information, Files, and Links

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions