🐛 [BUG] - RTX 4090 got an invalid device symbol #1811
Description
Description
Hi, I'm using specfem3d on a server with a RTX 4090, and the following command is utilized to compile the software:
./configure FC=gfortran CC=gcc MPIFC=mpif90 --with-mpi=/opt/openmpi/5.0.7 --with-cuda=cuda11Since RTX 4090 is a member of Ada Lovelace architecture devices, and they are not listed in the device lists, I use CUDA 11 alternatively. In my opinion, Ada Lovelace is quite similar to Ampere.
The generated Makefile is
...
GENCODE_20 = -gencode=arch=compute_20,code=\"sm_20,compute_20\"
GENCODE_30 = -gencode=arch=compute_30,code=\"sm_30,compute_30\"
GENCODE_35 = -gencode=arch=compute_35,code=\"sm_35,compute_35\"
GENCODE_37 = -gencode=arch=compute_37,code=\"sm_37\"
GENCODE_50 = -gencode=arch=compute_50,code=\"sm_50,compute_50\"
GENCODE_52 = -gencode=arch=compute_52,code=\"sm_52,compute_52\"
GENCODE_60 = -gencode=arch=compute_60,code=\"sm_60,compute_60\"
GENCODE_70 = -gencode=arch=compute_70,code=\"sm_70,compute_70\"
GENCODE_75 = -gencode=arch=compute_75,code=\"sm_75,compute_75\"
GENCODE_80 = -gencode=arch=compute_80,code=\"sm_80,compute_80\"
GENCODE_90 = -gencode=arch=compute_90,code=\"sm_90,compute_90\"
# cuda preprocessor flag
# CUDA version 12.0
#GENCODE = $(GENCODE_90) $(FC_DEFINE)GPU_DEVICE_Hopper
# CUDA version 11.0
GENCODE = $(GENCODE_80) $(FC_DEFINE)GPU_DEVICE_Ampere
# CUDA version 10.0
#GENCODE = $(GENCODE_75) $(FC_DEFINE)GPU_DEVICE_Turing
# CUDA version 9.0
#GENCODE = $(GENCODE_70) $(FC_DEFINE)GPU_DEVICE_Volta
# CUDA version 8.0
#GENCODE = $(GENCODE_60) $(FC_DEFINE)GPU_DEVICE_Pascal
# CUDA version 7.x
#GENCODE = $(GENCODE_52) $(FC_DEFINE)GPU_DEVICE_Maxwell
# CUDA version 6.5
#GENCODE = $(GENCODE_37) $(FC_DEFINE)GPU_DEVICE_K80
# CUDA version 5.x
#GENCODE = $(GENCODE_35) $(FC_DEFINE)GPU_DEVICE_K20
# CUDA version 4.x
#GENCODE = $(GENCODE_30)
## old CUDA toolkit versions < 5
#GENCODE = $(GENCODE_20)
...
The compilation is finished succefully, but an error occurs when I try to execute a simple example of the software meshfem3D_examples/simple_model with the 383rd line in DATA/Par_file be modified as
GPU_MODE = .true.
to enable the GPU acceleration.
However, an error occurs as
xxx@xxx:~/Desktop/specfem3d-4.1.1/EXAMPLES/applications/meshfem3D_examples/simple_model$ ./run_this_example.sh
running example: 2025年 06月 03日 星期二 21:28:06 CST
setting up example...
running mesher on 4 processors...
running database generation on 4 processors...
running solver on 4 processors...
Error in setConst_hprime_xx: invalid device symbol
The problem is maybe that the architecture specified in the Makefile (-gencode=arch=compute_**,code="sm_**,compute_**") doesn't fit your card, please doublecheck
Error in setConst_hprime_xx: invalid device symbol
The problem is maybe that the architecture specified in the Makefile (-gencode=arch=compute_**,code="sm_**,compute_**") doesn't fit your card, please doublecheck
Error in setConst_hprime_xx: invalid device symbol
The problem is maybe that the architecture specified in the Makefile (-gencode=arch=compute_**,code="sm_**,compute_**") doesn't fit your card, please doublecheck
Error in setConst_hprime_xx: invalid device symbol
The problem is maybe that the architecture specified in the Makefile (-gencode=arch=compute_**,code="sm_**,compute_**") doesn't fit your card, please doublecheck
--------------------------------------------------------------------------
prterun detected that one or more processes exited with non-zero status,
thus causing the job to be terminated. The first process to do so was:
Process name: [prterun-asrock-35143@1,3]
Exit code: 1
--------------------------------------------------------------------------
It is appreciated that anyone could tell me how to configure the Par_file to fix it.
Affected SPECFEM3D version
4.1.1
Your software and hardware environment
Ubuntu desktop 20.04.1, MPI 5.0.7, gcc 9.4.0, RTX 4090, CUDA driver 570.124.06, CUDA version 12.8
Reproduction steps
1. cd specfem3d-4.1.1
2. ./configure FC=gfortran CC=gcc MPIFC=mpif90 --with-mpi=/opt/openmpi/5.0.7 --with-cuda=cuda11 --cuda-libraries=/usr/local/cuda/lib64
3. make
4. vim ./DATA/Par_file (and modify `GPU_MODE` from false to true).
5. ./run_this_example.shScreenshots
Logs
OS
Linux