Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When I run multifindbestfit.py, It reports an error: CUDA error: invalid configuration argument. How can I fix it ? #15

Open
laiyingxin2 opened this issue Mar 10, 2022 · 6 comments

Comments

@laiyingxin2
Copy link

No description provided.

@Eaton2022
Copy link

python -m btcdet.datasets.multifindbestfit
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda-11.1:/usr/local/cuda-11.1:'
Traceback (most recent call last):
File "/home/scy/anaconda3/envs/pcdet1/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/scy/anaconda3/envs/pcdet1/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/scy/project/btcdet/BtcDet/btcdet/datasets/multifindbestfit.py", line 23, in
from ..ops.chamfer_distance import ChamferDistance
File "/home/scy/project/btcdet/BtcDet/btcdet/ops/chamfer_distance/init.py", line 1, in
from .chamfer_distance import ChamferDistance
File "/home/scy/project/btcdet/BtcDet/btcdet/ops/chamfer_distance/chamfer_distance.py", line 11, in
cd = load(name="cd",
File "/home/scy/anaconda3/envs/pcdet1/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1080, in load
return _jit_compile(
File "/home/scy/anaconda3/envs/pcdet1/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1293, in _jit_compile
_write_ninja_file_and_build_library(
File "/home/scy/anaconda3/envs/pcdet1/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1392, in _write_ninja_file_and_build_library
_write_ninja_file_to_build_library(
File "/home/scy/anaconda3/envs/pcdet1/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1783, in _write_ninja_file_to_build_library
cuda_flags = common_cflags + COMMON_NVCC_FLAGS + _get_cuda_arch_flags()
File "/home/scy/anaconda3/envs/pcdet1/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1562, in _get_cuda_arch_flags
arch_list[-1] += '+PTX'
IndexError: list index out of range

can anybody help me?

@hiepbk
Copy link

hiepbk commented Apr 19, 2022

open bashrc file and add following commands:
export PATH="/usr/local/cuda-11.6/bin:$PATH"
export LD_LIBRARY_PATH="'usr/local/cuda-11.6/lib64:$LD_LIBRARY_PATH"
export CUDA_HOME=/usr/local/cuda-11.6
replace '11.6' corresponding with your cuda version

@Castiel-Lee
Copy link

Hello, I meet the same problem. But,

'''
open bashrc file and add following commands:
export PATH="/usr/local/cuda-11.6/bin:$PATH"
export LD_LIBRARY_PATH="'usr/local/cuda-11.6/lib64:$LD_LIBRARY_PATH"
export CUDA_HOME=/usr/local/cuda-11.6
replace '11.6' corresponding with your cuda version
''''

it didn't solve it.

No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda-11.1'
Traceback (most recent call last):
File "/home/xxx/miniconda/envs/Btcdet/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/xxx/miniconda/envs/Btcdet/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/xxx/shuangzh_workspace/BtcDet_env/BtcDet/btcdet/datasets/multifindbestfit.py", line 21, in
from ..ops.chamfer_distance import ChamferDistance
File "/home/xxx/shuangzh_workspace/BtcDet_env/BtcDet/btcdet/ops/chamfer_distance/init.py", line 1, in
from .chamfer_distance import ChamferDistance
File "/home/xxx/shuangzh_workspace/BtcDet_env/BtcDet/btcdet/ops/chamfer_distance/chamfer_distance.py", line 11, in
cd = load(name="cd",
File "/home/xxx/miniconda/envs/Btcdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1079, in load
return _jit_compile(
File "/home/xxx/miniconda/envs/Btcdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1292, in _jit_compile
_write_ninja_file_and_build_library(
File "/home/xxx/miniconda/envs/Btcdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1391, in _write_ninja_file_and_build_library
_write_ninja_file_to_build_library(
File "/home/xxx/miniconda/envs/Btcdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1782, in _write_ninja_file_to_build_library
cuda_flags = common_cflags + COMMON_NVCC_FLAGS + _get_cuda_arch_flags()
File "/home/xxx/miniconda/envs/Btcdet/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1561, in _get_cuda_arch_flags
arch_list[-1] += '+PTX'
IndexError: list index out of range

@Rumilku
Copy link

Rumilku commented May 7, 2022

I meet the same problem. If your gpus<4, this error may be caused by line 10 'os.environ["CUDA_VISIBLE_DEVICES"] = "3"', remove it.

@Castiel-Lee
Copy link

Thanks. It solved my problem!

@ElizabethSF
Copy link

I solve this probelm using export TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6+PTX"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants