Compiling the GACODE in the JET Heimdall computer

[franal12]

Hi GACODE team!

I am planning to compile the gacode on the JET Heimdall computer using the existing makefile JAC-64 in the build directory. I can compile TGLF but am unable to compile any other codes due to an error in the compilation of the math. The compilation error is shown below

mpif90 -module /home/wh4980/gacode/modules -r8  -O1  -fPIC -c math_constants.f90
mpif90 -module /home/wh4980/gacode/modules -r8  -O1  -fPIC -c gauss_legendre.f90
mpif90 -module /home/wh4980/gacode/modules -r8  -O1  -fPIC -c half_hermite.f90
PGF90-S-0038-Symbol, erf, has not been explicitly declared (half_hermite.f90)
  0 inform,   0 warnings,   1 severes, 0 fatal for pseudo_maxwell_pliocene
make: *** [half_hermite.o] Error 2

Any tips on how to resolve this error?
Thanks,
Francis

[jcandy]

@franal12 Is there a version of mpif90 built with gfortran?

[franal12]

@franal12 Is there a version of mpif90 built with gfortran?

Thanks for your response! Yes, a version of openmpi can be loaded along with gfortran.

[jcandy]

Generally for commodity CPU systems I recommend:

• gfortran
• mpich
• openblas (https://www.openblas.net)
• fftw

For openblas use these settings

BINARY=64
USE_THREAD=0
USE_LOCKING=1
NO_SHARED=1
NO_CBLAS=1
NO_LAPACKE=1

Then, the MINT platform setup will be close to what you can use.

[jcandy]

@franal12 Would it be possible to add some minimal credentials to you github profile?

[jcandy]

@franal12 has there been progress?

[franal12]

Hi @jcandy! I compiled the openblas by using the settings mentioned by you and I am getting the following error

test_post_fork.c: In function ‘__ctest_fork_safety_after_fork_in_parent_run’:
test_post_fork.c:79:12: warning: unused variable ‘nthreads_omp’ [-Wunused-variable]
     int i, nthreads_omp;
            ^
cc -O2 -DSMALL_MATRIX_OPT -DUTEST_CHECK -DSANITY_CHECK -DREFNAME=f_ -DMAX_STACK_ALLOC=2048 -DUSE_LOCKING -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_LAPACK -DNO_LAPACKE -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=16 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.26.dev\" -msse3 -mssse3 -msse4.1 -mavx -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME= -DASMFNAME=_ -DNAME=_ -DCNAME= -DCHAR_NAME=\"_\" -DCHAR_CNAME=\"\" -DNO_AFFINITY -I..  -o openblas_utest utest_main.o test_min.o test_amax.o test_ismin.o test_rotmg.o test_axpy.o test_dotu.o test_dsdot.o test_swap.o test_rot.o test_dnrm2.o test_zscal.o test_amin.o test_axpby.o test_fork.o test_post_fork.o ../libopenblas_sandybridge-r0.3.26.dev.a -lm -lm -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/local/lib -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../..  -lgfortran -lm -lquadmath -lm -lc 
test_zscal.o: In function `__ctest_zscal_i_nan_run':
test_zscal.c:(.text+0x40): undefined reference to `cblas_zscal'
test_zscal.o: In function `__ctest_zscal_i_nan_inc_2_run':
test_zscal.c:(.text+0x110): undefined reference to `cblas_zscal'
test_zscal.o: In function `__ctest_zscal_nan_i_run':
test_zscal.c:(.text+0x1e0): undefined reference to `cblas_zscal'
test_zscal.o: In function `__ctest_zscal_nan_i_inc_2_run':
test_zscal.c:(.text+0x2b0): undefined reference to `cblas_zscal'
test_zscal.o: In function `__ctest_zscal_i_inf_run':
test_zscal.c:(.text+0x380): undefined reference to `cblas_zscal'
test_zscal.o:test_zscal.c:(.text+0x450): more undefined references to `cblas_zscal' follow
collect2: error: ld returned 1 exit status
make[1]: *** [openblas_utest] Error 1
make[1]: Leaving directory `/home/wh4980/OpenBLAS-develop/utest'
make: *** [tests] Error 2

I was asking for help from the local IT support so I couldn't get back to you quickly. I updated my profile now on Github.

[jcandy]

I assume libopenblas.a was generated. Can you assume for the moment it's good and try to build GACODE with it?

[franal12]

yes it is generated so I will try to build GACODE and update the thread.

[franal12]

The make file contains the following lines based on the MINT makefile

# Compilers and flags

FC      = mpif90 -fall-intrinsics -std=f2008 -I$(GACODE_ROOT)/modules -J$(GACODE_ROOT)/modules -fPIC
F77     = mpif77
FOMP    =-fopenmp
FMATH   =-fdefault-real-8 -fdefault-double-8
FOPT    =-Ofast
FDEBUG  =-Wall -fcheck=all -fbacktrace -g -ffpe-trap=invalid,zero,overflow -fimplicit-none -finit-real=nan
F2PY    = f2py

# System math libraries
LMATH = /home/wh4980/OpenBLAS-develop/libopenblas.a

# netCDF (not optional any more)
# NETCDF_HOME=/usr/local/depot/netcdf-4.2.1.1/
# NETCDF_DIR is set by the module
NETCDF=-L${NETCDF_DIR}/lib -lnetcdff
NETCDF_INC=${NETCDF_DIR}/include

# Archive 

ARCH = ar cr

and I am getting the following error

mpif90 -module /home/wh4980/gacode/modules -r8  -O1  -fPIC -c math_constants.f90
gfortran: error: unrecognized command line option ‘-module’
gfortran: error: unrecognized command line option ‘-r8’
make: *** [math_constants.o] Error 1

Is there a document or a guide to understand how makefiles are created? I found some guides on the internet but they explain basic makefile syntaxes.

[jcandy]

Is it possible for me to get an account on Heimdall?

[franal12]

I don't think that is possible but we can have a ZOOM meeting and try compiling the Gacode together.

[franal12]

@jcandy
I successfully compiled neo, vgen and tglf but cgyro gives me the following error

Error: Symbol ‘omega_eb’ at (1) has no IMPLICIT type
cgyro_advect_wavenumber.f90:36:40:
 
                  he(j,ir) = omega_eb*h_x(icc+j,in)
                                        1
Error: Rank mismatch in array reference at (1) (2/3)
cgyro_advect_wavenumber.f90:47:23:

Yes, there is a cgyro_globals.mod file in gacode/module. There are many instances of “Rank mismatch” errors.

Compiling gyro gives this error at the end
gfortran: error: gyro_lib.a: No such file or directory
make: [all] Error 1 (ignored)

[jcandy]

Please generate a make log:

$ cd cgyro
$ make clean
$ make &> log.txt

and attach the file here

[franal12]

Here is the log file after cgyro compilation.
log.txt

[jcandy]

Are you using an old version of GACODE (like from 2022?)

[franal12]

Apparently, it is an older version of code from October 2023. I downloaded the latest GACODE and recompiled the cgyro again. I attached the log file for your reference
log.txt

[jcandy]

The error log shows that you don't have FFTW3 installed.

[franal12]

The necessary file fftw.f03 is in /usr/include/fftw3.f03. How can I change the makefile to include this file?

[jcandy]

Here are the typical settings:

LMATH = ${ROOT}/OpenBLAS/libopenblas.a $(FFTWDIR)/libfftw3.a $(FFTWDIR)/libfftw3_omp.a
FFTW_INC=/usr/include

So, where are the libfftw3.a and libfftw3_omp.a libraries?

[franal12]

Thanks! I found libfftt3.a and libfftw3_omp.a libraries in usr/lib64 directory and I made the corresponding changes in the make file. After compiling the cgyro code, I am getting undefined reference to `cgyro_nl_fftw_comm2_test_' . I have attached the log file for your reference.
log.txt

[jcandy]

@sfiligoi @franal12 It seems like the code that generates the use, intrinsic :: iso_fortran_env warning might be an issue. I see this warning all the time, but it doesn't cause problems.

[sfiligoi]

I don't think the iso_fortran_env is the root cause. It is just a warning.
It has something to do with name mangling.
All the failing subroutines are the ones in files that are not modules.

@franal12 To be pedantic,
can you please make sure the work area is clean?
(After a make clean, check that there are no leftover files that are not managed by git)

Note: cgyro_rhs has no warning, but still fails.

[franal12]

@sfiligoi I checked the gacode directory and there are no extra files present.

[jcandy]

To recap, please do the usual steps:

• make clean in the top-level gacode
• check gacode/modules for any stale mod files
• check gacode/cgyro/src for any stale mod files
• type make in the gacode/cgyro directory and pipe all the output to log.txt
• post the log.txt file

[sfiligoi]

@franal12 I just tried with conda installed gfortran and mpi, and it compiled without a problem using the MINT platform:

conda create -n gacode -c conda-forge gfortran python mpich fftw openblas make binutils
#   gfortran           conda-forge/linux-64::gfortran-13.2.0-hc7bed06_7 
conda activate gacode
export GACODE_PLATFORM=MINT
export GACODE_ROOT=$HOME/gacode
. $GACODE_ROOT/shared/bin/gacode_setup
cd ~/gacode/cgyro
make

Just had to tweak the paths in make.inc.MINT:

(gacode) sfiligoi@petra:~/gacode/platform/build$ git diff
diff --git a/platform/build/make.inc.MINT b/platform/build/make.inc.MINT
index b18525d2a..43f9ab6ae 100644
--- a/platform/build/make.inc.MINT
+++ b/platform/build/make.inc.MINT
@@ -19,8 +19,8 @@ FDEBUG =-Wall -W -fcheck=all -g -fbacktrace -ffpe-trap=invalid,zero,overflow -fi
 F2PY   = f2py
 
 # System math libraries
-LMATH = ${ROOT}/OpenBLAS/libopenblas.a $(FFTWDIR)/libfftw3.a $(FFTWDIR)/libfftw3_omp.a
-FFTW_INC=/usr/include
+LMATH = ${CONDA_PREFIX}/lib/libopenblas.a -L${CONDA_PREFIX}/lib -lfftw3
+FFTW_INC=${CONDA_PREFIX}/include
 
 # OPTIONAL NetCDF:
 NETCDF=-L/usr/lib -lnetcdff -lnetcdf

Something seems to be wrong with your setup.
Maybe mixing up compiler binaries, libraries and/or include files?

[sfiligoi]

I also run the CGYRO validation tests, and they all pass.
But OpenBLAS would complain about OpenMP use, unless one explicitly sets:

export OPENBLAS_NUM_THREADS=1

[franal12]

@jcandy and @sfiligoi I checked modules and cgyro/src directories for stale mod files and they are not present.
This is my makefile for your reference.
`# Compilers and flags
FC = mpif90 -fall-intrinsics -std=f2008 -I$(GACODE_ROOT)/modules -J$(GACODE_ROOT)/modules -fPIC
F77 = mpif77
FOMP =-fopenmp
FMATH =-fdefault-real-8 -fdefault-double-8
FOPT =-Ofast
FDEBUG =-Wall -fcheck=all -fbacktrace -g -ffpe-trap=invalid,zero,overflow -fimplicit-none -finit-real=nan
F2PY = f2py

System math libraries

LMATH = /home/wh4980/OpenBLAS-develop/libopenblas.a /usr/lib64/libfftw3.a /usr/lib64/libfftw3_omp.a
FFTW_INC=/usr/include

# netCDF (not optional any more)

# NETCDF_HOME=/usr/local/depot/netcdf-4.2.1.1/

# NETCDF_DIR is set by the module

NETCDF=-L${NETCDF_DIR}/lib -lnetcdff
NETCDF_INC=${NETCDF_DIR}/include

Archive

ARCH = ar cr
`

[franal12]

@jcandy Here is the log file after the compilation
log.txt

[jcandy]

A make clean was not done here. Can you start with a make clean in the gacode top directory?

[franal12]

I did the top level make clean and here is the log file
log1.txt
I did the make in the cgyro directory after doing a make clean at the cgyro directory too and here is the log file
log3.txt

[sfiligoi]

@franal12 What version of fortran and MPI are you using?

Here is mine:

(gacode) sfiligoi@petra:~$ mpif90 --version
GNU Fortran (conda-forge gcc 13.2.0-7) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

(gacode) sfiligoi@petra:~$ which mpif90
/home/sfiligoi/miniconda3/envs/gacode/bin/mpif90
(gacode) sfiligoi@petra:~$ which gfortran
/home/sfiligoi/miniconda3/envs/gacode/bin/gfortran
(gacode) sfiligoi@petra:~$ mpiexec --version
HYDRA build details:
    Version:                                 4.2.1
    Release Date:                            Wed Apr 17 15:30:02 CDT 2024
...

[franal12]

@sfiligoi Here are the details:

wh4980@heimdall139> mpif90 --version
GNU Fortran (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

wh4980@heimdall139> which mpif90
/usr/local/depot/openmpi-3.0.1-gcc-7.3.0/bin/mpif90
wh4980@heimdall139> which gfortran
/usr/local/depot/gfortran-7.2.0/bin/gfortran
wh4980@heimdall139> mpiexec --version
mpiexec (OpenRTE) 3.0.1

Report bugs to http://www.open-mpi.org/community/help/

[sfiligoi]

That's a very old version of gfortran!
Any chance you can upgrade to something better?
(Even use the one from conda?)

[franal12]

Sure! I will ask the IT support for a newer version and update the thread. I will confirm if I can use COONDA at JET computers.

[franal12]

@sfiligoi I installed the necessary modules via conda and compiled the cgyro again after doing the necessary steps.
Makefile is updated as follow:

FC      = mpif90 -fall-intrinsics -std=f2008 -I$(GACODE_ROOT)/modules -J$(GACODE_ROOT)/modules -fPIC
F77     = mpif77
FOMP    =-fopenmp
FMATH   =-fdefault-real-8 -fdefault-double-8
FOPT    =-Ofast
FDEBUG =-Wall -W -fcheck=all -g -fbacktrace -ffpe-trap=invalid,zero,overflow -fi
F2PY    = f2py

LMATH = ${CONDA_PREFIX}/lib/libopenblas.a -L${CONDA_PREFIX}/lib -lfftw3
FFTW_INC=${CONDA_PREFIX}/include
NETCDF=-L${NETCDF_DIR}/lib -lnetcdff
NETCDF_INC=${NETCDF_DIR}/include

ARCH = ar cr

I am still getting the compilation error and I have attached the log file for your reference
log.txt .

[jcandy]

@franal12 Can we see the new software versions?

[franal12]

Sure! Here is the version information.
(gacode) wh4980@heimdall139> gfortran --version GNU Fortran (conda-forge gcc 13.2.0-8) 13.2.0 Copyright (C) 2023 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[franal12]

Further information if needed.
(gacode) wh4980@heimdall139> mpiexec --version HYDRA build details: Version: 4.2.1 Release Date: Wed Apr 17 15:30:02 CDT 2024 CC: x86_64-conda-linux-gnu-cc -I/home/wh4980/anaconda3/envs/gacode/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/wh4980/anaconda3/envs/gacode/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/wh4980/anaconda3/envs/gacode/include -I/home/wh4980/anaconda3/envs/gacode/include -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -L/home/wh4980/anaconda3/envs/gacode/lib -Wl,-rpath,/home/wh4980/anaconda3/envs/gacode/lib Configure options: '--disable-option-checking' '--prefix=/home/wh4980/anaconda3/envs/gacode' '--with-hwloc=embedded' '--disable-dependency-tracking' '--enable-cxx' '--enable-fortran' '--enable-f08' '--enable-wrapper-dl-type=none' '--disable-opencl' '--with-device=ch4' 'build_alias=x86_64-conda-linux-gnu' 'host_alias=x86_64-conda-linux-gnu' 'MPICHLIB_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix' 'MPICHLIB_CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/wh4980/anaconda3/envs/gacode/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/wh4980/anaconda3/envs/gacode/include' 'MPICHLIB_CXXFLAGS=-fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix' 'MPICHLIB_FFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix' 'MPICHLIB_FCFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix' 'CC=x86_64-conda-linux-gnu-cc' 'CFLAGS=-I/home/wh4980/anaconda3/envs/gacode/include -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -O2' 'LDFLAGS=-L/home/wh4980/anaconda3/envs/gacode/lib -Wl,-rpath,/home/wh4980/anaconda3/envs/gacode/lib' 'CPPFLAGS=-I/home/wh4980/anaconda3/envs/gacode/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/wh4980/anaconda3/envs/gacode/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/wh4980/anaconda3/envs/gacode/include -DNETMOD_INLINE=__netmod_inline_ofi__ -I/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work/src/mpl/include -I/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work/modules/json-c -I/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work/modules/hwloc/include -D_REENTRANT -I/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work/src/mpi/romio/include -I/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work/src/pmi/include -I/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work/modules/yaksa/src/frontend/include -I/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work/modules/libfabric/include' 'CPP=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/_build_env/bin/x86_64-conda-linux-gnu-cpp' 'CXX=x86_64-conda-linux-gnu-c++' 'CXXFLAGS=-I/home/wh4980/anaconda3/envs/gacode/include -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -O2' 'FC=x86_64-conda-linux-gnu-gfortran' 'FCFLAGS=-I/home/wh4980/anaconda3/envs/gacode/include -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -O2' 'FFLAGS=-I/home/wh4980/anaconda3/envs/gacode/include -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/wh4980/anaconda3/envs/gacode/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/mpich-mpi_1717242149661/work=/usr/local/src/conda/mpich-4.2.1 -fdebug-prefix-map=/home/wh4980/anaconda3/envs/gacode=/usr/local/src/conda-prefix -O2' '--cache-file=/dev/null' '--srcdir=.' 'LIBS=' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs cobalt Demux engines available: poll select

[franal12]

@jcandy, I reinstalled the conda environment and still got the same error. I don't know what is causing the same error. Do you have any pointers to resolve this problem?
Thanks!

[sfiligoi]

@franal12 If you are using the latest gacode from git, I do not know what to suggest.
The same exact procedure on another machine works.

Maybe one more thought:
Can you check which binary/version of mpif90 and mpif77 are you using?
(Just to make sure it is the same as the mpiexec version)

[franal12]

sorry for my delay in response, I was busy with my EPS conferene preparations in the last couple of weeks.
I checked the versions of mpif90 and mpif77 and they both have the same version.
GNU Fortran (conda-forge gcc 13.2.0-11) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc
The version for mpiexec is
HYDRA build details:
Version: 4.2.1
Release Date: Wed Apr 17 15:30:02 CDT 2024
Years for versions are different between mpiexec and mpif90 and mpif77 but you also have the same difference so I think it should be fine.
I tried compiling the code again doing a make clean and removing any stale module files. This time I am getting an error in compiling TGLF and vgen and the error is saved in this log_upd.txt file (errors for vgen and tglf are near the end of the file).

I can compile the gacode on my personal linux machine with the same procedure and the makefile so I beleive the error could be due to the environement settings of JET Heimdall system.