Update to Spack 0.23.0

[heatherkellyucl]

0.23.0 just released.

• Make a 0.23 branch of this repo
• Update README.md
• Update version and sites_root in spacksites/settings/spack_sites.ini
• Update spacksites/myriad-utilities/git-pull-on-myriad.sh and spacksites/myriad-utilities/init-spacksites-on-myriad.sh
• Update the buildcache location in initial_site_mirrors.yaml
• Check our ucl local package repo to see if specs we got from develop now exist in builtin. If they do, delete the local one.
• Check whether the major changes/deprecations for the new version require any alterations to the spack commands spacksites runs, initial_site_modules.yaml, any other config or format changes or changes in default behaviour.
• Check out the new branch in a new directory, as when starting from scratch above.
• Create the new sites_root in /shared/ucl/apps/spack/$version
• Create a new buildcache in the sites_root, checking whether the versions we build are still available in the new Spack version and updating our site .yaml files if they do not.

Once we think everything is working:

• Make 0.23 the default branch of this repo
• Pull on Myriad build01
  • Update Myriad buildcache - compiler
  • Update Myriad buildcache - all of base.yaml, myriad.yaml
  • Update Myriad buildcache - gpu
• Pull on Kathleen build01
  • Update Kathleen buildcache - compiler
  • Update Kathleen buildcache - base.yaml
• Pull on Young login for /lustre/shared/ucl/apps-by-arch/avx512 (except gpu)
  • Update avx512 buildcache - compiler
  • Update avx512 buildcache - base.yaml
• Pull on Michael login for /lustre/shared/ucl/apps-by-arch/avx2
  • Update avx2 buildcache - compiler
  • Update avx2 buildcache - base.yaml
• Finish on Young GPU which uses /lustre/shared/ucl/apps-by-arch/avx2

[heatherkellyucl]

Orca: still a newer point release, 6.0.1 bugfix release.

[heatherkellyucl]

Also keeping local ucl repo charmpp because of the issues I was working on - no new updates in the package anyway.

We do need to keep our local gromacs too because the main package doesn't know about plumed 2.9.2 and what it can patch yet.

[heatherkellyucl]

There are changes to behaviour and new commands, I do not think any of the deprecations and removals affect us.
https://github.com/spack/spack/releases/tag/v0.23.0

[heatherkellyucl]

First compiler install in progress on Myriad, Kathleen, Young.

Need to check on this:

# SPACKSITES: Have set spacks's external compiler and python dependencies - now calling spack -e compiler install
==> Warning: Use of plain text `access_token` in mirror config is deprecated, use environment variables instead (access_token_variable)
==> Fetching https://ghcr.io/v2/spack/bootstrap-buildcache-v1/blobs/sha256:3cc99d42a12e6c34bc187d5146cbdba71ca16d19163fe106f1a4cd3e59b01d2c
==> Fetching https://ghcr.io/v2/spack/bootstrap-buildcache-v1/blobs/sha256:e8518de25baff7a74bdb42193e6e4b0496e7d0688434c42ce4bdc92fe4293a09
==> Installing "clingo-bootstrap@=spack%gcc@=10.2.1~docs+ipo+optimized+python+static_libstdcpp build_system=cmake build_type=Release generator=make patches=bebb819,e
c99431 arch=linux-centos7-x86_64" from a buildcache

[heatherkellyucl]

The access_token part comes from https://github.com/UCL-ARC/hpc-spack/blob/0.23/spacksites/settings/initial_site_mirrors.yaml where we set it to null. It doesn't get populated later in the site creation process so remains like that.

Changing the template to access_token_variable: null would work, or removing it entirely since we don't use it.

I don't see much point in defining all of these as null for our local buildcache so I think I will remove them. They'll just get out of date in future like this one did.

      access_pair:
      - null
      - null
      access_token: null
      profile: null
      endpoint_url: null

[heatherkellyucl]

For a site that already exists, you can do spack config edit mirrors or directly edit your site's version of /shared/ucl/apps/spack/0.23/hk-stack/spack/etc/spack/mirrors.yaml

[heatherkellyucl]

Many things are building.

[heatherkellyucl]

It tried to install a version of git that doesn't pass can't compile its own unit tests:

==> Installing git-2.47.0-vm6esyh3dpaivwte27zzrya4skvc3xws [76/147]
==> No binary for git-2.47.0-vm6esyh3dpaivwte27zzrya4skvc3xws found: installing from source
==> Fetching https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.47.0.tar.gz
==> Fetching https://www.kernel.org/pub/software/scm/git/git-manpages-2.47.0.tar.gz
==> Moving resource stage
        source: /lustre/shared/ucl/apps/spack/0.23/hk-stack/build_stage/ccspapp/resource-git-manpages-vm6esyh3
dpaivwte27zzrya4skvc3xws/spack-src/
        destination: /lustre/shared/ucl/apps/spack/0.23/hk-stack/build_stage/ccspapp/spack-stage-git-2.47.0-vm
6esyh3dpaivwte27zzrya4skvc3xws/spack-src/git-manpages
==> Ran patch() for git
==> git: Executing phase: 'autoreconf'
==> git: Executing phase: 'configure'
==> git: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    '/lustre/shared/ucl/apps/spack/0.23/hk-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_place
holder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack/linux-rhel7-cascadelake/gcc-11.2.1/gmake-4.4.1-mdbzk23bg5ywzqeapoyiokykdtdj2gl3/bin/make' '-j6'

2 errors found in build log:
     948        GEN t/unit-tests/clar.suite
     949        CC t/unit-tests/ctype.o
     950        CC t/unit-tests/strvec.o
     951        CC t/unit-tests/clar/clar.o
     952        CC t/unit-tests/unit-test.o
     953    t/unit-tests/clar/clar.c: In function 'clar_time_now':
  >> 954    t/unit-tests/clar/clar.c:274:25: error: storage size of 'tz' isn't known
     955      274 |         struct timezone tz;
     956          |                         ^~

Appears to be this: git/git@1ee7dbd#diff-06ab9c0d690e6c9ba2afc2a145fe5ca943972607262a0414e589f443020e72cbL274

Pulled in by [email protected], needed by py-setuptools-scm-8.0.4.

[heatherkellyucl]

And my licensed software installs are failing again so it isn't using the source cache again, will have to find what it doesn't like this time:

==> Installing namd-2.14-zqnqyjkopuy4fcbffpsgjzw6wnlgi4wv [102/147]
==> No binary for namd-2.14-zqnqyjkopuy4fcbffpsgjzw6wnlgi4wv found: installing from source
==> Error: FetchError: Manual download is required for namd. Refer to https://www.ks.uiuc.edu/Research/namd/ for download instructions.
==> Installing castep-23.1-x6rjjfwms7acvuxk5p2txx4zoswleive [103/147]
==> No binary for castep-23.1-x6rjjfwms7acvuxk5p2txx4zoswleive found: installing from source
==> Error: FetchError: Manual download is required for castep. Refer to http://castep.org for download instructions.
==> Installing castep-24.1-gbvjcoj4crbtdur5glhnnus47cv35ab6 [105/147]
==> No binary for castep-24.1-gbvjcoj4crbtdur5glhnnus47cv35ab6 found: installing from source
==> Error: FetchError: Manual download is required for castep. Refer to http://castep.org for download instructions.

[heatherkellyucl]

I cheated and copied the castep 24.1 source-archive from the 0.22 area to the 0.23 area so that it could be found - in that, it is only named using sha256sum and in a subdirectory of the first two characters.

(spacksite: hk-stack) [ccspapp@build01 castep]$ sha256sum castep-24.1.tar.gz
97d77a4f3ce3f5c5b87e812f15a2c2cb23918acd7034c91a872b6d66ea0f7dbb  castep-24.1.tar.gz
(spacksite: hk-stack) [ccspapp@build01 castep]$ mkdir -p /shared/ucl/apps/spack/0.23/hk-stack/spack/var/spack/cache/_source-cache/archive/97
(spacksite: hk-stack) [ccspapp@build01 castep]$ cp /shared/ucl/apps/spack/0.22/hk-stack/spack/var/spack/cache/_source-cache/archive/97/97d77a4f3ce3f5c5b87e812f15a2c2cb23918acd7034c91a872b6d66ea0f7dbb.tar.gz /shared/ucl/apps/spack/0.23/hk-stack/spack/var/spack/cache/_source-cache/archive/97/97d77a4f3ce3f5c5b87e812f15a2c2cb23918acd7034c91a872b6d66ea0f7dbb.tar.gz

To work out what a mirror has to look like now I made a new one, since the docs are out of date:

spack mirror create -d /shared/ucl/apps/pkg-store/spack-mirror --reuse [email protected]
==> Concretizing input specs
==> Adding package [email protected] to mirror
==> Using cached archive: /lustre/shared/ucl/apps/spack/0.23/hk-stack/spack/var/spack/cache/_source-cache/archive/97/97d77a4f3ce3f5c5b87e812f15a2c2cb23918acd7034c91a872b6d66ea0f7dbb.tar.gz
==> Summary for mirror in /shared/ucl/apps/pkg-store/spack-mirror
==> Archive stats:
    0    already present
    1    added
    0    failed to fetch.

(spacksite: hk-stack) [ccspapp@build01 hpc-spack]$ ls -al /shared/ucl/apps/pkg-store/spack-mirror
total 24
drwxr-xr-x  4 ccspapp ccsp  4096 Nov 22 13:14 .
drwxrwxr-x 18 ccspapp ccsp 12288 Nov 22 13:14 ..
drwxr-xr-x  2 ccspapp ccsp  4096 Nov 22 13:14 castep
drwxr-xr-x  3 ccspapp ccsp  4096 Nov 22 13:14 _source-cache
(spacksite: hk-stack) [ccspapp@build01 hpc-spack]$ ls -al /shared/ucl/apps/pkg-store/spack-mirror/castep/
total 12
drwxr-xr-x 2 ccspapp ccsp  4096 Nov 22 13:14 .
drwxr-xr-x 4 ccspapp ccsp  4096 Nov 22 13:14 ..
lrwxrwxrwx 1 ccspapp ccsp    99 Nov 22 13:14 castep-24.1.tar.gz -> ../_source-cache/archive/97/97d77a4f3ce3f5c5b87e812f15a2c2cb23918acd7034c91a872b6d66ea0f7dbb.tar.gz

(I then fixed the group and other permissions so only lgcastep can access).

So we have a human-readable name symlinked to the sha256sum archive. (And the layout in /lustre/shared/ucl/apps/spack/0.23/hk-stack/spack/var/spack/cache is the same so also has the new human-readable symlinks).

Now in /shared/ucl/apps/pkg-store/spack-mirror/castep I have this and hopefully castep-23.1 can install too now:

(spacksite: hk-stack) [ccspapp@build01 castep]$ ls -al
total 20
drwxr-x--- 2 ccspapp lgcastep 4096 Nov 22 13:56 .
drwxr-xr-x 4 ccspapp ccsp     4096 Nov 22 13:45 ..
lrwxrwxrwx 1 ccspapp ccsp       95 Nov 22 13:56 Castep_23.1_build_fixes.diff.gz-5f31daf.gz -> ../_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
lrwxrwxrwx 1 ccspapp ccsp       99 Nov 22 13:47 castep-23.1.tar.gz -> ../_source-cache/archive/7f/7fba0450d3fd71586c8498ce51975bbdde923759ab298a656409280c29bf45b5.tar.gz
lrwxrwxrwx 1 ccspapp ccsp       99 Nov 22 13:14 castep-24.1.tar.gz -> ../_source-cache/archive/97/97d77a4f3ce3f5c5b87e812f15a2c2cb23918acd7034c91a872b6d66ea0f7dbb.tar.gz

[heatherkellyucl]

That worked.