v0.16.0 (2020-11-18)

v0.16.0 is a major feature release.

Major features in this release

1. New concretizer (experimental) Our new backtracking concretizer is now in Spack as an experimental feature. You will need to install clingo@master+python and set concretizer: clingo in config.yaml to use it. The original concretizer is not exhaustive and is not guaranteed to find a solution if one exists. We encourage you to use the new concretizer and to report any bugs you find with it. We anticipate making the new concretizer the default and including all required dependencies for it in Spack v0.17. For more details, see #19501.

2. spack test (experimental) Users can add test() methods to their packages to run smoke tests on installations with the new spack test command (the old spack test is now spack unit-test). spack test is environment-aware, so you can spack install an environment and spack test run smoke tests on all of its packages. Historical test logs can be perused with spack test results. Generic smoke tests for MPI implementations, C, C++, and Fortran compilers as well as specific smoke tests for 18 packages. This is marked experimental because the test API (self.run_test()) is likely to be change, but we encourage users to upstream tests, and we will maintain and refactor any that are added to mainline packages (#15702).

3. spack develop New spack develop command allows you to develop several packages at once within a Spack environment. Running spack develop foo@v1 and spack develop bar@v2 will check out specific versions of foo and bar into subdirectories, which you can then build incrementally with spack install (#15256).

4. More parallelism Spack previously installed the dependencies of a single spec in parallel. Entire environments can now be installed in parallel, greatly accelerating builds of large environments. get parallelism from individual specs. Spack now parallelizes entire environment builds (#18131).

5. Customizable base images for spack containerize spack containerize previously only output a Dockerfile based on official Spack images from Dockerhub. You may now specify any base image of your choosing (#15028).

6. more external finding spack external find was added in v0.15, but only cmake had support. spack external find can now find bison, cuda, findutils, flex, git, lustre m4, mpich, mvapich2, ncurses, openmpi, perl, spectrum-mpi, tar, and texinfo on your system and add them automatically to packages.yaml.

7. Support aocc, nvhpc, and oneapi compilers We are aggressively pursuing support for the newest vendor compilers, especially those for the U.S. exascale and pre-exascale systems. Compiler classes and auto-detection for aocc, nvhpc, oneapi are now in Spack (#19345, #19294, #19330).

Additional new features of note

• New spack mark command can be used to designate packages as explicitly installed, so that spack gc will not garbage-collect them (#16662).
• install_tree can be customized with Spack's projection format (#18341)
• sbang now lives in the install_tree so that all users can access it (#11598)
• csh and tcsh users no longer need to set SPACK_ROOT before sourcing setup-env.csh (#18225)
• Spec syntax now supports variant=* syntax for finding any package that has a particular variant (#19381).
• Spack respects SPACK_GNUPGHOME variable for custom GPG directories (#17139)
• Spack now recognizes Graviton chips

Major refactors

• Use spawn instead of fork on Python >= 3.8 on macOS (#18205)
• Use indexes for public build caches (#19101, #19117, #19132, #19141, #19209)
• sbang is an external package now (https://github.com/spack/sbang, #19582)
• archspec is an external package now (https://github.com/archspec/archspec, #19600)

Deprecations and Removals

• spack bootstrap was deprecated in v0.14.0, and has now been removed.
• spack setup is deprecated as of v0.16.0.
• What was spack test is now called spack unit-test. spack test is now the smoke testing feature in (2) above.

Bugfixes

Some of the most notable bugfixes in this release include:

• Better warning messages for deprecated syntax in packages.yaml (#18013)
• buildcache list --allarch now works properly (#17827)
• Many fixes and tests for buildcaches and binary relocation (#15687, *#17455, #17418, #17455, #15687, #18110)

Package Improvements

Spack now has 5050 total packages, 720 of which were added since v0.15.

• ROCm packages (hip, aomp, more) added by AMD (#19957, #19832, others)
• Many improvements for ARM support
• llvm-flang, flang, and f18 removed, as llvm has real flang support since Flang was merged to LLVM mainline
• Emerging support for spack external find and spack test in packages.

Infrastructure

• Major infrastructure improvements to pipelines on gitlab.spack.io
• Support for testing PRs from forks (#19248) is being enabled for all forks to enable rolling, up-to-date binary builds on develop