Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full merge from upstream 2023-09-21 #192

Merged

Conversation

greenc-FNAL
Copy link
Member

Full merge of upstream into fnal-develop as of 2023-09-21.

  • Also includes audited improvements from v0.21-dev.fermi.

vanderwb and others added 30 commits September 6, 2023 16:12
* cgns: Add -fPIC to Fortan/C compilation flags
  This should fix spack#39699
* cgns: Add -fPIC compilation flags when +fortran
  Incorporating suggestions of @aumuell with addition of making +pic required when +fortran
* seacas: Set TPL_ENABLE_Pthread=ON when +thread_safe
  This should fix spack#39702
  Basically, following suggestion of error message and setting
  TPL_ENABLE_Pthread to the value of the boolean spack variant thread_safe
* seacas: Fix style issue
  Add space after comment #
The condition probably did not get updated when the behavior of the version specifier changed.
* r-tinytiger: Add tinytiger R package.
* Fix homepage.

---------

Co-authored-by: Tamara Dahlgren <[email protected]>
* Add Figcone Package
* Fix the style
* add url
* fix style removing whitespace of a blanck line
* remove versions
Unzip required during build otherwise configuration of luarocks fails during installation of [email protected] on ubuntu 22.04 w/o system unzip.
* neko: add v0.6.1 and fix package
* Change to a conditional variant for gslib
* allow to build latest version of cp2k with cmake
* apply black
Edit configuration file to use the Spack compiler wrappers.
* Write timing information for installs from cache

* CI: aggregate and upload install_times.json to artifacts

* CI: Don't change root directory for artifact generation

* Flat event based timer variation

Event based timer allows for easily starting and stopping timers without
wiping sub-timer data. It also requires less branching logic when
tracking time.

The json output is non-hierarchical in this version and hierarchy is
less rigidly enforced between starting and stopping.

* Add and write timers for top level install

* Update completion

* remove unused subtimer api

* Fix unit tests

* Suppress timing summary option

* Save timers summaries to user_data artifacts

* Remove completion from fish

* Move spack python to script section

* Write timer correctly for non-cache installs

* Re-add hash to timer file

* Fish completion updates

* Fix null timer yield value

* fix type hints

* Remove timer-summary-file option

* Add "." in front of non-package timer name

---------

Co-authored-by: Harmen Stoppels <[email protected]>
Co-authored-by: Harmen Stoppels <[email protected]>
* Docs/Packaging guide: Add BundlePackage
* Adjusted version ordering note to reflect convention.
…k#39883)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.1 to 7.4.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@7.4.1...7.4.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Perform external spec detection with multiple workers

The logic to perform external spec detection has been refactored
into classes. These classes use the GoF "template" pattern to account
for the small differences between searching for "executables" and
for "libraries", while unifying the larger part of the algorithm.

A ProcessPoolExecutor is used to parallelize the work.

* Speed-up external find by tagging detectable packages automatically

Querying packages by tag is much faster than inspecting the repository,
since tags are cached. This commit adds a "detectable" tag to every
package that implements the detection protocol, and external detection
uses it to search for packages.

* Pass package names instead of package classes to workers

The slowest part of the search is importing the Python modules
associated with candidate packages. The import is done serially
before we distribute the work to the pool of executors.

This commit pushes the import of the Python module to the job
performed by the workers, and passes just the name of the packages
to the executors.

In this way imports can be done in parallel.

* Rework unit-tests for Windows

Some unit tests were doing a full e2e run of a command
just to check a input handling. Make the test more
focused by just stressing a specific function.

Mark as xfailed 2 tests on Windows, that will be fixed
by a PR in the queue. The tests are failing because we
monkeypatch internals in the parent process, but the
monkeypatching is not done in the "spawned" child
process.
Smart alias completion introduced in spack#39499 wasn't as smart as it needed to be, and
would complete any invalid command prefix and some env names with alias names.

- [x] don't complete aliases if there are no potential completions
      e.g., don't convert `spack isnotacommand` -> `spack concretize`

- [x] don't complete with an aliases if we're not looking at a top-level subcommand.
Extensionless archives requiring two-stage decompression and extraction
require intermediate archives to be renamed after decompression/extraction
to prevent collision. Prior behavior attempted to cleanup the intermediate
archive with the original name, this PR ensures the renamed folder is
cleaned instead.

Co-authored-by: Dan Lipsa <[email protected]>
Co-authored-by: John Parent <[email protected]>
* Cloverleaf-ref: proposal for a -ref version only
* Cloverleaf-ref: fixing typo.

---------

Co-authored-by: fpanichi <[email protected]>
* updates for VELOC 1.6

* veloc: add v1.7

---------

Co-authored-by: Bogdan Nicolae <[email protected]>
LydDeb and others added 24 commits September 21, 2023 06:36
* [add] py-ansimarkup: new recipe required by py-cylc-flow

* py-ansimarkup: remove version constraint on python, add version 2.1.0
…e-externals-from-buildcache"

This reverts commit 45f8936, reversing
changes made to 4343707.
Put back normalization of the "virtuals" input as a sorted tuple. 

Without this we might get edges that differ just for the order of 
virtuals, or that have lists, which are not hashable.

Add unit-tests to prevent regressions.
…ured

This looks to me like the best compromise regarding externals in a
buildcache. I wouldn't want `spack install` on my machine to install
specs that were marked external on another. At the same time there are
centers that control the target systems on which spack is used, and
would want to use external in buildcaches.

As a solution, reuse concretization will now consider those externals
used in buildcaches that match a locally configured external in
packages.yaml.

So for example person A installs and pushes specs with this config:

```yaml
packages:
  ncurses:
    externals:
    - spec: [email protected] +feature
      prefix: /usr
```

and person B concretizes and installs using that buildcache with the
following config:

```yaml
packages:
  ncurses:
    externals:
    - spec: ncurses@6
    prefix: /usr
```

the spec will be reused (or rather, will be considered for reuse...)
Fixes spack#39622

Add a timeout to compiler detection and allow Spack to proceed when
this timeout occurs.

In all cases, the timeout is 120 seconds: it is assumed any compiler
invocation we do for the purposes of verifying it would resolve in
that amount of time.

Also refine executables that are tested as being possible MSVC
instances, and limit where we try to detect MSVC. In more detail:

* Compiler detection should timeout after a certain period of time.
  Because compiler detection executes arbitrary executables on the
  system, we could encounter a program that just hangs, or even a
  compiler that hangs on a license key or similar. A timeout
  prevents this from hanging Spack.
* Prevents things like cl-.* from being detected as potential MSVC
  installs. cl is always just cl in all cases that Spack supports.
  Change the MSVC class to indicate this.
* Prevent compilers unsupported on certain platforms from being
  detected there (i.e. don't look for MSVC on systems other than
  Windows).

The first point alone is sufficient to address spack#39622, but the
next two reduce the likelihood of timeouts (which is useful since
those slow down the user even if they are survivable).
…39991)

* [add] py-graphql-ws: new recipe, required by py-cylc-uiserver

* py-graphql-ws: remove constraint on version for python
* py-nanobind add cmake path

* fix style

---------

Co-authored-by: Robert Underwood <[email protected]>
Add missing `linux` platform in Charm++ when building for `arm8`.
…ck#40125)

* py-fenics-dolfinx: add upper bound on Python version

* Small fix

* Update var/spack/repos/builtin/packages/py-fenics-dolfinx/package.py

Co-authored-by: Adam J. Stewart <[email protected]>

---------

Co-authored-by: Adam J. Stewart <[email protected]>
Be less specific about gettext with ~libxml2

can use libc+iconv or gettext with or without libxml2

initial round of ups compatability from prev release

readonly bootstrap (cvmfs) workaround

package updates from fnal-develop

added ups compat files

more missing files

moderncmakedomain package

per_os_scope

Redo of FNALssi/spack spack#94

redo of FNALssi spack#162

Revert "redo of FNALssi spack#162"

This reverts commit f92ed6e.

Someone fixed the geant4 recipe with a for loop in the recipe instead,
so not doing this after all.

redo of FNALssi spack#161

more cleanups from fnal-develop

More lib64 fun

speling of startswith

version
Copy link

@marcmengel marcmengel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just going to approve this;

@marcmengel marcmengel merged commit 684d814 into FNALssi:fnal-develop Sep 21, 2023
8 of 12 checks passed
@gartung
Copy link
Member

gartung commented Sep 21, 2023

There are issues with binary_distribiution.py.
https://github.com/FNALssi/spack/actions/runs/6267163081/job/17019655713?pr=192
Just overwrite it with upstream.

@greenc-FNAL
Copy link
Member Author

Argh. I was fixing those! Just going to force-remerge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.