Problems on running the code

[ghost]

Dear authors,

I am interested in your PRL paper and trying to run the code locally on my PC.
After installing all the required packages, the package still cannot work properly.
The command line "python bin/stochastic_reconfiguration.py run_id=MyTestRun" cannot give proper outputs,
but return the following error messages:

"
/home/dbai/anaconda3/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'config': Defaults list is missing _self_. See https://hydra.cc/docs/upgrades/1.0_to_1.1/default_composition_order for more information
warnings.warn(msg, UserWarning)
/home/dbai/anaconda3/lib/python3.8/site-packages/hydra/core/default_element.py:122: UserWarning: In 'wavefunction/DeepSets': Usage of deprecated keyword in package header '# @Package group'.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/changes_to_package_header for more information
deprecation_warning(
/home/dbai/anaconda3/lib/python3.8/site-packages/hydra/core/default_element.py:122: UserWarning: In 'optimizer/Flat': Usage of deprecated keyword in package header '# @Package group'.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/changes_to_package_header for more information
deprecation_warning(
/home/dbai/anaconda3/lib/python3.8/site-packages/hydra/core/default_element.py:122: UserWarning: In 'hamiltonian/Nuclear': Usage of deprecated keyword in package header '# @Package group'.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/changes_to_package_header for more information
deprecation_warning(
"

My platform is Debian GNU/Linux 10 (buster).
We also try to install the packages on other Linux distributions. Unfortunately, none of the installations work properly.

If possible, would you please take a look at the problem? Or could you please explain in more detail the environment that you run the code? Thanks a lot.

[coreyjadams]

Hi @dbai1989 - thanks for your interest, happy to help as much as possible.

I just did a fresh clone of this repository and ran the command you have above:
python bin/stochastic_reconfiguration.py run_id=Test

I see the same warnings you do, but I also see things continue and complete the few iterations specified in the default. You should see output like this:

$ python bin/stochastic_reconfiguration.py run_id=Test
/Users/corey.adams/SOFTWARE/larcv-env/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'config': Defaults list is missing `_self_`. See https://hydra.cc/docs/upgrades/1.0_to_1.1/default_composition_order for more information
  warnings.warn(msg, UserWarning)
/Users/corey.adams/SOFTWARE/larcv-env/lib/python3.8/site-packages/hydra/core/default_element.py:122: UserWarning: In 'wavefunction/DeepSets': Usage of deprecated keyword in package header '# @package _group_'.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/changes_to_package_header for more information
  deprecation_warning(
/Users/corey.adams/SOFTWARE/larcv-env/lib/python3.8/site-packages/hydra/core/default_element.py:122: UserWarning: In 'optimizer/Flat': Usage of deprecated keyword in package header '# @package _group_'.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/changes_to_package_header for more information
  deprecation_warning(
/Users/corey.adams/SOFTWARE/larcv-env/lib/python3.8/site-packages/hydra/core/default_element.py:122: UserWarning: In 'hamiltonian/Nuclear': Usage of deprecated keyword in package header '# @package _group_'.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/changes_to_package_header for more information
  deprecation_warning(
2021-09-23 12:40:05,271 - INFO - hamiltonian:
  mass: 938.95
  form: NuclearPotential
optimizer:
  delta: 0.001
  epsilon: 0.0001
  form: Flat
wavefunction:
  form: DeepSets
  n_filters_per_layer: 16
  n_layers: 2
  bias: true
  activation: tanh
  residual: false
  mean_subtract: true
  confinement: 0.1
iterations: 5
sampler:
  n_thermalize: 5000
  n_void_steps: 200
  n_observable_measurements: 20
  n_walkers_per_observation: 200
  n_concurrent_obs_per_rank: 20
nparticles: 2
dimension: 3
run_id: Test
save_path: output/${hamiltonian.form}/${nparticles}particles/${dimension}D/${optimizer.form}.${run_id}/
model_name: ${hamiltonian.form}_${nparticles}part_${dimension}D.model

2021-09-23 12:40:05,290 - INFO - Setting vkr to 4 in the nuclear potential by default.
2021-09-23 12:40:05,291 - INFO - Using vkr = 4.0
2021-09-23 12:40:05,455 - INFO - Number of parameters in this network: 896
2021-09-23 12:40:05,460 - INFO - Trying to restore model
2021-09-23 12:40:05,460 - DEBUG - Failed to load weights!
2021-09-23 12:40:05,460 - DEBUG - NuclearPotential_2part_3D.model not found.
2021-09-23 12:40:05,460 - INFO - About to thermalize.
2021-09-23 12:40:16,146 - INFO - Finished thermalization.
2021-09-23 12:40:16,146 - INFO - About to compile.
2021-09-23 12:40:18,248 - INFO - Finished compilation.
2021-09-23 12:40:26,351 - INFO - step = 0, energy = 4.939, err = 0.192
2021-09-23 12:40:26,351 - INFO - step = 0, energy_jf = 4.969, err = 0.237
2021-09-23 12:40:26,351 - INFO - acc  = 0.922
2021-09-23 12:40:26,351 - INFO - time = 8.038
2021-09-23 12:40:29,504 - INFO - step = 1, energy = 4.356, err = 0.356
2021-09-23 12:40:29,504 - INFO - step = 1, energy_jf = 4.391, err = 0.440
2021-09-23 12:40:29,504 - INFO - acc  = 0.922
2021-09-23 12:40:29,504 - INFO - time = 3.124
2021-09-23 12:40:32,784 - INFO - step = 2, energy = 4.309, err = 0.325
2021-09-23 12:40:32,784 - INFO - step = 2, energy_jf = 4.119, err = 0.318
2021-09-23 12:40:32,785 - INFO - acc  = 0.924
2021-09-23 12:40:32,785 - INFO - time = 3.258
2021-09-23 12:40:36,040 - INFO - step = 3, energy = 4.569, err = 0.173
2021-09-23 12:40:36,040 - INFO - step = 3, energy_jf = 4.691, err = 0.203
2021-09-23 12:40:36,040 - INFO - acc  = 0.919
2021-09-23 12:40:36,040 - INFO - time = 3.249
2021-09-23 12:40:38,971 - INFO - step = 4, energy = 3.994, err = 0.316
2021-09-23 12:40:38,971 - INFO - step = 4, energy_jf = 3.673, err = 0.366
2021-09-23 12:40:38,971 - INFO - acc  = 0.919
2021-09-23 12:40:38,971 - INFO - time = 2.918

Do you see more beyond the warnings posted above?

[coreyjadams]

I went ahead and cleaned up those warnings and pushed an update. So, on my laptop, this runs without any warnings now. I am on mac OSX, using python from macports and tensorflow from pip. But I've also run this on: Nvidia A100, V100, intel skylake, AMD Rome, on a mix of ubuntu, SUSE, and RHEL.

Are you using a job scheduler perhaps? Maybe it's splitting stderr and stdout?