Error when importing data via load_whas command

[femkejanssen]

Describe the bug

When wanting to import one of DeepSurv's original datasets, whas.h5, the command raises an error and cannot find the data in the corresponding folder.

Steps/code to reproduce

Code for execution:

X_train, Y_train, E_train, = load_whas(partition='training', data_type='np')
X_test, Y_test, E_test = load_whas(partition='testing', data_type='np') 

When manually pasting the data in this folder (because it was empty), the command works.

Expected results

Loading the right data.

Actual results

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-3-ad08096f6a64> in <module>
----> 1 X_train, Y_train, E_train, = load_whas(partition='training', data_type='np')
      2 X_test, Y_test, E_test = load_whas(partition='testing', data_type='np')

~\AppData\Roaming\Python\Python37\site-packages\deepsurvk\datasets\__init__.py in load_whas(partition, **kwargs)
    526     .. [#hosmer] Hosmer Jr, David W., Stanley Lemeshow, and Susanne May. Applied survival analysis: regression modeling of time-to-event data. Vol. 618. John Wiley & Sons, 2011.
    527     """
--> 528     return _load_dataset('whas.h5', partition=partition, **kwargs)

~\AppData\Roaming\Python\Python37\site-packages\deepsurvk\datasets\__init__.py in _load_dataset(filename, partition, data_type, **kwargs)
     51 
     52     # Read training data.
---> 53     with h5py.File(filename_, 'r') as f:
     54         X_train = f['train']['x'][()]
     55         E_train = f['train']['e'][()]

C:\ProgramData\Anaconda3\envs\PCa_project\lib\site-packages\h5py\_hl\files.py in __init__(self, name, mode, driver, libver, userblock_size, swmr, rdcc_nslots, rdcc_nbytes, rdcc_w0, track_order, fs_strategy, fs_persist, fs_threshold, **kwds)
    443                                fapl, fcpl=make_fcpl(track_order=track_order, fs_strategy=fs_strategy,
    444                                fs_persist=fs_persist, fs_threshold=fs_threshold),
--> 445                                swmr=swmr)
    446 
    447             if isinstance(libver, tuple):

C:\ProgramData\Anaconda3\envs\PCa_project\lib\site-packages\h5py\_hl\files.py in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
    193         if swmr and swmr_support:
    194             flags |= h5f.ACC_SWMR_READ
--> 195         fid = h5f.open(name, flags, fapl=fapl)
    196     elif mode == 'r+':
    197         fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)

h5py\_objects.pyx in h5py._objects.with_phil.wrapper()

h5py\_objects.pyx in h5py._objects.with_phil.wrapper()

h5py\h5f.pyx in h5py.h5f.open()

FileNotFoundError: [Errno 2] Unable to open file (unable to open file: name = 'C:\Users\fja2102.54813\AppData\Roaming\Python\Python37\site-packages\deepsurvk\datasets\data\whas.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

Versions

• DeepSurvK version: 0.2.0
• Python version: 3.7
• Operating System: Windows 10 Enterprise

[arturomoncadatorres]

This was probably caused because the data files were not explicitly added. In commit 21379a9, they are now included in MANIFEST.in. This should solve the problem.

[arturomoncadatorres]

After some testing, I discovered that said commit didn't fix it.

While I come up with a permanent solution, I recommend copying the data directory found here into DeepSurvK's datasets directory, found in its installation path (usually something like C:\ProgramData\Anaconda3\envs\dsk_test\Lib\site-packages\deepsurvk\datasets, or wherever you have your environment).