Problem downloading ASF data

[Smitrgeo15]

Set these variables to None and you will be prompted to enter your username and password below.

asf = ASF(asf_username, asf_password)

Optimized scene downloading from ASF - only the required subswaths and polarizations.

print(asf.download_scenes(DATADIR, SCENES, SUBSWATH))

There are two ways to download orbits; you can use any one or both together.

try:
# RESORB orbit downloading from ASF has recently failed.
print(asf.download_orbits(DATADIR))
except Exception as e:
print (e)
# Download missed orbits in case ASF orbit downloading fails.
esa = ESA(esa_username, esa_password)
print (esa.download_orbits(DATADIR))
using this commands in PyGMTSAR i am unable to download the scenes

ERROR:
ASF Downloading Sentinel-1 SLC:  33%
 8/24 [00:12<03:10, 11.88s/it]

_RemoteTraceback Traceback (most recent call last)
_RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/remotezip.py", line 226, in fetch
res, range_header = self._request(kwargs)
File "/usr/local/lib/python3.10/dist-packages/remotezip.py", line 192, in _request
res.raise_for_status()
File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: NOT FOUND for url: https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SDV_20170324T011358_20170324T011425_015827_01A12B_940F%20%20%20%20.zip

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
r = call_item()
File "/usr/local/lib/python3.10/dist-packages/joblib/externals/loky/process_executor.py", line 291, in call
return self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.10/dist-packages/joblib/parallel.py", line 589, in call
return [func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/joblib/parallel.py", line 589, in
return [func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pygmtsar/ASF.py", line 105, in download_scene
with asf_search.remotezip(url, session) as remotezip:
File "/usr/local/lib/python3.10/dist-packages/asf_search/download/download.py", line 89, in remotezip
return RemoteZip(url, session=session)
File "/usr/local/lib/python3.10/dist-packages/remotezip.py", line 245, in init
super(RemoteZip, self).init(rio)
File "/usr/lib/python3.10/zipfile.py", line 1269, in init
self._RealGetContents()
File "/usr/lib/python3.10/zipfile.py", line 1332, in _RealGetContents
endrec = _EndRecData(fp)
File "/usr/lib/python3.10/zipfile.py", line 266, in _EndRecData
fpin.seek(0, 2)
File "/usr/local/lib/python3.10/dist-packages/remotezip.py", line 145, in seek
self.buffer = self._fetch_fun((-size, None), stream=False)
File "/usr/local/lib/python3.10/dist-packages/remotezip.py", line 230, in fetch
raise RemoteIOError(str(e))
remotezip.RemoteIOError: 404 Client Error: NOT FOUND for url: https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SDV_20170324T011358_20170324T011425_015827_01A12B_940F%20%20%20%20.zip
"""

The above exception was the direct cause of the following exception:

RemoteIOError Traceback (most recent call last)
in <cell line: 4>()
2 asf = ASF(asf_username, asf_password)
3 # Optimized scene downloading from ASF - only the required subswaths and polarizations.
----> 4 print(asf.download_scenes(DATADIR, SCENES, SUBSWATH))
5 # There are two ways to download orbits; you can use any one or both together.
6 try:

6 frames
/usr/local/lib/python3.10/dist-packages/joblib/parallel.py in _return_or_raise(self)
752 try:
753 if self.status == TASK_ERROR:
--> 754 raise self._result
755 return self._result
756 finally:

RemoteIOError: 404 Client Error: NOT FOUND for url: https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SDV_20170324T011358_20170324T011425_015827_01A12B_940F%20%20%20%20.zip
how to solve this problem

[AlexeyPechnikov]

The error message is clear:

NOT FOUND for url: https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SDV_20170324T011358_20170324T011425_015827_01A12B_940F%20%20%20%20.zip

The scene cannot be found because its identifier ends with a set of whitespaces ('...940F%20%20%20%20.zip'). You need to fix the scene names.

[Smitrgeo15]

How to correct the scene ID link ?
I have just copied the scene ID from ASF but i do not how to clear the whitespaces "NOT FOUND for url: https://datapool.asf.alaska.edu/SLC/SA/S1A_IW_SLC__1SDV_20170324T011358_20170324T011425_015827_01A12B_940F%20%20%20%20.zip"

[AlexeyPechnikov]

Check for any invisible whitespace characters (tiling) and remove them.

[y-okur]

Hi Alexey,

I have a problem with downloading ASF data.

I am trying to run one of your Google Colab Notebook: https://colab.research.google.com/drive/1TARVTB7z8goZyEVDRWyTAKJpyuqZxzW2?usp=sharing

At cell [16], I am receiving this error:

ASF Downloading Sentinel-1 SLC::  67%
 8/12 [00:18<01:14, 18.61s/it]

_RemoteTraceback Traceback (most recent call last)
_RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
r = call_item()
File "/usr/local/lib/python3.10/dist-packages/joblib/externals/loky/process_executor.py", line 291, in call
return self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.10/dist-packages/joblib/parallel.py", line 598, in call
return [func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/joblib/parallel.py", line 598, in
return [func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pygmtsar/ASF.py", line 148, in download_scene
assert os.path.getsize(fullname + '.tmp') == filesize,
AssertionError: ERROR: Downloaded incomplete scene content
"""

The above exception was the direct cause of the following exception:

AssertionError Traceback (most recent call last)
in <cell line: 4>()
2 asf = ASF(asf_username, asf_password)
3 # Optimized scene downloading from ASF - only the required subswaths and polarizations.
----> 4 print(asf.download_scenes(DATADIR, SCENES, SUBSWATH))

6 frames
/usr/local/lib/python3.10/dist-packages/joblib/parallel.py in _return_or_raise(self)
761 try:
762 if self.status == TASK_ERROR:
--> 763 raise self._result
764 return self._result
765 finally:

AssertionError: ERROR: Downloaded incomplete scene content

I am also receiving the same error when I try download different scenes with SUBSWATH=123 option.

Do you have any idea what might be the problem?

Thanks in advance

[AlexeyPechnikov]

The error is pretty clear:

ERROR: Downloaded incomplete scene content

You suffer from a network error or issue at the ASF datastore. Run the cell again or relaunch the entire notebook to continue the downloading.

[y-okur]

The error is pretty clear:

ERROR: Downloaded incomplete scene content

You suffer from a network error or issue at the ASF datastore. Run the cell again or relaunch the entire notebook to continue the downloading.

Hi Alexey,

Thank you for the quick response.

I tried downloading one swath at a time using:

print(asf.download_scenes(DATADIR, SCENES, 1))
print(asf.download_scenes(DATADIR, SCENES, 2))
print(asf.download_scenes(DATADIR, SCENES, 3))

and the problem is resolved now.

[AlexeyPechnikov]

Nice. FYI, this is not related to the number of swaths but only a temporary downloading error. PyGMTSAR example notebooks can produce such errors once or even for hours when an unpredictable network connection issue arises or the ASF datastore is in a maintenance state.