Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wrong qrexec-client exit code on failed service call #9618

Closed
marmarek opened this issue Dec 3, 2024 · 5 comments · Fixed by QubesOS/qubes-core-qrexec#183
Closed

Wrong qrexec-client exit code on failed service call #9618

marmarek opened this issue Dec 3, 2024 · 5 comments · Fixed by QubesOS/qubes-core-qrexec#183
Labels
affects-4.2 This issue affects Qubes OS 4.2. affects-4.3 This issue affects Qubes OS 4.3. C: core diagnosed Technical diagnosis has been performed (see issue comments). P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. pr submitted A pull request has been submitted for this issue. r4.3-host-cur-test r4.3-vm-bookworm-cur-test r4.3-vm-fc39-cur-test r4.3-vm-fc40-cur-test r4.3-vm-fc41-cur-test r4.3-vm-trixie-cur-test T: bug Type: bug report. A problem or defect resulting in unintended behavior in something that exists.

Comments

@marmarek
Copy link
Member

marmarek commented Dec 3, 2024

Observation

openQA test in scenario qubesos-4.3-pull-requests-x86_64-system_tests_usbproxy@64bit fails in
TC_20_USBProxy_core3_fedora-40-xfce

Traceback (most recent call last):
  File "/usr/lib/python3.13/site-packages/qubesusbproxy/core3ext.py", line 680, in on_device_attach_usb
    await vm.run_service_for_stdio(
    ...<5 lines>...
    )
  File "/usr/lib/python3.13/site-packages/qubes/vm/qubesvm.py", line 1760, in run_service_for_stdio
    raise subprocess.CalledProcessError(
        p.returncode, args[0], *stdouterr
    )
subprocess.CalledProcessError: Command 'qubes.USBAttach' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.13/contextlib.py", line 85, in inner
    return func(*args, **kwds)
  File "/usr/lib/python3.13/site-packages/qubesusbproxy/tests.py", line 507, in test_070_attach_not_installed_front
    self.loop.run_until_complete(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.frontend.devices["usb"].attach(ass)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 721, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/site-packages/qubes/devices.py", line 240, in attach
    await self._vm.fire_event_async(
    ...<4 lines>...
    )
  File "/usr/lib/python3.13/site-packages/qubes/events.py", line 243, in fire_event_async
    effect = task.result()
  File "/usr/lib/python3.13/site-packages/qubesusbproxy/core3ext.py", line 693, in on_device_attach_usb
    raise QubesUSBException(
    ...<2 lines>...
    )
qubesusbproxy.core3ext.QubesUSBException: Device attach failed:  2024-12-02 17:33:09.297 qrexec-client[85923]: process_io.c:39:handle_vchan_error: Error while vchan send(handle_input stdout), exiting_

Test suite description

Reproducible

Fails since (at least) Build 2024120220-4.3 (current job)

Expected result

Last good: 2024120211-4.3 (or more recent)

Further details

Always latest result in this scenario: latest

When remote service is not available, the exit code should be 127, not 1.
@marmarek marmarek added T: bug Type: bug report. A problem or defect resulting in unintended behavior in something that exists. C: core P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. affects-4.2 This issue affects Qubes OS 4.2. affects-4.3 This issue affects Qubes OS 4.3. labels Dec 3, 2024
marmarek added a commit to marmarek/qubes-core-qrexec that referenced this issue Dec 3, 2024
@marmarek
Try to retrieve exit code of a failed service call
54e7803 
When service call fails, the remote (service) side sends EOF + exit code
and closes the vchan immediately. If the client side tries to send
something, it will fail and exit immediately (with code 1). But reading
data that was queued in the vchan before its closing is still possible.
So, on send error check if there is anything interesting to receive
(especially exit code, but potentially also some service output) and, if
yes, don't exit immediately. Since the service exit code is sent last
(after all stdout/stderr data and their EOF), it's okay to check just
for remote_status.
This is relevant only to the service client side, as exit status of the
service-handling process is not relevant.

QubesOS/qubes-issues#9618
@marmarek marmarek mentioned this issue Dec 3, 2024
Merged
@andrewdavidwong andrewdavidwong added diagnosed Technical diagnosis has been performed (see issue comments). pr submitted A pull request has been submitted for this issue. labels Dec 3, 2024
marmarek added a commit to marmarek/qubes-core-qrexec that referenced this issue Dec 10, 2024
@marmarek
Add a test for qrexec-client race condition
55fa313 
Check if the service exit code is correctly retrieved even if the
the service terminates at the exact moment the qrexec-client tries to
send some data.
Try to win the race by initially sending SIGSTOP to the qrexec-client
process, and sending SIGCONT only after preparing both local and remote
data streams. qrexec-client will handle local data stream first, at
which point remote socket is already closed.

Similar issue applies to qrexec-client-vm, but since the implementation
is shared, one test is enough.

QubesOS/qubes-issues#9618
marmarek added a commit to marmarek/qubes-core-qrexec that referenced this issue Dec 10, 2024
@marmarek
Add a test for qrexec-client race condition
df25090 
Check if the service exit code is correctly retrieved even if the
the service terminates at the exact moment the qrexec-client tries to
send some data.
Try to win the race by initially sending SIGSTOP to the qrexec-client
process, and sending SIGCONT only after preparing both local and remote
data streams. qrexec-client will handle local data stream first, at
which point remote socket is already closed.

Similar issue applies to qrexec-client-vm, but since the implementation
is shared, one test is enough.

QubesOS/qubes-issues#9618
@qubesos-bot qubesos-bot mentioned this issue Dec 11, 2024
Open
@qubesos-bot
Copy link

Automated announcement from builder-github

The package core-qrexec has been pushed to the r4.3 testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing bookworm-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

@qubesos-bot
Copy link

Automated announcement from builder-github

The package core-qrexec has been pushed to the r4.3 testing repository for the Debian template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing trixie-testing (or appropriate equivalent for your template version), then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

@qubesos-bot
Copy link

Automated announcement from builder-github

The component core-qrexec (including package core-qrexec) has been pushed to the r4.3 testing repository for the Fedora template.
To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update

@qubesos-bot
Copy link

Automated announcement from builder-github

The component core-qrexec (including package core-qrexec) has been pushed to the r4.3 testing repository for the Fedora template.
To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update

@qubesos-bot
Copy link

Automated announcement from builder-github

The component core-qrexec (including package core-qrexec) has been pushed to the r4.3 testing repository for the Fedora template.
To test this update, please install it with the following command:

sudo dnf update --enablerepo=qubes-vm-r4.3-current-testing

Changes included in this update

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-4.2 This issue affects Qubes OS 4.2. affects-4.3 This issue affects Qubes OS 4.3. C: core diagnosed Technical diagnosis has been performed (see issue comments). P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. pr submitted A pull request has been submitted for this issue. r4.3-host-cur-test r4.3-vm-bookworm-cur-test r4.3-vm-fc39-cur-test r4.3-vm-fc40-cur-test r4.3-vm-fc41-cur-test r4.3-vm-trixie-cur-test T: bug Type: bug report. A problem or defect resulting in unintended behavior in something that exists.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Try to retrieve exit code of a failed service call marmarek/qubes-core-qrexec
3 participants
@marmarek @andrewdavidwong @qubesos-bot