Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Assertion 'stopped_or_unexpected_exit()' failed to hold. #3741

Open
tysmith opened this issue Apr 29, 2024 · 3 comments
Open

Assertion 'stopped_or_unexpected_exit()' failed to hold. #3741

tysmith opened this issue Apr 29, 2024 · 3 comments

Comments

@tysmith
Copy link

tysmith commented Apr 29, 2024

Found with Firefox debug build m-c 20240428-d1651fe33156

https://firefox-ci-tc.services.mozilla.com/tasks/index/gecko.v2.mozilla-central.revision.d1651fe33156a2aab5884ab1eaaaa8c5dd64631d.firefox/linux64-debug

This was triggered during recording using rr commit 2bb38b9.

[FATAL src/Task.cc:1117:regs()] 
 (task 10643 (rec:10643) at time 122921)
 -> Assertion `stopped_or_unexpected_exit()' failed to hold. 
Tail of trace dump:
{
  real_time:11973.908743 global_time:122901, event:`SYSCALL: futex' (state:ENTERING_SYSCALL) tid:10570, ticks:6372253
rax:0xffffffffffffffda rbx:0x70d20e1fffa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x189 rdi:0x70d21b1292c4 rbp:0x70d21b129298 rsp:0x70d20e1ffdf0 r8:0x0 r9:0xffffffff r10:0x0 r11:0x246 r12:0x0 r13:0x0 r14:0x3f r15:0x70d21b1292c4 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x70d215924640 gs_base:0x0
}
{
  real_time:11973.908746 global_time:122902, event:`SYSCALLBUF_RESET' tid:10570, ticks:6372253
}
{
  real_time:11973.908766 global_time:122903, event:`SYSCALL: futex' (state:EXITING_SYSCALL) tid:10320, ticks:1026185
rax:0x0 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x2 rsi:0x80 rdi:0x70d245d00020 rbp:0x70d2123f97a0 rsp:0x70d215220df0 r8:0x0 r9:0x59 r10:0x0 r11:0x246 r12:0x70d245d001c0 r13:0xf0 r14:0x70d245d00000 r15:0x50 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xca fs_base:0x70d2123fc640 gs_base:0x0
}
{
  real_time:11973.909351 global_time:122904, event:`SYSCALLBUF_FLUSH' tid:10320, ticks:1388141
  { syscall:'futex', ret:0x0, size:0x14 }
  { syscall:'read', ret:0xfe8, size:0xff8, desched:1 }
  { syscall:'read', ret:0xab3, size:0xac3, desched:1 }
  { syscall:'read', ret:0x0, size:0x10, desched:1 }
  { syscall:'close', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
  { syscall:'madvise', ret:0x0, size:0x10 }
}
{
  real_time:11973.909401 global_time:122905, event:`PATCH_SYSCALL' tid:10320, ticks:1388141
rax:0x65 rbx:0x70d2123fa380 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2991 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d2123fa100 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89400 r13:0xffffffffffffffe5 r14:0x2991 r15:0x70d2123fa380 rip:0x70d245f1bfec eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x70d2123fc640 gs_base:0x0
  { tid:10320, addr:0x70d24653cdf4, length:0x5e }
  { tid:10320, addr:0x70d245f1bfec, length:0x8 }
}
{
  real_time:11973.909405 global_time:122906, event:`SYSCALLBUF_RESET' tid:10320, ticks:1388141
}
{
  real_time:11973.909479 global_time:122907, event:`SYSCALL: ptrace' (state:ENTERING_SYSCALL) tid:10320, ticks:1388151
rax:0xffffffffffffffda rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2991 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d215220df0 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89400 r13:0xffffffffffffffe5 r14:0x2991 r15:0x70d2123fa380 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0
}
{
  real_time:11973.909500 global_time:122908, event:`SYSCALL: ptrace' (state:EXITING_SYSCALL) tid:10320, ticks:1388151
rax:0x0 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2991 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d215220df0 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89400 r13:0xffffffffffffffe5 r14:0x2991 r15:0x70d2123fa380 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0
}
{
  real_time:11973.909543 global_time:122909, event:`SIGNAL: SIGCHLD(async)' tid:10320, ticks:1388151
rax:0x0 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2991 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d215220df0 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89400 r13:0xffffffffffffffe5 r14:0x2991 r15:0x70d2123fa380 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0 st0:0x0 st1:0x0 st2:0x0 st3:0x0 st4:0x0 st5:0x0 st6:0x403ade0b6b3a76400012 st7:0x403ade0b6b3a76400000 ymm0:0x1b000070d201d89400 ymm1:0x70d2180160000000000000000400 ymm2:0x70d203a97900000000000000030f ymm3:0x7000000029910000000000001000 ymm4:0x0 ymm5:0xd01dee63164c6d9a ymm6:0x0 ymm7:0xa000070d20387a269 ymm8:0xe5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5 ymm9:0x0 ymm10:0x0 ymm11:0x0 ymm12:0x0 ymm13:0xb32e8000 ymm14:0x3f800001 ymm15:0xb1000000
}
{
  real_time:11973.909577 global_time:122910, event:`SIGNAL_DELIVERY: SIGCHLD(async)' tid:10320, ticks:1388151
rax:0x0 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2991 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d215220df0 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89400 r13:0xffffffffffffffe5 r14:0x2991 r15:0x70d2123fa380 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0
  { tid:10320, addr:0x70d215220df0, length:(nil) }
}
{
  real_time:11973.909621 global_time:122911, event:`SYSCALL: wait4' (state:ENTERING_SYSCALL) tid:10320, ticks:1388172
rax:0xffffffffffffffda rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x40000000 rsi:0x70d2123fa260 rdi:0x2991 rbp:0x70d2123fa370 rsp:0x70d215220df0 r8:0x0 r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89400 r13:0x70d23eb59e90 r14:0x2991 r15:0x70d2123fa260 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3d fs_base:0x70d2123fc640 gs_base:0x0
}
{
  real_time:11973.909648 global_time:122912, event:`SYSCALL: wait4' (state:EXITING_SYSCALL) tid:10320, ticks:1388172
rax:0x2991 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x40000000 rsi:0x70d2123fa260 rdi:0x2991 rbp:0x70d2123fa370 rsp:0x70d215220df0 r8:0x0 r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89400 r13:0x70d23eb59e90 r14:0x2991 r15:0x70d2123fa260 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3d fs_base:0x70d2123fc640 gs_base:0x0
  { tid:10320, addr:0x70d2123fa260, length:0x4 }
}
{
  real_time:11973.909682 global_time:122913, event:`SYSCALL: ptrace' (state:ENTERING_SYSCALL) tid:10320, ticks:1388196
rax:0xffffffffffffffda rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2991 rdi:0xc rbp:0x70d2123fa370 rsp:0x70d215220df0 r8:0xb r9:0x70d218016000 r10:0x70d2123fa188 r11:0x246 r12:0x70d201d89400 r13:0x70d23eb59e90 r14:0x2991 r15:0x70d2123fa260 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0
}
{
  real_time:11973.909703 global_time:122914, event:`SYSCALL: ptrace' (state:EXITING_SYSCALL) tid:10320, ticks:1388196
rax:0x0 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2991 rdi:0xc rbp:0x70d2123fa370 rsp:0x70d215220df0 r8:0xb r9:0x70d218016000 r10:0x70d2123fa188 r11:0x246 r12:0x70d201d89400 r13:0x70d23eb59e90 r14:0x2991 r15:0x70d2123fa260 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0
  { tid:10320, addr:0x70d2123fa188, length:0xd8 }
}
{
  real_time:11973.909756 global_time:122915, event:`SYSCALL: ptrace' (state:ENTERING_SYSCALL) tid:10320, ticks:1388222
rax:0xffffffffffffffda rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2993 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d215220df0 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89420 r13:0xffffffffffffffe5 r14:0x2993 r15:0x70d2123fa380 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0
}
{
  real_time:11973.909777 global_time:122916, event:`SYSCALL: ptrace' (state:EXITING_SYSCALL) tid:10320, ticks:1388222
rax:0x0 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2993 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d215220df0 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89420 r13:0xffffffffffffffe5 r14:0x2993 r15:0x70d2123fa380 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0
}
{
  real_time:11973.909811 global_time:122917, event:`SIGNAL: SIGCHLD(async)' tid:10320, ticks:1388222
rax:0x0 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2993 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d215220df0 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89420 r13:0xffffffffffffffe5 r14:0x2993 r15:0x70d2123fa380 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0 st0:0x0 st1:0x0 st2:0x0 st3:0x0 st4:0x0 st5:0x0 st6:0x403ade0b6b3a76400012 st7:0x403ade0b6b3a76400000 ymm0:0x70d210fe578f000070d210fe5790 ymm1:0x70d210f03701000070d210fe5830 ymm2:0x5 ymm3:0x680000000000000000000001 ymm4:0x0 ymm5:0x2b000070d210fe5780 ymm6:0x70d2089218900000000000000000 ymm7:0x1ffffffffffffffff ymm8:0xe5e5e5e5e5e5e5e5e5e5e5e5e5e5e5e5 ymm9:0x0 ymm10:0x0 ymm11:0x0 ymm12:0x0 ymm13:0xb32e8000 ymm14:0x3f800001 ymm15:0xb1000000
}
{
  real_time:11973.909838 global_time:122918, event:`SIGNAL_DELIVERY: SIGCHLD(async)' tid:10320, ticks:1388222
rax:0x0 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x0 rsi:0x2993 rdi:0x10 rbp:0x70d2123fa170 rsp:0x70d215220df0 r8:0xf r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89420 r13:0xffffffffffffffe5 r14:0x2993 r15:0x70d2123fa380 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x65 fs_base:0x70d2123fc640 gs_base:0x0
  { tid:10320, addr:0x70d215220df0, length:(nil) }
}
{
  real_time:11973.909874 global_time:122919, event:`SYSCALL: wait4' (state:ENTERING_SYSCALL) tid:10320, ticks:1388243
rax:0xffffffffffffffda rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x40000000 rsi:0x70d2123fa260 rdi:0x2993 rbp:0x70d2123fa370 rsp:0x70d215220df0 r8:0x0 r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89420 r13:0x70d23eb59e90 r14:0x2993 r15:0x70d2123fa260 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3d fs_base:0x70d2123fc640 gs_base:0x0
}
{
  real_time:11973.909898 global_time:122920, event:`SYSCALL: wait4' (state:EXITING_SYSCALL) tid:10320, ticks:1388243
rax:0x2993 rbx:0x70d215220fa0 rcx:0xffffffffffffffff rdx:0x40000000 rsi:0x70d2123fa260 rdi:0x2993 rbp:0x70d2123fa370 rsp:0x70d215220df0 r8:0x0 r9:0x70d218016000 r10:0x0 r11:0x246 r12:0x70d201d89420 r13:0x70d23eb59e90 r14:0x2993 r15:0x70d2123fa260 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x3d fs_base:0x70d2123fc640 gs_base:0x0
  { tid:10320, addr:0x70d2123fa260, length:0x4 }
}
[FATAL src/log.cc:469:start_emergency_debug()] (session doesn't look interactive, aborting emergency debugging)
@tysmith
Copy link
Author

tysmith commented May 10, 2024

I can reproduce this fairly consistently with a Firefox debug build loading http://saleepoch.com. --chaos might be required. It only seems to happen after a browser crash/assertion is triggered.

@rocallahan
Copy link
Collaborator

Sorry for the delay Tyson... can you still reproduce this? Can you pass -F to rr record, forcing it to stop, and get an rr stack trace?

@tysmith
Copy link
Author

tysmith commented May 17, 2024

Hey no problem :)

I am seeing this consistently when using our tools but when I try to reproduce it using just rr and Firefox I cannot. I've tried to isolate specific environment tweaks that trigger the issue but I've been unsuccessful.

I've logged #3748 to propose a way to help collect debugging issue in a manner more compatible with automation/non-interactive mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rocallahan @tysmith