Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apu2 iommu fix #592

Open
wants to merge 11 commits into
base: dasharo
Choose a base branch
from
Open

Apu2 iommu fix #592

wants to merge 11 commits into from

Conversation

miczyg1
Copy link
Contributor

@miczyg1 miczyg1 commented Nov 23, 2024

No description provided.

Move the GNB IOAPIC enabling to northbridge init where the IOAPIC is
initialized.

Remove the comment about IoapicSbFeatureEn, this bit is not touched
here.

Signed-off-by: Michał Żygowski <[email protected]>
@pietrushnic pietrushnic mentioned this pull request Nov 23, 2024
10 tasks
Introduce PCI_DEVID macro which allows to construct PCI device
addresses or range with bus numbers.

Signed-off-by: Michał Żygowski <[email protected]>
@miczyg1 miczyg1 force-pushed the apu2_iommu_fix branch 2 times, most recently from 2b58e23 to aed43dc Compare November 26, 2024 23:23
…e area

Use the IORESOURCE_SOFT_RESERVE attribute to reserve CC6 save state DRAM.
Using regualr reserved memory makes it hard on EDK2 to detect if it is MMIO
or reserved DRAM, as TOM2 is equal to the base of the CC6 save state area,
not its limit.

Signed-off-by: Michał Żygowski <[email protected]>
Per family 16h models 30h-3fh BKDG the IoapicSbFeatureEn must be
configured according to the interrupt routing mode selecte by OS.
If OS chose APIC mode, the IoapicSbFeatureEn must be cleared.
Otherwise, it must be set, meaning PIC mode is used.

Add a hook to _PIC method to call SoC/northbridge specific code to
set/clear the bit to configure GNB IOAPIC properly.

ACPI specification says that _PIC method is optional and can be
called by OSPM to provide the interrupt routing mode information to
the firmware. However, if the method is not called, the firmware
must assume PIC mode is used. AGESA sets the IoapicSbFeatureEn
already to be compliant with ACPI. Previously, coreboot cleared the
bit unconditionally and left a comment to move that part to DSDT.
The hook allows to clear the IoapicSbFeatureEn bit if OS chooses APIC
mode for interrupt routing.

Signed-off-by: Michał Żygowski <[email protected]>
@SergiiDmytruk
Copy link
Member

SergiiDmytruk commented Dec 5, 2024

I get a panic from Xen when booting with coreboot+EDK2 that mentions IOMMU and thought that this PR might fix it as well, but it doesn't. Any idea if this is even related? Also wondering whether @krystian-hebel has seen this panic.

Xen boot log

(XEN) Xen version 4.20-unstable (@) (gcc (Debian 12.2.0-14) 12.2.0) debug=y Mon Nov 18 18:13:43 UTC 2024
(XEN) Latest ChangeSet: Sun Mar 17 00:58:26 2024 +0200 git:b9d7cd4cfe-dirty
(XEN) build-id: ded12aa3eca5672e8673993a9d30a975fe8b337e
(XEN) Bootloader: GRUB 2.13
(XEN) Command line: placeholder console=tty0 console=ttyS0,115200 dom0_mem=min:512M dom0_mem=max:4096M ucode=scan smt=off gnttf
(XEN) Xen image load base address: 0xcda00000
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 0 EDD information structures
(XEN) CPU Vendor: AMD, Family 22 (0x16), Model 48 (0x30), Stepping 1 (raw 00730f01)
(XEN) EFI RAM map:
(XEN)  [0000000000000000, 0000000000000fff] (reserved)
(XEN)  [0000000000001000, 000000000009ffff] (usable)
(XEN)  [00000000000a0000, 00000000000fffff] (reserved)
(XEN)  [0000000000100000, 00000000ce2c7fff] (usable)
(XEN)  [00000000ce2c8000, 00000000ce2cafff] (reserved)
(XEN)  [00000000ce2cb000, 00000000ce2d0fff] (usable)
(XEN)  [00000000ce2d1000, 00000000ce38dfff] (reserved)
(XEN)  [00000000ce38e000, 00000000ce933fff] (usable)
(XEN)  [00000000ce934000, 00000000ce97dfff] (reserved)
(XEN)  [00000000ce97e000, 00000000ceb1ffff] (usable)
(XEN)  [00000000ceb20000, 00000000ceb2ffff] (ACPI NVS)
(XEN)  [00000000ceb30000, 00000000ceb49fff] (usable)
(XEN)  [00000000ceb4a000, 00000000ceb59fff] (ACPI NVS)
(XEN)  [00000000ceb5a000, 00000000ceb9bfff] (usable)
(XEN)  [00000000ceb9c000, 00000000ceba3fff] (ACPI NVS)
(XEN)  [00000000ceba4000, 00000000cebccfff] (usable)
(XEN)  [00000000cebcd000, 00000000cebd1fff] (ACPI data)
(XEN)  [00000000cebd2000, 00000000cec99fff] (usable)
(XEN)  [00000000cec9a000, 00000000cec9cfff] (ACPI data)
(XEN)  [00000000cec9d000, 00000000ced5afff] (usable)
(XEN)  [00000000ced5b000, 00000000ced73fff] (reserved)
(XEN)  [00000000ced74000, 00000000ced98fff] (usable)
(XEN)  [00000000ced99000, 00000000ced9afff] (ACPI data)
(XEN)  [00000000ced9b000, 00000000ced9dfff] (usable)
(XEN)  [00000000ced9e000, 00000000ced9efff] (ACPI data)
(XEN)  [00000000ced9f000, 00000000cf280fff] (usable)
(XEN)  [00000000cf281000, 00000000cf284fff] (reserved)
(XEN)  [00000000cf285000, 00000000cf697fff] (usable)
(XEN)  [00000000cf698000, 00000000cf698fff] (ACPI data)
(XEN)  [00000000cf699000, 00000000cf6adfff] (usable)
(XEN)  [00000000cf6ae000, 00000000cf7f1fff] (reserved)
(XEN)  [00000000cf7f2000, 00000000cf7f5fff] (ACPI NVS)
(XEN)  [00000000cf7f6000, 00000000cf7fdfff] (ACPI data)
(XEN)  [00000000cf7fe000, 00000000cfc4efff] (usable)
(XEN)  [00000000cfc4f000, 00000000cfffffff] (reserved)
(XEN)  [00000000ff800000, 00000000ff83ffff] (reserved)
(XEN)  [0000000100000000, 000000012effffff] (usable)
(XEN)  [000000012f000000, 000000012fffffff] (reserved)
(XEN) BSP microcode revision: 0x07030105
(XEN) Microcode: AuthenticAMD.bin not found during CPIO scan
(XEN) ACPI: RSDP CF7FD014, 0024 (r2 COREv4)
(XEN) ACPI: XSDT CF7FC0E8, 0084 (r1 COREv4 COREBOOT        0       1000013)
(XEN) ACPI: FACP CF7F9000, 0114 (r6 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: DSDT CF7FA000, 17A7 (r2 COREv4 COREBOOT    10001 INTL 20230628)
(XEN) ACPI: FACS CF7F5000, 0040
(XEN) ACPI: SSDT CF7F8000, 064E (r2 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: MCFG CF7F7000, 003C (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: TPM2 CF7F6000, 004C (r4 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: APIC CED9E000, 007E (r3 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: SPCR CED9A000, 0058 (r4 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: HEST CED99000, 01D0 (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: IVRS CEC9C000, 00D0 (r2 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: SSDT CEBCD000, 48A6 (r2 AMD    AGESA           2 MSFT  4000000)
(XEN) ACPI: SSDT CEC9A000, 07C8 (r1 AMD    AGESA           1 AMD         1)
(XEN) ACPI: HPET CEC9B000, 0038 (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: FPDT CF698000, 0034 (r1 INTEL  EDK2            2       1000013)
(XEN) System RAM: 4073MB (4171020kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000012f000000
(XEN) Domain heap initialised
(XEN) SMBIOS 3.3 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x818 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0]
(XEN) ACPI:             wakeup_vec[cf7f500c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 33, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec20000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 1, version 33, address 0xfec20000, GSI 24-55
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) ACPI: HPET id: 0x10228201 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
(XEN) HEST: Table parsing has been initialized
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 56 GSI, 776 MSI/MSI-X
(XEN) CPU0: 1000 (600 ... 1400) MHz
(XEN) xstate: size: 0x340 and states: 0x7
(XEN) CPU0: AMD Fam16h machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware hints:
(XEN)   Hardware features:
(XEN)   Compiled-in support: INDIRECT_THUNK SHADOW_PAGING HARDEN_ARRAY HARDEN_BRANCH HARDEN_GUEST_ACCESS HARDEN_LOCK
(XEN)   Xen settings: BTI-Thunk: RETPOLINE, SPEC_CTRL: No, Other: BRANCH_HARDEN
(XEN)   Support for HVM VMs: MSR_VIRT_SPEC_CTRL RSB
(XEN)   Support for PV VMs: RSB
(XEN)   XPTI (64-bit PV only): Dom0 disabled, DomU disabled (without PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU disabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 998.131 MHz processor.
(XEN) Freed 1016kB unused BSS memory
(XEN) EFI memory map:
(XEN)  0000000001000-0000000004fff type=2 attr=000000000000000f
(XEN)  0000000005000-0000000086fff type=7 attr=000000000000000f
(XEN)  0000000087000-0000000087fff type=4 attr=000000000000000f
(XEN)  0000000088000-000000008ffff type=7 attr=000000000000000f
(XEN)  0000000090000-000000009ffff type=2 attr=000000000000000f
(XEN)  0000000100000-00000007fffff type=7 attr=000000000000000f
(XEN)  0000000800000-00000015fffff type=4 attr=000000000000000f
(XEN)  0000001600000-00000028c1fff type=2 attr=000000000000000f
(XEN)  00000028c2000-00000c9c3ffff type=7 attr=000000000000000f
(XEN)  00000c9c40000-00000cbc3ffff type=1 attr=000000000000000f
(XEN)  00000cbc40000-00000cbc5ffff type=4 attr=000000000000000f
(XEN)  00000cbc60000-00000cdbfffff type=7 attr=000000000000000f
(XEN)  00000cdc00000-00000cdffefff type=2 attr=000000000000000f
(XEN)  00000cdfff000-00000ce13afff type=7 attr=000000000000000f
(XEN)  00000ce13b000-00000ce200fff type=1 attr=000000000000000f
(XEN)  00000ce201000-00000ce2c7fff type=7 attr=000000000000000f
(XEN)  00000ce2c8000-00000ce2cafff type=6 attr=800000000000000f
(XEN)  00000ce2cb000-00000ce2cdfff type=7 attr=000000000000000f
(XEN)  00000ce2ce000-00000ce2cefff type=2 attr=000000000000000f
(XEN)  00000ce2cf000-00000ce2d0fff type=4 attr=000000000000000f
(XEN)  00000ce2d1000-00000ce38dfff type=6 attr=800000000000000f
(XEN)  00000ce38e000-00000ce933fff type=4 attr=000000000000000f
(XEN)  00000ce934000-00000ce97dfff type=0 attr=000000000000000f
(XEN)  00000ce97e000-00000ce9b1fff type=4 attr=000000000000000f
(XEN)  00000ce9b2000-00000ce9cafff type=3 attr=000000000000000f
(XEN)  00000ce9cb000-00000ce9d8fff type=4 attr=000000000000000f
(XEN)  00000ce9d9000-00000ce9f1fff type=3 attr=000000000000000f
(XEN)  00000ce9f2000-00000cea12fff type=4 attr=000000000000000f
(XEN)  00000cea13000-00000cea1bfff type=3 attr=000000000000000f
(XEN)  00000cea1c000-00000cea20fff type=4 attr=000000000000000f
(XEN)  00000cea21000-00000cea2cfff type=3 attr=000000000000000f
(XEN)  00000cea2d000-00000cea2dfff type=4 attr=000000000000000f
(XEN)  00000cea2e000-00000ceaacfff type=3 attr=000000000000000f
(XEN)  00000ceaad000-00000ceaaffff type=4 attr=000000000000000f
(XEN)  00000ceab0000-00000ceacafff type=3 attr=000000000000000f
(XEN)  00000ceacb000-00000ceacefff type=4 attr=000000000000000f
(XEN)  00000ceacf000-00000ceaf0fff type=3 attr=000000000000000f
(XEN)  00000ceaf1000-00000ceaf1fff type=4 attr=000000000000000f
(XEN)  00000ceaf2000-00000ceaf7fff type=3 attr=000000000000000f
(XEN)  00000ceaf8000-00000ceaf8fff type=4 attr=000000000000000f
(XEN)  00000ceaf9000-00000ceafbfff type=3 attr=000000000000000f
(XEN)  00000ceafc000-00000ceafdfff type=4 attr=000000000000000f
(XEN)  00000ceafe000-00000ceb19fff type=3 attr=000000000000000f
(XEN)  00000ceb1a000-00000ceb1cfff type=4 attr=000000000000000f
(XEN)  00000ceb1d000-00000ceb1ffff type=3 attr=000000000000000f
(XEN)  00000ceb20000-00000ceb2ffff type=10 attr=000000000000000f
(XEN)  00000ceb30000-00000ceb44fff type=3 attr=000000000000000f
(XEN)  00000ceb45000-00000ceb46fff type=4 attr=000000000000000f
(XEN)  00000ceb47000-00000ceb49fff type=3 attr=000000000000000f
(XEN)  00000ceb4a000-00000ceb59fff type=10 attr=000000000000000f
(XEN)  00000ceb5a000-00000ceb7ffff type=3 attr=000000000000000f
(XEN)  00000ceb80000-00000ceb83fff type=4 attr=000000000000000f
(XEN)  00000ceb84000-00000ceb97fff type=3 attr=000000000000000f
(XEN)  00000ceb98000-00000ceb9bfff type=4 attr=000000000000000f
(XEN)  00000ceb9c000-00000ceba3fff type=10 attr=000000000000000f
(XEN)  00000ceba4000-00000cebbbfff type=3 attr=000000000000000f
(XEN)  00000cebbc000-00000cebbcfff type=4 attr=000000000000000f
(XEN)  00000cebbd000-00000cebbefff type=3 attr=000000000000000f
(XEN)  00000cebbf000-00000cebccfff type=4 attr=000000000000000f
(XEN)  00000cebcd000-00000cebd1fff type=9 attr=000000000000000f
(XEN)  00000cebd2000-00000cebf5fff type=3 attr=000000000000000f
(XEN)  00000cebf6000-00000cebfcfff type=4 attr=000000000000000f
(XEN)  00000cebfd000-00000cec99fff type=3 attr=000000000000000f
(XEN)  00000cec9a000-00000cec9cfff type=9 attr=000000000000000f
(XEN)  00000cec9d000-00000ced24fff type=4 attr=000000000000000f
(XEN)  00000ced25000-00000ced5afff type=3 attr=000000000000000f
(XEN)  00000ced5b000-00000ced73fff type=6 attr=800000000000000f
(XEN)  00000ced74000-00000ced98fff type=3 attr=000000000000000f
(XEN)  00000ced99000-00000ced9afff type=9 attr=000000000000000f
(XEN)  00000ced9b000-00000ced9dfff type=3 attr=000000000000000f
(XEN)  00000ced9e000-00000ced9efff type=9 attr=000000000000000f
(XEN)  00000ced9f000-00000ceda7fff type=4 attr=000000000000000f
(XEN)  00000ceda8000-00000ceda9fff type=3 attr=000000000000000f
(XEN)  00000cedaa000-00000cedbdfff type=4 attr=000000000000000f
(XEN)  00000cedbe000-00000cedf1fff type=3 attr=000000000000000f
(XEN)  00000cedf2000-00000cf207fff type=4 attr=000000000000000f
(XEN)  00000cf208000-00000cf20bfff type=3 attr=000000000000000f
(XEN)  00000cf20c000-00000cf20ffff type=4 attr=000000000000000f
(XEN)  00000cf210000-00000cf215fff type=3 attr=000000000000000f
(XEN)  00000cf216000-00000cf218fff type=4 attr=000000000000000f
(XEN)  00000cf219000-00000cf227fff type=3 attr=000000000000000f
(XEN)  00000cf228000-00000cf22cfff type=4 attr=000000000000000f
(XEN)  00000cf22d000-00000cf268fff type=3 attr=000000000000000f
(XEN)  00000cf269000-00000cf269fff type=4 attr=000000000000000f
(XEN)  00000cf26a000-00000cf26efff type=3 attr=000000000000000f
(XEN)  00000cf26f000-00000cf273fff type=4 attr=000000000000000f
(XEN)  00000cf274000-00000cf275fff type=3 attr=000000000000000f
(XEN)  00000cf276000-00000cf277fff type=4 attr=000000000000000f
(XEN)  00000cf278000-00000cf279fff type=3 attr=000000000000000f
(XEN)  00000cf27a000-00000cf27cfff type=4 attr=000000000000000f
(XEN)  00000cf27d000-00000cf27efff type=3 attr=000000000000000f
(XEN)  00000cf27f000-00000cf280fff type=4 attr=000000000000000f
(XEN)  00000cf281000-00000cf284fff type=0 attr=000000000000000f
(XEN)  00000cf285000-00000cf285fff type=3 attr=000000000000000f
(XEN)  00000cf286000-00000cf286fff type=4 attr=000000000000000f
(XEN)  00000cf287000-00000cf29efff type=3 attr=000000000000000f
(XEN)  00000cf29f000-00000cf697fff type=4 attr=000000000000000f
(XEN)  00000cf698000-00000cf698fff type=9 attr=000000000000000f
(XEN)  00000cf699000-00000cf69efff type=4 attr=000000000000000f
(XEN)  00000cf69f000-00000cf6a2fff type=3 attr=000000000000000f
(XEN)  00000cf6a3000-00000cf6adfff type=4 attr=000000000000000f
(XEN)  00000cf6ae000-00000cf72dfff type=5 attr=800000000000000f
(XEN)  00000cf72e000-00000cf7edfff type=6 attr=800000000000000f
(XEN)  00000cf7ee000-00000cf7f1fff type=0 attr=000000000000000f
(XEN)  00000cf7f2000-00000cf7f5fff type=10 attr=000000000000000f
(XEN)  00000cf7f6000-00000cf7fdfff type=9 attr=000000000000000f
(XEN)  00000cf7fe000-00000cf9fffff type=4 attr=000000000000000f
(XEN)  00000cfa00000-00000cfb8ffff type=7 attr=000000000000000f
(XEN)  00000cfb90000-00000cfbaffff type=4 attr=000000000000000f
(XEN)  00000cfbb0000-00000cfc02fff type=3 attr=000000000000000f
(XEN)  00000cfc03000-00000cfc03fff type=4 attr=000000000000000f
(XEN)  00000cfc04000-00000cfc3ffff type=3 attr=000000000000000f
(XEN)  00000cfc40000-00000cfc4efff type=7 attr=000000000000000f
(XEN)  0000100000000-000012effffff type=7 attr=000000000000000f
(XEN)  0000000000000-0000000000fff type=0 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=000000000000000f
(XEN)  00000cfc4f000-00000cfc83fff type=0 attr=000000000000000f
(XEN)  00000cfc84000-00000cfc93fff type=0 attr=800000000000000f
(XEN)  00000cfc94000-00000cfffffff type=0 attr=000000000000000f
(XEN)  00000ff800000-00000ff83ffff type=11 attr=8000000000000001
(XEN)  000012f000000-000012fffffff type=0 attr=000000000000000f
(XEN) alt table ffff82d0404a41d8 -> ffff82d0404b6c9a
(XEN) AMD-Vi: IOMMU Extended Features:
(XEN) - Peripheral Page Service Request
(XEN) - Invalidate All Command
(XEN) - Guest APIC
(XEN) - Performance Counters
(XEN) - Host Address Translation Size: 0x2
(XEN) - Guest Address Translation Size: 0
(XEN) - Guest CR3 Root Table Level: 0
(XEN) - Maximum PASID: 0x8
(XEN) - SMI Filter Register: 0x1
(XEN) - SMI Filter Register Count: 0x2
(XEN) - Guest Virtual APIC Modes: 0
(XEN) - Dual PPR Log: 0
(XEN) - Dual Event Log: 0
(XEN) - Device Table Segmentation: 0
(XEN) - Memory Access Routing and Control: 0
(XEN) - VM Table Size: 0
(XEN) AMD-Vi: Disabled HAP memory map sharing with IOMMU
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabling APIC mode.  Using 2 I/O APICs
(XEN) ----[ Xen-4.20-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d040327dbc>] __ioapic_write_entry+0x83/0x95
(XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx: 0000000000010000
(XEN) rdx: 0000000000006000   rsi: 0000000000000000   rdi: 0000000000000001
(XEN) rbp: ffff82d040467c58   rsp: ffff82d040467c48   r8:  0000000000000010
(XEN) r9:  ffff82cffffff000   r10: ffff82d0404bef00   r11: 0000000000000001
(XEN) r12: 0000000000000011   r13: ffff830127800058   r14: 0000000000000000
(XEN) r15: 0000000000000001   cr0: 000000008005003b   cr4: 00000000000406e0
(XEN) cr3: 00000000cdeca000   cr2: 0000000000000000
(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d040327dbc> (__ioapic_write_entry+0x83/0x95):
(XEN)  89 ff 89 f6 48 8b 04 f8 <88> 0c 30 5b 41 5c 5d c3 48 89 ca e8 1e d7 f4 ff
(XEN) Xen stack trace from rsp=ffff82d040467c48:
(XEN)    ffff830127b06000 ffff830128926600 ffff82d040467cc0 ffff82d04027459e
(XEN)    0000000000010000 0000000000000000 0000000000000246 0000000000000000
(XEN)    0000000000010000 0000000000000001 0000000000000000 0000000000000000
(XEN)    ffff82d0404be484 ffff82d0404be484 ffff82d0404be480 ffff82d040467cd0
(XEN)    ffff82d0402754f4 ffff82d040467cf0 ffff82d040327dcc 0000000000000001
(XEN)    0000000000000000 ffff82d040467d18 ffff82d040328137 0000000000010000
(XEN)    0000000000000000 0000000000000001 ffff82d040467d48 ffff82d0403281f6
(XEN)    00000000ffffffff 00000000ffffffff ffff82d0404cd880 0000000000000000
(XEN)    ffff82d040467d98 ffff82d040447bce ffff82d0404be480 0000000000000000
(XEN)    0000000000000000 ffff82d0405e73a0 ffff82d0405e69c0 ffff82d0404cd880
(XEN)    ffff82d0404bfa20 0000000000000001 ffff82d040467db0 ffff82d0404503bd
(XEN)    ffff82d0405e69c0 ffff82d040467ee8 ffff82d04044f592 ffff82d0404a2f80
(XEN)    ffff82d0404a2f80 000000012a918000 0000000001000000 ffff82d0405f7e80
(XEN)    00000000046e7400 ffff82d04049df60 0000000000100000 ffff82d000800163
(XEN)    00000000000001ff 000000000012f000 ffff82d04049e2a8 00000000000046e8
(XEN)    0000000300000000 ffff82d0404a2f80 ffff830128928000 ffff82d040467ef8
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000800000000 000000010000006e 0000000000000003
(XEN)    00000000000002f8 ffff82d0405de000 ffff82d0404e0000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff82d040327dbc>] R __ioapic_write_entry+0x83/0x95
(XEN)    [<ffff82d04027459e>] F amd_iommu_ioapic_update_ire+0x1ea/0x273
(XEN)    [<ffff82d0402754f4>] F iommu_update_ire_from_apic+0xa/0xc
(XEN)    [<ffff82d040327dcc>] F __ioapic_write_entry+0x93/0x95
(XEN)    [<ffff82d040328137>] F arch/x86/io_apic.c#clear_IO_APIC_pin+0x7c/0x10e
(XEN)    [<ffff82d0403281f6>] F arch/x86/io_apic.c#clear_IO_APIC+0x2d/0x61
(XEN)    [<ffff82d040447bce>] F enable_IO_APIC+0x2e3/0x34f
(XEN)    [<ffff82d0404503bd>] F smp_prepare_cpus+0x293/0x2b9
(XEN)    [<ffff82d04044f592>] F __start_xen+0x1d9b/0x24cf
(XEN)    [<ffff82d0402033f5>] F __high_start+0xd5/0xe0
(XEN)
(XEN) Pagetable walk from 0000000000000000:
(XEN)  L4[0x000] = 000000012891b063 ffffffffffffffff
(XEN)  L3[0x000] = 000000012891a063 ffffffffffffffff
(XEN)  L2[0x000] = 0000000128919063 ffffffffffffffff
(XEN)  L1[0x000] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0002]
(XEN) Faulting linear address: 0000000000000000
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.

P.S. This is MultiBoot2 boot but EFI boot behaves the same at least when booting from GRUB.

@SergiiDmytruk
Copy link
Member

I get a panic from Xen when booting with coreboot+EDK2 that mentions IOMMU and thought that this PR might fix it as well, but it doesn't. Any idea if this is even related?

Looks like it's not. I don't get the error on aem_intel_fixes_v0.4.1, the output above is from unstable staging.

@miczyg1
Copy link
Contributor Author

miczyg1 commented Dec 16, 2024

I get a panic from Xen when booting with coreboot+EDK2 that mentions IOMMU and thought that this PR might fix it as well, but it doesn't. Any idea if this is even related?

Looks like it's not. I don't get the error on aem_intel_fixes_v0.4.1, the output above is from unstable staging.

It may be because of these lines being removed:
https://github.com/Dasharo/coreboot/pull/592/files#diff-6561ea6e72079a03a85582ba94798c7902833a11ee14cf47d8a78718c0a3418eL519-L522

The IoapicSbFeatureEn controls the routing of masked GNB IOAPIC interrupts to SB IOAPIC. When the software intends to use PIC mode for interrupt routing, this bit should remain set. When IOAPIC mode is to be used, then it should be cleared. ACPI specification requires the pre-OS environment (i.e. firmware) to assume PIC mode as default if _PIC method is not implemented. So I moved the IoapicSbFeatureEn to the ACPI NAPE function which is called in _PIC function, so that te bit will always be set to the value required by selected interrupt routing mode.

I have talked with @andyhhp about it and he said that Xen does not talk ACPI and simply assumes IOAPIC mode always. This is not compliant with the ACPI specification, however he suggested to make it compliant with the spec. That of course probably causes the panic you see @SergiiDmytruk . If all Xen versions behave like that, then we will have to make it an runtime option? maybe @andyhhp can say more.

@SergiiDmytruk
Copy link
Member

@miczyg1, you didn't break anything! The issue happens without this PR as well, I just thought the PR could fix it. The problem must be somewhere in Xen staging because https://github.com/TrenchBoot/xen/tree/aem_intel_fixes_v0.4.1 boots fine in EFI mode.

@andyhhp
Copy link

andyhhp commented Dec 16, 2024

This is Xen falling over a NULL pointer while configuring an IO-APIC pin.

@SergiiDmytruk Can you try reverting https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=86001b3970fea4536048607ea6e12541736c48e1 ? or at least put in a NULL pointer check for io_apic_pin_eoi[apic] in __ioapic_write_entry() too?

@SergiiDmytruk
Copy link
Member

@andyhhp, reverting 86001b3970fea4536048607ea6e12541736c48e1 gets rid of the crash. Adding the NULL check has the same effect:

 xen/arch/x86/io_apic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index d44d2c9a41..509b032502 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -304,7 +304,7 @@ void __ioapic_write_entry(
          * something different than the CPU vector, and hence need to be cached
          * for performing EOI.
          */
-        if ( io_apic_pin_eoi )
+        if ( io_apic_pin_eoi && io_apic_pin_eoi[apic] )
             io_apic_pin_eoi[apic][pin] = e.vector;
     }
     else

Thanks!

@andyhhp
Copy link

andyhhp commented Dec 16, 2024

@SergiiDmytruk Thanks for confirming. I'll take this report over to Xen and we'll try to figure out what to do.

@royger
Copy link
Contributor

royger commented Dec 16, 2024

@SergiiDmytruk Could you provide a boot log of the crash without any out of tree patches applied to Xen? So that the base commit is unmodified from Xen upstream repository.

@SergiiDmytruk
Copy link
Member

@royger, here is a boot log for Xen at https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=58ba55959ae1bca0651396d0752c2076a45b5ee6

Boot log

Xen 4.20-unstable
(XEN) Xen version 4.20-unstable (@) (gcc (Debian 12.2.0-14) 12.2.0) debug=y Mon Dec 16 17:25:57 UTC 2024
(XEN) Latest ChangeSet: Mon Dec 16 13:07:19 2024 +0100 git:58ba55959a
(XEN) build-id: 9765e59e8b2c356a01a6ce373b09810ce500cc81
(XEN) CPU Vendor: AMD, Family 22 (0x16), Model 48 (0x30), Stepping 1 (raw 00730f01)
(XEN) BSP microcode revision: 0x07030105
(XEN) Microcode: AuthenticAMD.bin not found during CPIO scan
(XEN) Bootloader: GRUB 2.13
(XEN) Command line: placeholder console=tty0 console=ttyS0,115200 dom0_mem=min:512M dom0_mem=max:4096M ucode=scan smt=off gnttf
(XEN) Xen image load base address: 0x7c400000
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  [0000000000000000, 0000000000000fff] (reserved)
(XEN)  [0000000000001000, 000000000009ffff] (usable)
(XEN)  [00000000000a0000, 00000000000fffff] (reserved)
(XEN)  [0000000000100000, 000000007cd24fff] (usable)
(XEN)  [000000007cd25000, 000000007cd27fff] (reserved)
(XEN)  [000000007cd28000, 000000007cd2cfff] (usable)
(XEN)  [000000007cd2d000, 000000007cde9fff] (reserved)
(XEN)  [000000007cdea000, 000000007d937fff] (usable)
(XEN)  [000000007d938000, 000000007d981fff] (reserved)
(XEN)  [000000007d982000, 000000007daaefff] (usable)
(XEN)  [000000007daaf000, 000000007daaffff] (ACPI data)
(XEN)  [000000007dab0000, 000000007db1ffff] (usable)
(XEN)  [000000007db20000, 000000007db2ffff] (ACPI NVS)
(XEN)  [000000007db30000, 000000007db49fff] (usable)
(XEN)  [000000007db4a000, 000000007db59fff] (ACPI NVS)
(XEN)  [000000007db5a000, 000000007db99fff] (usable)
(XEN)  [000000007db9a000, 000000007dba1fff] (ACPI NVS)
(XEN)  [000000007dba2000, 000000007dbccfff] (usable)
(XEN)  [000000007dbcd000, 000000007dbd1fff] (ACPI data)
(XEN)  [000000007dbd2000, 000000007dbfbfff] (usable)
(XEN)  [000000007dbfc000, 000000007dbfcfff] (ACPI data)
(XEN)  [000000007dbfd000, 000000007dc99fff] (usable)
(XEN)  [000000007dc9a000, 000000007dc9afff] (ACPI data)
(XEN)  [000000007dc9b000, 000000007dc9bfff] (usable)
(XEN)  [000000007dc9c000, 000000007dc9cfff] (ACPI data)
(XEN)  [000000007dc9d000, 000000007dd5afff] (usable)
(XEN)  [000000007dd5b000, 000000007dd73fff] (reserved)
(XEN)  [000000007dd74000, 000000007dd98fff] (usable)
(XEN)  [000000007dd99000, 000000007dd99fff] (ACPI data)
(XEN)  [000000007dd9a000, 000000007dd9cfff] (usable)
(XEN)  [000000007dd9d000, 000000007dd9efff] (ACPI data)
(XEN)  [000000007dd9f000, 000000007e280fff] (usable)
(XEN)  [000000007e281000, 000000007e284fff] (reserved)
(XEN)  [000000007e285000, 000000007e6adfff] (usable)
(XEN)  [000000007e6ae000, 000000007e7f1fff] (reserved)
(XEN)  [000000007e7f2000, 000000007e7f5fff] (ACPI NVS)
(XEN)  [000000007e7f6000, 000000007e7fdfff] (ACPI data)
(XEN)  [000000007e7fe000, 000000007ec4efff] (usable)
(XEN)  [000000007ec4f000, 000000007fffffff] (reserved)
(XEN)  [00000000ff800000, 00000000ff83ffff] (reserved)
(XEN) System RAM: 2025MB (2073868kB)
(XEN) ACPI: RSDP 7E7FD014, 0024 (r2 COREv4)
(XEN) ACPI: XSDT 7E7FC0E8, 0084 (r1 COREv4 COREBOOT        0       1000013)
(XEN) ACPI: FACP 7E7F9000, 0114 (r6 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: DSDT 7E7FA000, 17A7 (r2 COREv4 COREBOOT    10001 INTL 20230628)
(XEN) ACPI: FACS 7E7F5000, 0040
(XEN) ACPI: SSDT 7E7F8000, 064E (r2 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: MCFG 7E7F7000, 003C (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: TPM2 7E7F6000, 004C (r4 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: APIC 7DD9E000, 007E (r3 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: SPCR 7DD9D000, 0058 (r4 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: HEST 7DD99000, 01D0 (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: IVRS 7DC9C000, 00D0 (r2 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: SSDT 7DBCD000, 48A6 (r2 AMD    AGESA           2 MSFT  4000000)
(XEN) ACPI: SSDT 7DBFC000, 07C8 (r1 AMD    AGESA           1 AMD         1)
(XEN) ACPI: HPET 7DC9A000, 0038 (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: FPDT 7DAAF000, 0034 (r1 INTEL  EDK2            2       1000013)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000007ec4f000
(XEN) Domain heap initialised
(XEN) SMBIOS 3.3 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x818 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0]
(XEN) ACPI:             wakeup_vec[7e7f500c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 33, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec20000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 1, version 33, address 0xfec20000, GSI 24-55
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) ACPI: HPET id: 0x10228201 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
(XEN) HEST: Table parsing has been initialized
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 56 GSI, 776 MSI/MSI-X
(XEN) CPU0: 1000 (600 ... 1400) MHz
(XEN) xstate: size: 0x340 and states: 0x7
(XEN) CPU0: AMD Fam16h machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware hints:
(XEN)   Hardware features:
(XEN)   Compiled-in support: INDIRECT_THUNK SHADOW_PAGING HARDEN_ARRAY HARDEN_BRANCH HARDEN_GUEST_ACCESS HARDEN_LOCK
(XEN)   Xen settings: BTI-Thunk: RETPOLINE, SPEC_CTRL: No, Other: BRANCH_HARDEN
(XEN)   Support for HVM VMs: MSR_VIRT_SPEC_CTRL RSB
(XEN)   Support for PV VMs: RSB
(XEN)   XPTI (64-bit PV only): Dom0 disabled, DomU disabled (without PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU disabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 998.130 MHz processor.
(XEN) Freed 1016kB unused BSS memory
(XEN) EFI memory map:
(XEN)  0000000001000-0000000004fff type=2 attr=000000000000000f
(XEN)  0000000005000-0000000086fff type=7 attr=000000000000000f
(XEN)  0000000087000-0000000087fff type=4 attr=000000000000000f
(XEN)  0000000088000-000000008ffff type=7 attr=000000000000000f
(XEN)  0000000090000-000000009ffff type=2 attr=000000000000000f
(XEN)  0000000100000-00000007fffff type=7 attr=000000000000000f
(XEN)  0000000800000-00000015fffff type=4 attr=000000000000000f
(XEN)  0000001600000-00000028c1fff type=2 attr=000000000000000f
(XEN)  00000028c2000-0000078c3ffff type=7 attr=000000000000000f
(XEN)  0000078c40000-000007ac3ffff type=1 attr=000000000000000f
(XEN)  000007ac40000-000007ac5ffff type=4 attr=000000000000000f
(XEN)  000007ac60000-000007c5fffff type=7 attr=000000000000000f
(XEN)  000007c600000-000007c9f7fff type=2 attr=000000000000000f
(XEN)  000007c9f8000-000007cb97fff type=7 attr=000000000000000f
(XEN)  000007cb98000-000007cc5dfff type=1 attr=000000000000000f
(XEN)  000007cc5e000-000007cd24fff type=7 attr=000000000000000f
(XEN)  000007cd25000-000007cd27fff type=6 attr=800000000000000f
(XEN)  000007cd28000-000007cd29fff type=7 attr=000000000000000f
(XEN)  000007cd2a000-000007cd2afff type=4 attr=000000000000000f
(XEN)  000007cd2b000-000007cd2bfff type=2 attr=000000000000000f
(XEN)  000007cd2c000-000007cd2cfff type=4 attr=000000000000000f
(XEN)  000007cd2d000-000007cde9fff type=6 attr=800000000000000f
(XEN)  000007cdea000-000007d937fff type=4 attr=000000000000000f
(XEN)  000007d938000-000007d981fff type=0 attr=000000000000000f
(XEN)  000007d982000-000007d9b5fff type=4 attr=000000000000000f
(XEN)  000007d9b6000-000007d9cefff type=3 attr=000000000000000f
(XEN)  000007d9cf000-000007d9dcfff type=4 attr=000000000000000f
(XEN)  000007d9dd000-000007d9f9fff type=3 attr=000000000000000f
(XEN)  000007d9fa000-000007da1afff type=4 attr=000000000000000f
(XEN)  000007da1b000-000007da3ffff type=3 attr=000000000000000f
(XEN)  000007da40000-000007da40fff type=4 attr=000000000000000f
(XEN)  000007da41000-000007da9bfff type=3 attr=000000000000000f
(XEN)  000007da9c000-000007da9dfff type=4 attr=000000000000000f
(XEN)  000007da9e000-000007daaefff type=3 attr=000000000000000f
(XEN)  000007daaf000-000007daaffff type=9 attr=000000000000000f
(XEN)  000007dab0000-000007dab1fff type=4 attr=000000000000000f
(XEN)  000007dab2000-000007dacafff type=3 attr=000000000000000f
(XEN)  000007dacb000-000007dacbfff type=4 attr=000000000000000f
(XEN)  000007dacc000-000007dacefff type=3 attr=000000000000000f
(XEN)  000007dacf000-000007dad1fff type=4 attr=000000000000000f
(XEN)  000007dad2000-000007daf0fff type=3 attr=000000000000000f
(XEN)  000007daf1000-000007daf1fff type=4 attr=000000000000000f
(XEN)  000007daf2000-000007daf7fff type=3 attr=000000000000000f
(XEN)  000007daf8000-000007daf8fff type=4 attr=000000000000000f
(XEN)  000007daf9000-000007dafbfff type=3 attr=000000000000000f
(XEN)  000007dafc000-000007dafdfff type=4 attr=000000000000000f
(XEN)  000007dafe000-000007db17fff type=3 attr=000000000000000f
(XEN)  000007db18000-000007db1cfff type=4 attr=000000000000000f
(XEN)  000007db1d000-000007db1ffff type=3 attr=000000000000000f
(XEN)  000007db20000-000007db2ffff type=10 attr=000000000000000f
(XEN)  000007db30000-000007db44fff type=3 attr=000000000000000f
(XEN)  000007db45000-000007db46fff type=4 attr=000000000000000f
(XEN)  000007db47000-000007db49fff type=3 attr=000000000000000f
(XEN)  000007db4a000-000007db59fff type=10 attr=000000000000000f
(XEN)  000007db5a000-000007db7ffff type=3 attr=000000000000000f
(XEN)  000007db80000-000007db80fff type=4 attr=000000000000000f
(XEN)  000007db81000-000007db94fff type=3 attr=000000000000000f
(XEN)  000007db95000-000007db99fff type=4 attr=000000000000000f
(XEN)  000007db9a000-000007dba1fff type=10 attr=000000000000000f
(XEN)  000007dba2000-000007dbbbfff type=3 attr=000000000000000f
(XEN)  000007dbbc000-000007dbccfff type=4 attr=000000000000000f
(XEN)  000007dbcd000-000007dbd1fff type=9 attr=000000000000000f
(XEN)  000007dbd2000-000007dbf5fff type=3 attr=000000000000000f
(XEN)  000007dbf6000-000007dbfbfff type=4 attr=000000000000000f
(XEN)  000007dbfc000-000007dbfcfff type=9 attr=000000000000000f
(XEN)  000007dbfd000-000007dc99fff type=3 attr=000000000000000f
(XEN)  000007dc9a000-000007dc9afff type=9 attr=000000000000000f
(XEN)  000007dc9b000-000007dc9bfff type=4 attr=000000000000000f
(XEN)  000007dc9c000-000007dc9cfff type=9 attr=000000000000000f
(XEN)  000007dc9d000-000007dd24fff type=4 attr=000000000000000f
(XEN)  000007dd25000-000007dd5afff type=3 attr=000000000000000f
(XEN)  000007dd5b000-000007dd73fff type=6 attr=800000000000000f
(XEN)  000007dd74000-000007dd98fff type=3 attr=000000000000000f
(XEN)  000007dd99000-000007dd99fff type=9 attr=000000000000000f
(XEN)  000007dd9a000-000007dd9cfff type=3 attr=000000000000000f
(XEN)  000007dd9d000-000007dd9efff type=9 attr=000000000000000f
(XEN)  000007dd9f000-000007dda6fff type=4 attr=000000000000000f
(XEN)  000007dda7000-000007dda8fff type=3 attr=000000000000000f
(XEN)  000007dda9000-000007ddbdfff type=4 attr=000000000000000f
(XEN)  000007ddbe000-000007ddf1fff type=3 attr=000000000000000f
(XEN)  000007ddf2000-000007e207fff type=4 attr=000000000000000f
(XEN)  000007e208000-000007e20bfff type=3 attr=000000000000000f
(XEN)  000007e20c000-000007e20ffff type=4 attr=000000000000000f
(XEN)  000007e210000-000007e215fff type=3 attr=000000000000000f
(XEN)  000007e216000-000007e218fff type=4 attr=000000000000000f
(XEN)  000007e219000-000007e227fff type=3 attr=000000000000000f
(XEN)  000007e228000-000007e22cfff type=4 attr=000000000000000f
(XEN)  000007e22d000-000007e268fff type=3 attr=000000000000000f
(XEN)  000007e269000-000007e26afff type=4 attr=000000000000000f
(XEN)  000007e26b000-000007e26ffff type=3 attr=000000000000000f
(XEN)  000007e270000-000007e274fff type=4 attr=000000000000000f
(XEN)  000007e275000-000007e276fff type=3 attr=000000000000000f
(XEN)  000007e277000-000007e277fff type=4 attr=000000000000000f
(XEN)  000007e278000-000007e279fff type=3 attr=000000000000000f
(XEN)  000007e27a000-000007e27cfff type=4 attr=000000000000000f
(XEN)  000007e27d000-000007e27efff type=3 attr=000000000000000f
(XEN)  000007e27f000-000007e280fff type=4 attr=000000000000000f
(XEN)  000007e281000-000007e284fff type=0 attr=000000000000000f
(XEN)  000007e285000-000007e285fff type=3 attr=000000000000000f
(XEN)  000007e286000-000007e286fff type=4 attr=000000000000000f
(XEN)  000007e287000-000007e29efff type=3 attr=000000000000000f
(XEN)  000007e29f000-000007e699fff type=4 attr=000000000000000f
(XEN)  000007e69a000-000007e6a0fff type=3 attr=000000000000000f
(XEN)  000007e6a1000-000007e6adfff type=4 attr=000000000000000f
(XEN)  000007e6ae000-000007e72dfff type=5 attr=800000000000000f
(XEN)  000007e72e000-000007e7edfff type=6 attr=800000000000000f
(XEN)  000007e7ee000-000007e7f1fff type=0 attr=000000000000000f
(XEN)  000007e7f2000-000007e7f5fff type=10 attr=000000000000000f
(XEN)  000007e7f6000-000007e7fdfff type=9 attr=000000000000000f
(XEN)  000007e7fe000-000007e9fffff type=4 attr=000000000000000f
(XEN)  000007ea00000-000007eb8ffff type=7 attr=000000000000000f
(XEN)  000007eb90000-000007ebaffff type=4 attr=000000000000000f
(XEN)  000007ebb0000-000007ec02fff type=3 attr=000000000000000f
(XEN)  000007ec03000-000007ec03fff type=4 attr=000000000000000f
(XEN)  000007ec04000-000007ec3ffff type=3 attr=000000000000000f
(XEN)  000007ec40000-000007ec4efff type=7 attr=000000000000000f
(XEN)  0000000000000-0000000000fff type=0 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=000000000000000f
(XEN)  000007ec4f000-000007ec83fff type=0 attr=000000000000000f
(XEN)  000007ec84000-000007ec93fff type=0 attr=800000000000000f
(XEN)  000007ec94000-000007fffffff type=0 attr=000000000000000f
(XEN)  00000ff800000-00000ff83ffff type=11 attr=8000000000000001
(XEN) alt table ffff82d04049c3b8 -> ffff82d0404aef22
(XEN) AMD-Vi: IOMMU Extended Features:
(XEN) - Peripheral Page Service Request
(XEN) - Invalidate All Command
(XEN) - Guest APIC
(XEN) - Performance Counters
(XEN) - Host Address Translation Size: 0x2
(XEN) - Guest Address Translation Size: 0
(XEN) - Guest CR3 Root Table Level: 0
(XEN) - Maximum PASID: 0x8
(XEN) - SMI Filter Register: 0x1
(XEN) - SMI Filter Register Count: 0x2
(XEN) - Guest Virtual APIC Modes: 0
(XEN) - Dual PPR Log: 0
(XEN) - Dual Event Log: 0
(XEN) - Device Table Segmentation: 0
(XEN) - Memory Access Routing and Control: 0
(XEN) - VM Table Size: 0
(XEN) AMD-Vi: Disabled HAP memory map sharing with IOMMU
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabling APIC mode.  Using 2 I/O APICs
(XEN) ----[ Xen-4.20-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d040328046>] __ioapic_write_entry+0x83/0x95
(XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx: 0000000000010000
(XEN) rdx: 0000000000006000   rsi: 0000000000000000   rdi: 0000000000000001
(XEN) rbp: ffff82d04045fc60   rsp: ffff82d04045fc50   r8:  0000000000000010
(XEN) r9:  ffff82cffffff000   r10: ffff82d0404b6f00   r11: 0000000000000001
(XEN) r12: 0000000000000011   r13: ffff830077400058   r14: 0000000000000000
(XEN) r15: 0000000000000001   cr0: 000000008005003b   cr4: 00000000000406e0
(XEN) cr3: 000000007c8c2000   cr2: 0000000000000000
(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d040328046> (__ioapic_write_entry+0x83/0x95):
(XEN)  89 ff 89 f6 48 8b 04 f8 <88> 0c 30 5b 41 5c 5d c3 48 89 ca e8 41 d5 f4 ff
(XEN) Xen stack trace from rsp=ffff82d04045fc50:
(XEN)    ffff83007dc9e000 ffff83007e1fe600 ffff82d04045fcc8 ffff82d04027464b
(XEN)    0000000000010000 0000000000000000 0000000000000246 0000000000000000
(XEN)    0000000000010000 0000000000000001 0000000000000000 0000000000000000
(XEN)    ffff82d0404b6484 ffff82d0404b6484 ffff82d0404b6480 ffff82d04045fcd8
(XEN)    ffff82d0402755a1 ffff82d04045fcf8 ffff82d040328056 0000000000000001
(XEN)    0000000000000000 ffff82d04045fd20 ffff82d0403283c1 0000000000010000
(XEN)    0000000000000000 0000000000000001 ffff82d04045fd50 ffff82d040328480
(XEN)    00000000ffffffff 00000000ffffffff 00000000ffffffff 0000000000000000
(XEN)    ffff82d04045fda0 ffff82d0404448b7 ffff82d0404b6480 0000000000000000
(XEN)    0000000000000000 ffff82d0405cf3a0 ffff82d0405ce9c0 00000000ffffffff
(XEN)    00000000ffffffff ffff82d0404b7a20 ffff82d04045fdb8 ffff82d04044c9b0
(XEN)    ffff82d0405ce9c0 ffff82d04045fee8 ffff82d04044bec2 ffff82d04049af80
(XEN)    0000000077f18000 ffff82d0405dfe80 0000000001000000 00000000028c1400
(XEN)    ffff82d040495f60 000000000007ec00 ffff82d04049af80 00000000000001ff
(XEN)    0000000000800163 ffff82d040495f60 000000000007ec4f 0000000000000003
(XEN)    0000000300000001 ffff83007ea60000 ffff82d04045fef8 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000800000000 000000010000006e 0000000000000003 00000000000002f8
(XEN)    ffff82d0405c6000 ffff82d0404c8000 0000000000000000 000000007aaea8e0
(XEN) Xen call trace:
(XEN)    [<ffff82d040328046>] R __ioapic_write_entry+0x83/0x95
(XEN)    [<ffff82d04027464b>] F amd_iommu_ioapic_update_ire+0x1ea/0x273
(XEN)    [<ffff82d0402755a1>] F iommu_update_ire_from_apic+0xa/0xc
(XEN)    [<ffff82d040328056>] F __ioapic_write_entry+0x93/0x95
(XEN)    [<ffff82d0403283c1>] F arch/x86/io_apic.c#clear_IO_APIC_pin+0x7c/0x10e
(XEN)    [<ffff82d040328480>] F arch/x86/io_apic.c#clear_IO_APIC+0x2d/0x61
(XEN)    [<ffff82d0404448b7>] F enable_IO_APIC+0x2e3/0x34f
(XEN)    [<ffff82d04044c9b0>] F smp_prepare_cpus+0x254/0x27a
(XEN)    [<ffff82d04044bec2>] F __start_xen+0x1ce1/0x23ae
(XEN)    [<ffff82d0402033ae>] F __high_start+0x8e/0x90
(XEN)
(XEN) Pagetable walk from 0000000000000000:
(XEN)  L4[0x000] = 000000007dbfd063 ffffffffffffffff
(XEN)  L3[0x000] = 000000007dbfa063 ffffffffffffffff
(XEN)  L2[0x000] = 000000007dbcc063 ffffffffffffffff
(XEN)  L1[0x000] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0002]
(XEN) Faulting linear address: 0000000000000000
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

@royger
Copy link
Contributor

royger commented Dec 16, 2024

Could you provide the boot log with the following instrumentation patch applied?

diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index d9db2efc4f58..2b4df4388ad6 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -304,6 +304,7 @@ void __ioapic_write_entry(
          * something different than the CPU vector, and hence need to be cached
          * for performing EOI.
          */
+        printk("IOAPIC%u pin %u\n", apic, pin);
         if ( io_apic_pin_eoi )
             io_apic_pin_eoi[apic][pin] = e.vector;
     }
@@ -1373,6 +1374,7 @@ void __init enable_IO_APIC(void)

         if ( io_apic_pin_eoi )
         {
+            printk("IOAPIC%u EOI alloc nr: %u\n", apic, nr_ioapic_entries[apic]);
             io_apic_pin_eoi[apic] = xvmalloc_array(typeof(**io_apic_pin_eoi),
                                                    nr_ioapic_entries[apic]);
             BUG_ON(!io_apic_pin_eoi[apic]);

@SergiiDmytruk
Copy link
Member

Log with those debug prints:

Details

Xen 4.20-unstable
(XEN) Xen version 4.20-unstable (@) (gcc (Debian 12.2.0-14) 12.2.0) debug=y Mon Dec 16 18:01:36 UTC 2024
(XEN) Latest ChangeSet: Mon Dec 16 13:07:19 2024 +0100 git:58ba55959a-dirty
(XEN) build-id: 6dd37533dd28bd03e89403099105748b2468c635
(XEN) CPU Vendor: AMD, Family 22 (0x16), Model 48 (0x30), Stepping 1 (raw 00730f01)
(XEN) BSP microcode revision: 0x07030105
(XEN) Microcode: AuthenticAMD.bin not found during CPIO scan
(XEN) Bootloader: GRUB 2.13
(XEN) Command line: placeholder console=tty0 console=ttyS0,115200 dom0_mem=min:512M dom0_mem=max:4096M ucode=scan smt=off gnttf
(XEN) Xen image load base address: 0x7c400000
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  [0000000000000000, 0000000000000fff] (reserved)
(XEN)  [0000000000001000, 000000000009ffff] (usable)
(XEN)  [00000000000a0000, 00000000000fffff] (reserved)
(XEN)  [0000000000100000, 000000007cd24fff] (usable)
(XEN)  [000000007cd25000, 000000007cd27fff] (reserved)
(XEN)  [000000007cd28000, 000000007cd2cfff] (usable)
(XEN)  [000000007cd2d000, 000000007cde9fff] (reserved)
(XEN)  [000000007cdea000, 000000007d937fff] (usable)
(XEN)  [000000007d938000, 000000007d981fff] (reserved)
(XEN)  [000000007d982000, 000000007daaefff] (usable)
(XEN)  [000000007daaf000, 000000007daaffff] (ACPI data)
(XEN)  [000000007dab0000, 000000007db1ffff] (usable)
(XEN)  [000000007db20000, 000000007db2ffff] (ACPI NVS)
(XEN)  [000000007db30000, 000000007db49fff] (usable)
(XEN)  [000000007db4a000, 000000007db59fff] (ACPI NVS)
(XEN)  [000000007db5a000, 000000007db99fff] (usable)
(XEN)  [000000007db9a000, 000000007dba1fff] (ACPI NVS)
(XEN)  [000000007dba2000, 000000007dbccfff] (usable)
(XEN)  [000000007dbcd000, 000000007dbd1fff] (ACPI data)
(XEN)  [000000007dbd2000, 000000007dbfbfff] (usable)
(XEN)  [000000007dbfc000, 000000007dbfcfff] (ACPI data)
(XEN)  [000000007dbfd000, 000000007dc99fff] (usable)
(XEN)  [000000007dc9a000, 000000007dc9afff] (ACPI data)
(XEN)  [000000007dc9b000, 000000007dc9bfff] (usable)
(XEN)  [000000007dc9c000, 000000007dc9cfff] (ACPI data)
(XEN)  [000000007dc9d000, 000000007dd5afff] (usable)
(XEN)  [000000007dd5b000, 000000007dd73fff] (reserved)
(XEN)  [000000007dd74000, 000000007dd98fff] (usable)
(XEN)  [000000007dd99000, 000000007dd99fff] (ACPI data)
(XEN)  [000000007dd9a000, 000000007dd9cfff] (usable)
(XEN)  [000000007dd9d000, 000000007dd9efff] (ACPI data)
(XEN)  [000000007dd9f000, 000000007e280fff] (usable)
(XEN)  [000000007e281000, 000000007e284fff] (reserved)
(XEN)  [000000007e285000, 000000007e6adfff] (usable)
(XEN)  [000000007e6ae000, 000000007e7f1fff] (reserved)
(XEN)  [000000007e7f2000, 000000007e7f5fff] (ACPI NVS)
(XEN)  [000000007e7f6000, 000000007e7fdfff] (ACPI data)
(XEN)  [000000007e7fe000, 000000007ec4efff] (usable)
(XEN)  [000000007ec4f000, 000000007fffffff] (reserved)
(XEN)  [00000000ff800000, 00000000ff83ffff] (reserved)
(XEN) System RAM: 2025MB (2073868kB)
(XEN) ACPI: RSDP 7E7FD014, 0024 (r2 COREv4)
(XEN) ACPI: XSDT 7E7FC0E8, 0084 (r1 COREv4 COREBOOT        0       1000013)
(XEN) ACPI: FACP 7E7F9000, 0114 (r6 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: DSDT 7E7FA000, 17A7 (r2 COREv4 COREBOOT    10001 INTL 20230628)
(XEN) ACPI: FACS 7E7F5000, 0040
(XEN) ACPI: SSDT 7E7F8000, 064E (r2 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: MCFG 7E7F7000, 003C (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: TPM2 7E7F6000, 004C (r4 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: APIC 7DD9E000, 007E (r3 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: SPCR 7DD9D000, 0058 (r4 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: HEST 7DD99000, 01D0 (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: IVRS 7DC9C000, 00D0 (r2 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: SSDT 7DBCD000, 48A6 (r2 AMD    AGESA           2 MSFT  4000000)
(XEN) ACPI: SSDT 7DBFC000, 07C8 (r1 AMD    AGESA           1 AMD         1)
(XEN) ACPI: HPET 7DC9A000, 0038 (r1 COREv4 COREBOOT        0 CORE 20230628)
(XEN) ACPI: FPDT 7DAAF000, 0034 (r1 INTEL  EDK2            2       1000013)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000007ec4f000
(XEN) Domain heap initialised
(XEN) SMBIOS 3.3 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x818 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0]
(XEN) ACPI:             wakeup_vec[7e7f500c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 33, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec20000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 1, version 33, address 0xfec20000, GSI 24-55
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) ACPI: HPET id: 0x10228201 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
(XEN) HEST: Table parsing has been initialized
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 56 GSI, 776 MSI/MSI-X
(XEN) CPU0: 1000 (600 ... 1400) MHz
(XEN) xstate: size: 0x340 and states: 0x7
(XEN) CPU0: AMD Fam16h machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware hints:
(XEN)   Hardware features:
(XEN)   Compiled-in support: INDIRECT_THUNK SHADOW_PAGING HARDEN_ARRAY HARDEN_BRANCH HARDEN_GUEST_ACCESS HARDEN_LOCK
(XEN)   Xen settings: BTI-Thunk: RETPOLINE, SPEC_CTRL: No, Other: BRANCH_HARDEN
(XEN)   Support for HVM VMs: MSR_VIRT_SPEC_CTRL RSB
(XEN)   Support for PV VMs: RSB
(XEN)   XPTI (64-bit PV only): Dom0 disabled, DomU disabled (without PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU disabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 998.121 MHz processor.
(XEN) Freed 1016kB unused BSS memory
(XEN) EFI memory map:
(XEN)  0000000001000-0000000004fff type=2 attr=000000000000000f
(XEN)  0000000005000-0000000086fff type=7 attr=000000000000000f
(XEN)  0000000087000-0000000087fff type=4 attr=000000000000000f
(XEN)  0000000088000-000000008ffff type=7 attr=000000000000000f
(XEN)  0000000090000-000000009ffff type=2 attr=000000000000000f
(XEN)  0000000100000-00000007fffff type=7 attr=000000000000000f
(XEN)  0000000800000-00000015fffff type=4 attr=000000000000000f
(XEN)  0000001600000-00000028c1fff type=2 attr=000000000000000f
(XEN)  00000028c2000-0000078c3ffff type=7 attr=000000000000000f
(XEN)  0000078c40000-000007ac3ffff type=1 attr=000000000000000f
(XEN)  000007ac40000-000007ac5ffff type=4 attr=000000000000000f
(XEN)  000007ac60000-000007c5fffff type=7 attr=000000000000000f
(XEN)  000007c600000-000007c9f7fff type=2 attr=000000000000000f
(XEN)  000007c9f8000-000007cb97fff type=7 attr=000000000000000f
(XEN)  000007cb98000-000007cc5dfff type=1 attr=000000000000000f
(XEN)  000007cc5e000-000007cd24fff type=7 attr=000000000000000f
(XEN)  000007cd25000-000007cd27fff type=6 attr=800000000000000f
(XEN)  000007cd28000-000007cd29fff type=7 attr=000000000000000f
(XEN)  000007cd2a000-000007cd2afff type=4 attr=000000000000000f
(XEN)  000007cd2b000-000007cd2bfff type=2 attr=000000000000000f
(XEN)  000007cd2c000-000007cd2cfff type=4 attr=000000000000000f
(XEN)  000007cd2d000-000007cde9fff type=6 attr=800000000000000f
(XEN)  000007cdea000-000007d937fff type=4 attr=000000000000000f
(XEN)  000007d938000-000007d981fff type=0 attr=000000000000000f
(XEN)  000007d982000-000007d9b5fff type=4 attr=000000000000000f
(XEN)  000007d9b6000-000007d9cefff type=3 attr=000000000000000f
(XEN)  000007d9cf000-000007d9dcfff type=4 attr=000000000000000f
(XEN)  000007d9dd000-000007d9f9fff type=3 attr=000000000000000f
(XEN)  000007d9fa000-000007da1afff type=4 attr=000000000000000f
(XEN)  000007da1b000-000007da3ffff type=3 attr=000000000000000f
(XEN)  000007da40000-000007da40fff type=4 attr=000000000000000f
(XEN)  000007da41000-000007da9bfff type=3 attr=000000000000000f
(XEN)  000007da9c000-000007da9dfff type=4 attr=000000000000000f
(XEN)  000007da9e000-000007daaefff type=3 attr=000000000000000f
(XEN)  000007daaf000-000007daaffff type=9 attr=000000000000000f
(XEN)  000007dab0000-000007dab1fff type=4 attr=000000000000000f
(XEN)  000007dab2000-000007dacafff type=3 attr=000000000000000f
(XEN)  000007dacb000-000007dacbfff type=4 attr=000000000000000f
(XEN)  000007dacc000-000007dacefff type=3 attr=000000000000000f
(XEN)  000007dacf000-000007dad1fff type=4 attr=000000000000000f
(XEN)  000007dad2000-000007daf0fff type=3 attr=000000000000000f
(XEN)  000007daf1000-000007daf1fff type=4 attr=000000000000000f
(XEN)  000007daf2000-000007daf7fff type=3 attr=000000000000000f
(XEN)  000007daf8000-000007daf8fff type=4 attr=000000000000000f
(XEN)  000007daf9000-000007dafbfff type=3 attr=000000000000000f
(XEN)  000007dafc000-000007dafdfff type=4 attr=000000000000000f
(XEN)  000007dafe000-000007db17fff type=3 attr=000000000000000f
(XEN)  000007db18000-000007db1cfff type=4 attr=000000000000000f
(XEN)  000007db1d000-000007db1ffff type=3 attr=000000000000000f
(XEN)  000007db20000-000007db2ffff type=10 attr=000000000000000f
(XEN)  000007db30000-000007db44fff type=3 attr=000000000000000f
(XEN)  000007db45000-000007db46fff type=4 attr=000000000000000f
(XEN)  000007db47000-000007db49fff type=3 attr=000000000000000f
(XEN)  000007db4a000-000007db59fff type=10 attr=000000000000000f
(XEN)  000007db5a000-000007db7ffff type=3 attr=000000000000000f
(XEN)  000007db80000-000007db80fff type=4 attr=000000000000000f
(XEN)  000007db81000-000007db94fff type=3 attr=000000000000000f
(XEN)  000007db95000-000007db99fff type=4 attr=000000000000000f
(XEN)  000007db9a000-000007dba1fff type=10 attr=000000000000000f
(XEN)  000007dba2000-000007dbbbfff type=3 attr=000000000000000f
(XEN)  000007dbbc000-000007dbccfff type=4 attr=000000000000000f
(XEN)  000007dbcd000-000007dbd1fff type=9 attr=000000000000000f
(XEN)  000007dbd2000-000007dbf5fff type=3 attr=000000000000000f
(XEN)  000007dbf6000-000007dbfbfff type=4 attr=000000000000000f
(XEN)  000007dbfc000-000007dbfcfff type=9 attr=000000000000000f
(XEN)  000007dbfd000-000007dc99fff type=3 attr=000000000000000f
(XEN)  000007dc9a000-000007dc9afff type=9 attr=000000000000000f
(XEN)  000007dc9b000-000007dc9bfff type=4 attr=000000000000000f
(XEN)  000007dc9c000-000007dc9cfff type=9 attr=000000000000000f
(XEN)  000007dc9d000-000007dd24fff type=4 attr=000000000000000f
(XEN)  000007dd25000-000007dd5afff type=3 attr=000000000000000f
(XEN)  000007dd5b000-000007dd73fff type=6 attr=800000000000000f
(XEN)  000007dd74000-000007dd98fff type=3 attr=000000000000000f
(XEN)  000007dd99000-000007dd99fff type=9 attr=000000000000000f
(XEN)  000007dd9a000-000007dd9cfff type=3 attr=000000000000000f
(XEN)  000007dd9d000-000007dd9efff type=9 attr=000000000000000f
(XEN)  000007dd9f000-000007dda6fff type=4 attr=000000000000000f
(XEN)  000007dda7000-000007dda8fff type=3 attr=000000000000000f
(XEN)  000007dda9000-000007ddbdfff type=4 attr=000000000000000f
(XEN)  000007ddbe000-000007ddf1fff type=3 attr=000000000000000f
(XEN)  000007ddf2000-000007e207fff type=4 attr=000000000000000f
(XEN)  000007e208000-000007e20bfff type=3 attr=000000000000000f
(XEN)  000007e20c000-000007e20ffff type=4 attr=000000000000000f
(XEN)  000007e210000-000007e215fff type=3 attr=000000000000000f
(XEN)  000007e216000-000007e218fff type=4 attr=000000000000000f
(XEN)  000007e219000-000007e227fff type=3 attr=000000000000000f
(XEN)  000007e228000-000007e22cfff type=4 attr=000000000000000f
(XEN)  000007e22d000-000007e268fff type=3 attr=000000000000000f
(XEN)  000007e269000-000007e26afff type=4 attr=000000000000000f
(XEN)  000007e26b000-000007e26ffff type=3 attr=000000000000000f
(XEN)  000007e270000-000007e274fff type=4 attr=000000000000000f
(XEN)  000007e275000-000007e276fff type=3 attr=000000000000000f
(XEN)  000007e277000-000007e277fff type=4 attr=000000000000000f
(XEN)  000007e278000-000007e279fff type=3 attr=000000000000000f
(XEN)  000007e27a000-000007e27cfff type=4 attr=000000000000000f
(XEN)  000007e27d000-000007e27efff type=3 attr=000000000000000f
(XEN)  000007e27f000-000007e280fff type=4 attr=000000000000000f
(XEN)  000007e281000-000007e284fff type=0 attr=000000000000000f
(XEN)  000007e285000-000007e285fff type=3 attr=000000000000000f
(XEN)  000007e286000-000007e286fff type=4 attr=000000000000000f
(XEN)  000007e287000-000007e29efff type=3 attr=000000000000000f
(XEN)  000007e29f000-000007e699fff type=4 attr=000000000000000f
(XEN)  000007e69a000-000007e6a0fff type=3 attr=000000000000000f
(XEN)  000007e6a1000-000007e6adfff type=4 attr=000000000000000f
(XEN)  000007e6ae000-000007e72dfff type=5 attr=800000000000000f
(XEN)  000007e72e000-000007e7edfff type=6 attr=800000000000000f
(XEN)  000007e7ee000-000007e7f1fff type=0 attr=000000000000000f
(XEN)  000007e7f2000-000007e7f5fff type=10 attr=000000000000000f
(XEN)  000007e7f6000-000007e7fdfff type=9 attr=000000000000000f
(XEN)  000007e7fe000-000007e9fffff type=4 attr=000000000000000f
(XEN)  000007ea00000-000007eb8ffff type=7 attr=000000000000000f
(XEN)  000007eb90000-000007ebaffff type=4 attr=000000000000000f
(XEN)  000007ebb0000-000007ec02fff type=3 attr=000000000000000f
(XEN)  000007ec03000-000007ec03fff type=4 attr=000000000000000f
(XEN)  000007ec04000-000007ec3ffff type=3 attr=000000000000000f
(XEN)  000007ec40000-000007ec4efff type=7 attr=000000000000000f
(XEN)  0000000000000-0000000000fff type=0 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=000000000000000f
(XEN)  000007ec4f000-000007ec83fff type=0 attr=000000000000000f
(XEN)  000007ec84000-000007ec93fff type=0 attr=800000000000000f
(XEN)  000007ec94000-000007fffffff type=0 attr=000000000000000f
(XEN)  00000ff800000-00000ff83ffff type=11 attr=8000000000000001
(XEN) alt table ffff82d04049c3b8 -> ffff82d0404aef22
(XEN) IOAPIC0 pin 0
(XEN) AMD-Vi: IOMMU Extended Features:
(XEN) - Peripheral Page Service Request
(XEN) - Invalidate All Command
(XEN) - Guest APIC
(XEN) - Performance Counters
(XEN) - Host Address Translation Size: 0x2
(XEN) - Guest Address Translation Size: 0
(XEN) - Guest CR3 Root Table Level: 0
(XEN) - Maximum PASID: 0x8
(XEN) - SMI Filter Register: 0x1
(XEN) - SMI Filter Register Count: 0x2
(XEN) - Guest Virtual APIC Modes: 0
(XEN) - Dual PPR Log: 0
(XEN) - Dual Event Log: 0
(XEN) - Device Table Segmentation: 0
(XEN) - Memory Access Routing and Control: 0
(XEN) - VM Table Size: 0
(XEN) AMD-Vi: Disabled HAP memory map sharing with IOMMU
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) IOAPIC0 pin 0
(XEN) IOAPIC0 pin 1
(XEN) IOAPIC0 pin 2
(XEN) IOAPIC0 pin 3
(XEN) IOAPIC0 pin 4
(XEN) IOAPIC0 pin 5
(XEN) IOAPIC0 pin 6
(XEN) IOAPIC0 pin 7
(XEN) IOAPIC0 pin 8
(XEN) IOAPIC0 pin 9
(XEN) IOAPIC0 pin 10
(XEN) IOAPIC0 pin 11
(XEN) IOAPIC0 pin 12
(XEN) IOAPIC0 pin 13
(XEN) IOAPIC0 pin 14
(XEN) IOAPIC0 pin 15
(XEN) IOAPIC0 pin 16
(XEN) IOAPIC0 pin 17
(XEN) IOAPIC0 pin 18
(XEN) IOAPIC0 pin 19
(XEN) IOAPIC0 pin 20
(XEN) IOAPIC0 pin 21
(XEN) IOAPIC0 pin 22
(XEN) IOAPIC0 pin 23
(XEN) IOAPIC1 pin 0
(XEN) IOAPIC1 pin 1
(XEN) IOAPIC1 pin 2
(XEN) IOAPIC1 pin 3
(XEN) IOAPIC1 pin 4
(XEN) IOAPIC1 pin 5
(XEN) IOAPIC1 pin 6
(XEN) IOAPIC1 pin 7
(XEN) IOAPIC1 pin 8
(XEN) IOAPIC1 pin 9
(XEN) IOAPIC1 pin 10
(XEN) IOAPIC1 pin 11
(XEN) IOAPIC1 pin 12
(XEN) IOAPIC1 pin 13
(XEN) IOAPIC1 pin 14
(XEN) IOAPIC1 pin 15
(XEN) IOAPIC1 pin 16
(XEN) IOAPIC1 pin 17
(XEN) IOAPIC1 pin 18
(XEN) IOAPIC1 pin 19
(XEN) IOAPIC1 pin 20
(XEN) IOAPIC1 pin 21
(XEN) IOAPIC1 pin 22
(XEN) IOAPIC1 pin 23
(XEN) IOAPIC1 pin 24
(XEN) IOAPIC1 pin 25
(XEN) IOAPIC1 pin 26
(XEN) IOAPIC1 pin 27
(XEN) IOAPIC1 pin 28
(XEN) IOAPIC1 pin 29
(XEN) IOAPIC1 pin 30
(XEN) IOAPIC1 pin 31
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabling APIC mode.  Using 2 I/O APICs
(XEN) IOAPIC0 EOI alloc nr: 24
(XEN) IOAPIC0 pin 0
(XEN) IOAPIC0 pin 0
(XEN) IOAPIC0 pin 0
(XEN) IOAPIC0 pin 1
(XEN) IOAPIC0 pin 2
(XEN) IOAPIC0 pin 3
(XEN) IOAPIC0 pin 4
(XEN) IOAPIC0 pin 5
(XEN) IOAPIC0 pin 6
(XEN) IOAPIC0 pin 7
(XEN) IOAPIC0 pin 8
(XEN) IOAPIC0 pin 9
(XEN) IOAPIC0 pin 10
(XEN) IOAPIC0 pin 11
(XEN) IOAPIC0 pin 12
(XEN) IOAPIC0 pin 13
(XEN) IOAPIC0 pin 14
(XEN) IOAPIC0 pin 15
(XEN) IOAPIC0 pin 16
(XEN) IOAPIC0 pin 17
(XEN) IOAPIC0 pin 18
(XEN) IOAPIC0 pin 19
(XEN) IOAPIC0 pin 20
(XEN) IOAPIC0 pin 21
(XEN) IOAPIC0 pin 22
(XEN) IOAPIC0 pin 23
(XEN) IOAPIC1 pin 0
(XEN) ----[ Xen-4.20-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d040328076>] __ioapic_write_entry+0xb3/0xbc
(XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor
(XEN) rax: 0000000000000000   rbx: 0000000000010000   rcx: 0000000000000000
(XEN) rdx: ffff82d04045ffff   rsi: 000000000000000a   rdi: ffff82d0404c4b90
(XEN) rbp: ffff82d04045fc50   rsp: ffff82d04045fc38   r8:  0000000000000001
(XEN) r9:  0000000000000000   r10: 0000000000000001   r11: 000000000000000a
(XEN) r12: 0000000000000001   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000001   cr0: 000000008005003b   cr4: 00000000000406e0
(XEN) cr3: 000000007c8c2000   cr2: 0000000000000000
(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d040328076> (__ioapic_write_entry+0xb3/0xbc):
(XEN)  e4 45 89 ed 4a 8b 04 e0 <42> 88 1c 28 e9 72 ff ff ff 55 48 89 e5 41 57 41
(XEN) Xen stack trace from rsp=ffff82d04045fc38:
(XEN)    ffff83007dc9e000 ffff83007e1fe600 ffff830077400058 ffff82d04045fcb8
(XEN)    ffff82d04027464b 0000000000010000 0000000000000000 0000000000000246
(XEN)    0000000000000000 0000000000010000 0000000000000001 0000000000010000
(XEN)    0000000000000001 0000000000000000 ffff82d0404b6484 ffff82d0404b6480
(XEN)    ffff82d04045fcc8 ffff82d0402755a1 ffff82d04045fcf0 ffff82d040327ff1
(XEN)    0000000000000001 0000000000000000 ffff82d0404b6484 ffff82d04045fd18
(XEN)    ffff82d0403283e8 0000000000010000 0000000000000000 0000000000000001
(XEN)    ffff82d04045fd48 ffff82d0403284a7 00000000ffffffff 00000000ffffffff
(XEN)    00000000ffffffff 0000000000000000 ffff82d04045fda0 ffff82d0404448e3
(XEN)    000000004045fd60 ffff82d0404b6480 0000000000000000 0000000000000000
(XEN)    ffff82d0405cf3a0 ffff82d0405ce9c0 00000000ffffffff 00000000ffffffff
(XEN)    ffff82d0404b7a20 ffff82d04045fdb8 ffff82d04044c9dc ffff82d0405ce9c0
(XEN)    ffff82d04045fee8 ffff82d04044beee ffff82d04049af80 0000000077f18000
(XEN)    ffff82d0405dfe80 0000000001000000 00000000028c1400 ffff82d040495f60
(XEN)    000000000007ec00 ffff82d04049af80 00000000000001ff 0000000000800163
(XEN)    ffff82d040495f60 000000000007ec4f 0000000000000003 0000000300000001
(XEN)    ffff83007ea60000 ffff82d04045fef8 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000800000000
(XEN)    000000010000006e 0000000000000003 00000000000002f8 ffff82d0405c6000
(XEN) Xen call trace:
(XEN)    [<ffff82d040328076>] R __ioapic_write_entry+0xb3/0xbc
(XEN)    [<ffff82d04027464b>] F amd_iommu_ioapic_update_ire+0x1ea/0x273
(XEN)    [<ffff82d0402755a1>] F iommu_update_ire_from_apic+0xa/0xc
(XEN)    [<ffff82d040327ff1>] F __ioapic_write_entry+0x2e/0xbc
(XEN)    [<ffff82d0403283e8>] F arch/x86/io_apic.c#clear_IO_APIC_pin+0x7c/0x10e
(XEN)    [<ffff82d0403284a7>] F arch/x86/io_apic.c#clear_IO_APIC+0x2d/0x61
(XEN)    [<ffff82d0404448e3>] F enable_IO_APIC+0x30f/0x37b
(XEN)    [<ffff82d04044c9dc>] F smp_prepare_cpus+0x254/0x27a
(XEN)    [<ffff82d04044beee>] F __start_xen+0x1ce1/0x23ae
(XEN)    [<ffff82d0402033ae>] F __high_start+0x8e/0x90
(XEN)
(XEN) Pagetable walk from 0000000000000000:
(XEN)  L4[0x000] = 000000007dbfd063 ffffffffffffffff
(XEN)  L3[0x000] = 000000007dbfa063 ffffffffffffffff
(XEN)  L2[0x000] = 000000007dbcc063 ffffffffffffffff
(XEN)  L1[0x000] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0002]
(XEN) Faulting linear address: 0000000000000000
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

@andyhhp
Copy link

andyhhp commented Dec 16, 2024

Huh, so we really do allocate only one EOI bitmap, but there are two IO-APICs

@royger
Copy link
Contributor

royger commented Dec 16, 2024

Can you try the following patch:

diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c
index d9db2efc4f58..0614e71c0667 100644
--- a/xen/arch/x86/io_apic.c
+++ b/xen/arch/x86/io_apic.c
@@ -1389,14 +1389,15 @@ void __init enable_IO_APIC(void)
             /* If the interrupt line is enabled and in ExtInt mode
              * I have found the pin where the i8259 is connected.
              */
-            if ((entry.mask == 0) && (entry.delivery_mode == dest_ExtINT)) {
+            if ( entry.mask == 0 && entry.delivery_mode == dest_ExtINT &&
+                 ioapic_i8259.apic == -1 )
+            {
+                ASSERT(ioapic_i8259.pin == -1);
                 ioapic_i8259.apic = apic;
                 ioapic_i8259.pin  = pin;
-                goto found_i8259;
             }
         }
     }
- found_i8259:
     /* Look to see what if the MP table has reported the ExtINT */
     /* If we could not find the appropriate pin by looking at the ioapic
      * the i8259 probably is not connected the ioapic but give the

@SergiiDmytruk
Copy link
Member

Added that patch and Xen booted. In case you want to see the log (keeping debug prints made it rather long and GitHub has 64KiB limit): xen.log

@royger
Copy link
Contributor

royger commented Dec 16, 2024

Thanks for testing, I would like to add Reported-by: Sergii Dmytruk <[email protected]> to the patch, is that accurate, and are you OK with it?

@SergiiDmytruk
Copy link
Member

SergiiDmytruk commented Dec 16, 2024

Thanks for testing, I would like to add Reported-by: Sergii Dmytruk <[email protected]> to the patch, is that accurate, and are you OK with it?

No problem, the bug was getting in the way of my work. Yes, feel free to add Reported-by.

@andyhhp
Copy link

andyhhp commented Dec 17, 2024

Fixed in Xen staging as of this morning. https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=f38fd27c4ceadf7ec4e82e82d0731b6ea415c51e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants