Ubuntu 22.04 takes over a minute to wake from deep suspend

After looking at the dmesg, I can see that it looks like the cause of the delay is the PCIEport 0000:20:00:0 which is not ready.

running lspci -s 20:00.0 shows:

20:00.0 PCI bridge: Intel Corporation Thunderbolt 4 Bridge [Maple Ridge 4C 2020] (rev 02)

Result of dmesg:

[   27.942301] ACPI: PM: Waking up from system sleep state S3[   33.905213] ACPI: EC: interrupt unblocked[   34.946828] pcieport 0000:00:1d.0: broken device, retraining non-functional downstream link at 2.5GT/s[   35.954833] pcieport 0000:00:1d.0: retraining failed[   37.206830] pcieport 0000:00:1d.0: broken device, retraining non-functional downstream link at 2.5GT/s[   38.214812] pcieport 0000:00:1d.0: retraining failed[   38.214829] pcieport 0000:20:00.0: not ready 1023ms after resume; waiting[   39.254820] pcieport 0000:20:00.0: not ready 2047ms after resume; waiting[   41.334816] pcieport 0000:20:00.0: not ready 4095ms after resume; waiting[   45.686816] pcieport 0000:20:00.0: not ready 8191ms after resume; waiting[   54.134821] pcieport 0000:20:00.0: not ready 16383ms after resume; waiting[   70.774829] pcieport 0000:20:00.0: not ready 32767ms after resume; waiting[  105.590814] pcieport 0000:20:00.0: not ready 65535ms after resume; giving up[  105.591021] pcieport 0000:20:00.0: Unable to change power state from D3cold to D0, device inaccessible[  105.591264] pcieport 0000:21:02.0: Unable to change power state from D3cold to D0, device inaccessible[  105.591287] pcieport 0000:21:01.0: Unable to change power state from D3cold to D0, device inaccessible[  105.591331] pcieport 0000:21:00.0: Unable to change power state from D3cold to D0, device inaccessible[  105.591358] pcieport 0000:21:03.0: Unable to change power state from D3cold to D0, device inaccessible[  105.591442] thunderbolt 0000:22:00.0: Unable to change power state from D3cold to D0, device inaccessible[  105.591522] xhci_hcd 0000:56:00.0: Unable to change power state from D3cold to D0, device inaccessible[  105.654524] ACPI: EC: event unblocked

This is a Thinkpad X1 Extreme gen 5 with kernel 6.5.0-1020-oem.
I am stumped what to do next. Anyone have any ideas?