Now I've got some ideas.
It seems that some PCIe cards can't deal with 3.3V power well.
From the log above, we know that kernel disabled the fixed-regulator vcc3v3_pcie, as it was unused after PCIe probe failure. When I manually reloaded the module, the regulator got enabled instantly (PCIe device getting its power), and PCIe link training went on smoothly.
It leads me to think:
Some PCIe devices might require immediate link training after getting 3.3V power, otherwise it fails to probe.
AFAIK vcc3v3_pcie is enabled much earlier than the PCIe controller driver gets loaded, which might be the cause of the problem.
So, is there a way to enable that regulator only when needed? (when the PCIe driver gets loaded)