Description
BGC allow_fgc does not yield to the FGC in nativeaot builds resulting in the execution enviroment stalling for a horrifically long time
Affects all nativeaot builds
Configuration
CoreCLR
NativeAOT
.NET 10 (and is still an issue in main as of commit 9803759)
Regression?
BGC does not yield to FGC in nativeaot, unknown if this is a regression or simply never fixed since nativeaot's creation
Data
~65ms, of which ~60ms is spent waiting for the EE to suspend
(all other cooperative threads have been suspended. only the BGC is still executing)
Analysis
The cause of this is due to allow_fgc using a condition that is never true in NativeAOT GCToEEInterface::SuspendEE
|
if (g_fSuspensionPending > 0) |
g_fSuspensionPending > 0 is only updated via
GCHeap::SetSuspensionPending which at present is only ever used by ThreadStore::SetThreadTrapForSuspension which is not used by NativeAOT gcee
when an FGC is run on nativeaot it invokes this method
|
void GCToEEInterface::SuspendEE(SUSPEND_REASON reason) |
|
{ |
|
#ifdef FEATURE_EVENT_TRACE |
|
ETW::GCLog::ETW_GC_INFO Info; |
|
Info.SuspendEE.Reason = reason; |
|
Info.SuspendEE.GcCount = (((reason == SUSPEND_FOR_GC) || (reason == SUSPEND_FOR_GC_PREP)) ? |
|
(uint32_t)GCHeapUtilities::GetGCHeap()->GetGcCount() : (uint32_t)-1); |
|
#endif // FEATURE_EVENT_TRACE |
|
|
|
FireEtwGCSuspendEEBegin_V1(Info.SuspendEE.Reason, Info.SuspendEE.GcCount, GetClrInstanceId()); |
|
|
|
GetThreadStore()->LockThreadStore(); |
|
GCHeapUtilities::GetGCHeap()->SetGCInProgress(TRUE); |
|
GetThreadStore()->SuspendAllThreads(true); |
|
|
|
FireEtwGCSuspendEEEnd_V1(GetClrInstanceId()); |
|
} |
Which does not update, set or increment
g_fSuspensionPending in any way causing the
SuspendAllThreads method to spin wait until the BGC has either completed and exits or finished its phase and tries to acquire the spinlock (
|
enter_spin_lock(&gc_lock); |
such as here) which internally will swap from cooperative to preemptive and back allowing it to suspend
Note this is not an issue in a non nativeAOT enviroment as the coreclr gcenv uses the ThreadSuspend::SuspendEE method (
|
void GCToEEInterface::SuspendEE(SUSPEND_REASON reason) |
|
{ |
|
WRAPPER_NO_CONTRACT; |
|
|
|
static_assert(SUSPEND_FOR_GC == (int)ThreadSuspend::SUSPEND_FOR_GC); |
|
static_assert(SUSPEND_FOR_GC_PREP == (int)ThreadSuspend::SUSPEND_FOR_GC_PREP); |
|
|
|
_ASSERTE(reason == SUSPEND_FOR_GC || reason == SUSPEND_FOR_GC_PREP); |
|
|
|
if (g_pDebugInterface) |
|
g_pDebugInterface->SuspendForGarbageCollectionStarted(); |
|
|
|
ThreadSuspend::SuspendEE((ThreadSuspend::SUSPEND_REASON)reason); |
|
|
|
if (g_pDebugInterface) |
|
g_pDebugInterface->SuspendForGarbageCollectionCompleted(); |
|
} |
) which does update
g_fSuspensionPending
This results in situations where the entire EE is suspended for a very extended period of time for no reason (waiting on the BGC which will never yield)
Worst case this can cause the EE to stall for 300+ milliseconds depending on the phase size and speed of the BGC
One such way to possibly fix this is to replace the condition g_fSuspensionPending > 0 inside the bgc with g_fSuspensionPending > 0 || GCHeapUtilities::IsGCInProgress() which will allow it to yield to the FGC when the EE is being suspended
Alternativly nativeaot GCToEEInterface::SuspendEE invoke ThreadStore::SetThreadTrapForSuspension(); on suspend and UnsetThreadTrapForSuspension on resume
Description
BGC allow_fgc does not yield to the FGC in nativeaot builds resulting in the execution enviroment stalling for a horrifically long time
Affects all nativeaot builds
Configuration
CoreCLR
NativeAOT
.NET 10 (and is still an issue in main as of commit 9803759)
Regression?
BGC does not yield to FGC in nativeaot, unknown if this is a regression or simply never fixed since nativeaot's creation
Data
Analysis
The cause of this is due to allow_fgc using a condition that is never true in NativeAOT GCToEEInterface::SuspendEE
runtime/src/coreclr/gc/background.cpp
Line 1228 in 9803759
g_fSuspensionPending > 0is only updated viaGCHeap::SetSuspensionPendingwhich at present is only ever used by ThreadStore::SetThreadTrapForSuspension which is not used by NativeAOT gceewhen an FGC is run on nativeaot it invokes this method
runtime/src/coreclr/nativeaot/Runtime/gcenv.ee.cpp
Lines 37 to 53 in 9803759
Which does not update, set or increment
g_fSuspensionPendingin any way causing theSuspendAllThreadsmethod to spin wait until the BGC has either completed and exits or finished its phase and tries to acquire the spinlock (runtime/src/coreclr/gc/background.cpp
Line 2562 in 9803759
Note this is not an issue in a non nativeAOT enviroment as the coreclr gcenv uses the ThreadSuspend::SuspendEE method (
runtime/src/coreclr/vm/gcenv.ee.cpp
Lines 38 to 54 in 9803759
g_fSuspensionPendingThis results in situations where the entire EE is suspended for a very extended period of time for no reason (waiting on the BGC which will never yield)
Worst case this can cause the EE to stall for 300+ milliseconds depending on the phase size and speed of the BGC
One such way to possibly fix this is to replace the condition
g_fSuspensionPending > 0inside the bgc withg_fSuspensionPending > 0 || GCHeapUtilities::IsGCInProgress()which will allow it to yield to the FGC when the EE is being suspendedAlternativly nativeaot
GCToEEInterface::SuspendEEinvokeThreadStore::SetThreadTrapForSuspension();on suspend andUnsetThreadTrapForSuspensionon resume