Stop 0x139 (012424-6265-01.dmp):
Rich (BB code):
0: kd> .trap ffffb20ff756b480; knL
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=ffffb30059d60180 rbx=0000000000000000 rcx=0000000000000003
rdx=ffffe6019dc24680 rsi=0000000000000000 rdi=0000000000000000
rip=fffff80109c4ae3f rsp=ffffb20ff756b610 rbp=fffff80105d64180
r8=ffffe601905ec158 r9=0000000000000000 r10=ffffe601a06b4738
r11=ffffe60198b33080 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz na po nc
nt!KiWakeQueueWaiter+0x164fcf:
fffff801`09c4ae3f cd29 int 29h
*** Stack trace for last set context - .thread/.cxr resets it
# Child-SP RetAddr Call Site
00 ffffb20f`f756b610 fffff801`09acae90 nt!KiWakeQueueWaiter+0x164fcf << Crash here!
01 ffffb20f`f756b670 fffff801`09a7e681 nt!KeInsertQueueEx+0x1a0 << NULL Queue Entry
02 ffffb20f`f756b6e0 fffff801`09f401e6 nt!IopCompleteRequest+0x601
03 ffffb20f`f756b7b0 fffff801`09f27137 nt!IopSynchronousServiceTail+0x356
04 ffffb20f`f756b860 fffff801`09f26d83 nt!IopWriteFile+0x267
05 ffffb20f`f756b980 fffff801`09c2bbe8 nt!NtWriteFile+0x263
06 ffffb20f`f756ba70 00007ff9`8e56f474 nt!KiSystemServiceCopyEnd+0x28
07 000000da`65dfdcf8 00000000`00000000 0x00007ff9`8e56f474
Opera initiates a write request (IRP_MJ_WRITE) which is then completed, this I/O request is synchronous, so the calling thread (Opera) is placed in a wait state and waits to be told that the file write has completed. This is what the KiWakeQueueWaiter is doing. However, when it attempts to read the next queue entry, to check which thread should be woken, it can't because it can't due to the corrupt list entry.
Rich (BB code):
0: kd> dt _LIST_ENTRY ffffe601a06b4788
nt!_LIST_ENTRY
[ 0x00000000`00000000 - 0x00000000`00000000 ]
+0x000 Flink : (null)
+0x008 Blink : (null)
This is why the Stop 0x139 bugcheck gets raised.
Stop 0x96 (012424-6328-01.dmp):
Rich (BB code):
0: kd> knL
# Child-SP RetAddr Call Site
00 fffffd8e`121aa448 fffff800`0d8eb3c3 nt!KeBugCheckEx
01 fffffd8e`121aa450 fffff800`0d8e9d48 nt!KeRemoveQueueEx+0x1433
02 fffffd8e`121aa810 fffff800`0dda9e6e nt!IoRemoveIoCompletion+0x98
03 fffffd8e`121aa930 fffff800`0da2bbe8 nt!NtRemoveIoCompletionEx+0xfe
04 fffffd8e`121aaa70 00007ffb`578f2304 nt!KiSystemServiceCopyEnd+0x28
05 00000001`0061f178 00000000`00000000 0x00007ffb`578f2304
This is very similar to the Stop 0x139 bugcheck, however, instead this is an asynchronous I/O request so it gets queued up to a I/O completion port (basically another queue) but instead of a specific thread, any arbitrary thread will pick up an I/O item from the queue and then process it. Now, instead of pushing to the queue, we're removing from the queue but the reason for the crash is the same: a corrupt _LIST_ENTRY.
Rich (BB code):
0: kd> dt _LIST_ENTRY ffff85891488e380
nt!_LIST_ENTRY
[ 0x00000000`00000000 - 0xffff8589`103b2dd8 ]
+0x000 Flink : (null)
+0x008 Blink : 0xffff8589`103b2dd8 _LIST_ENTRY [ 0xffff8589`1488e380 - 0xffff8589`1488e380 ]
Stop 0xA (012424-6203-01):
Rich (BB code):
2: kd> knL
# Child-SP RetAddr Call Site
00 ffffc803`bef39ee8 fffff805`0842c4e9 nt!KeBugCheckEx
01 ffffc803`bef39ef0 fffff805`08427a34 nt!KiBugCheckDispatch+0x69
02 ffffc803`bef3a030 fffff805`0826fa39 nt!KiPageFault+0x474
03 ffffc803`bef3a1c0 fffff805`0826779f nt!KiCommitRescheduleContextEntry+0x49
04 ffffc803`bef3a260 fffff805`08269a3d nt!KiDeferredReadySingleThread+0x99f
05 ffffc803`bef3a630 fffff805`082691fb nt!KiExitDispatcher+0x19d
06 ffffc803`bef3a9e0 fffff805`087a9359 nt!KeSetEvent+0xeb
07 ffffc803`bef3aa70 fffff805`0842bbe8 nt!NtSetEvent+0xb9
08 ffffc803`bef3aae0 00007fff`ccb0f534 nt!KiSystemServiceCopyEnd+0x28
09 000000d0`f21ff238 00000000`00000000 0x00007fff`ccb0f534
This bugcheck is due an illegal page fault but if you examine the call stack leading up to the crash, then it you will see it involves thread scheduling; again, we very likely crash due to a corrupt list entry. This more difficult to prove though with just a Minidump since many of the structures do not parse correctly.
Rich (BB code):
2: kd> dt _KPRCB -y Deferred
nt!_KPRCB
+0x2d08 DeferredReadyListHead : _SINGLE_LIST_ENTRY
It is very likely that the thread was being inserted into this list.
Stop 0xF7 (012424-6187-01.dmp):
Rich (BB code):
2: kd> knL
# Child-SP RetAddr Call Site
00 fffffc84`583b8858 fffff802`228fb0f5 nt!KeBugCheckEx
01 fffffc84`583b8860 fffff802`226e9ed9 nt!_report_gsfailure+0x25
02 fffffc84`583b88a0 fffff802`22b3f65e nt!IoRemoveIoCompletion+0x229
03 fffffc84`583b89c0 fffff802`2282bbe8 nt!NtRemoveIoCompletion+0x13e
04 fffffc84`583b8a70 00007ff8`a5ccf494 nt!KiSystemServiceCopyEnd+0x28
05 00000003`7b7feea8 00007ff8`a3423f83 0x00007ff8`a5ccf494
06 00000003`7b7feeb0 00000000`00000000 0x00007ff8`a3423f83
It seems we encountered a buffer overrun while processing another queue and this was running in the context of Opera.
Rich (BB code):
2: kd> !dpx
Start memory scan : 0xfffffc84583b8858 ($csp)
End memory scan : 0xfffffc84583b9000 (Kernel Stack Base)
rsp : 0xfffffc84583b8858 : 0xfffff802228fb0f5 : nt!_report_gsfailure+0x25
0xfffffc84583b8858 : 0xfffff802228fb0f5 : nt!_report_gsfailure+0x25
0xfffffc84583b8898 : 0xfffff802226e9ed9 : nt!IoRemoveIoCompletion+0x229
0xfffffc84583b88d8 : 0xfffff80222b42417 : nt!ObpReferenceObjectByHandleWithTag+0x267
0xfffffc84583b8968 : 0xfffff80222b4219e : nt!ObReferenceObjectByHandle+0x2e
0xfffffc84583b89b8 : 0xfffff80222b3f65e : nt!NtRemoveIoCompletion+0x13e
0xfffffc84583b89d8 : 0xfffff80222b278a3 : nt!NtReadFile+0xd3
0xfffffc84583b8a68 : 0xfffff8022282bbe8 : nt!KiSystemServiceCopyEnd+0x28
0xfffffc84583b8ad8 : 0xfffff8022282bbe8 : nt!KiSystemServiceCopyEnd+0x28
0xfffffc84583b8ae0 : 0xffffc102acd9f080 : Trap @ fffffc84583b8ae0
Stop 0x139 (012424-6093-01.dmp):
Rich (BB code):
2: kd> knL
# Child-SP RetAddr Call Site
00 ffffc288`9e18df78 fffff805`1dc2c4e9 nt!KeBugCheckEx
01 ffffc288`9e18df80 fffff805`1dc2cab2 nt!KiBugCheckDispatch+0x69
02 ffffc288`9e18e0c0 fffff805`1dc2a806 nt!KiFastFailDispatch+0xb2
03 ffffc288`9e18e2a0 fffff805`1da6efee nt!KiRaiseSecurityCheckFailure+0x346
04 ffffc288`9e18e430 fffff805`1daeb0ad nt!KiCommitThreadWait+0x56e
05 ffffc288`9e18e4e0 fffff805`1dae9d48 nt!KeRemoveQueueEx+0x111d
06 ffffc288`9e18e8a0 fffff805`1df3f65e nt!IoRemoveIoCompletion+0x98
07 ffffc288`9e18e9c0 fffff805`1dc2bbe8 nt!NtRemoveIoCompletion+0x13e
08 ffffc288`9e18ea70 00007fff`8e54f494 nt!KiSystemServiceCopyEnd+0x28
09 00000014`731fefc8 00000000`00000000 0x00007fff`8e54f494
This Stop 0x139 is almost identical to the other Stop 0x139 which we encountered earlier. This seems to occur on the back of another read request from Opera. This is likely from the back of the queue removal and then attempting to update the WaitListEntry of the Opera thread.
Rich (BB code):
2: kd> dt _KTHREAD -y WaitListEntry
nt!_KTHREAD
+0x0d8 WaitListEntry : _LIST_ENTRY
With the latest bugcheck (Stop 0x1), APCs are very often used with thread scheduling and I/O request completion.