Tài liệu Memory Dump Analysis Anthology- P4 pdf

The saved processor state information context forms the so called Windows kernel trap frame: This Windows trap frame is not the same as an interrupt frame a processor saves on the curr

So finally we get our stack trace:

TRAP COMMAND ON X86

Now I explain WinDbg trap command and show how to simulate it manually

Upon an interrupt a processor saves the current instruction pointer and transfers

execution to an interrupt handler as explained in x86 Interrupts article (page 69) This

interrupt handler has to save full thread context before calling other functions to do

complex interrupt processing For example, if we disassemble KiTrap0E handler from

x86 Windows 2003 crash dump we would see that it saves a lot of registers including

e088bb2c mov word ptr [esp+2],0

e088bb33 push ebp

e088bb34 push ebx

e088bb35 push esi

e088bb36 push edi

e088bb37 push fs

e088bb39 mov ebx,30h

e088bb3e mov fs,bx

e088bb41 mov ebx,dword ptr fs:[0]

e088bb48 push ebx

e088bb49 sub esp,4

e088bb4c push eax

e088bb4d push ecx

e088bb4e push edx

e088bb4f push ds

e088bb50 push es

e088bb51 push gs

e088bb53 mov ax,23h

e088bb57 sub esp,30h

e088bb5a mov ds,ax

e088bb5d mov es,ax

e088bb60 mov ebp,esp

e088bb62 test dword ptr [esp+70h],20000h

e088bb6a jne nt!V86_kite_a (e088bb04)

The saved processor state information (context) forms the so called Windows

kernel trap frame:

This Windows trap frame is not the same as an interrupt frame a processor saves

on the current thread stack when an interrupt occurs in kernel mode The latter frame is

very small and consists only of EIP, CS, EFLAGS and ErrorCode When an interrupt occurs

in user mode an x86 processor additionally saves the current stack pointer SS:ESP

The trap command finds the trap frame on the current thread stack and sets the

current thread register context using the values from that saved structure We can see

that command in action for certain bugchecks when we use !analyze –v command:

3: kd> !analyze -v

KERNEL_MODE_EXCEPTION_NOT_HANDLED (8e)

Arguments: Arg1: c0000005, The exception code that was not handled Arg2: de65190c, The address that the exception occurred at Arg3: f24f8a74, Trap Frame Arg4: 00000000 … … … TRAP_FRAME: f24f8a74 — (.trap fffffffff24f8a74) trap fffffffff24f8a74 ErrCode = 00000000 eax=dbc128c0 ebx=dbe4a010 ecx=f24f8ac4 edx=00000001 esi=46525356 edi=00000000 eip=de65190c esp=f24f8ae8 ebp=f24f8b18 iopl=0 nv up ei pl nz na pe nc cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010206 driver!foo+0×16: de65190c 837e1c00 cmp dword ptr [esi+1Ch],0 ds:0023:46525372=???????? … … … If we look at the trap frame we would see the same register values that WinDbg reports above: 3: kd> dt _KTRAP_FRAME f24f8a74 +0x000 DbgEbp : 0xf24f8b18 +0x004 DbgEip : 0xde65190c +0x008 DbgArgMark : 0xbadb0d00 +0x00c DbgArgPointer : 1

+0x010 TempSegCs : 0xb0501cd +0x014 TempEsp : 0xdcc01cd0 +0x018 Dr0 : 0xf24f8aa8 +0x01c Dr1 : 0xde46c90a +0x020 Dr2 : 0

+0x024 Dr3 : 0

+0x028 Dr6 : 0xdbe4a000 +0x02c Dr7 : 0

+0x030 SegGs : 0

+0x034 SegEs : 0x23

+0x038 SegDs : 0x23

+0x03c Edx : 1

+0x040 Ecx : 0xf24f8ac4 +0x044 Eax : 0xdbc128c0 +0x048 PreviousPreviousMode : 0xdbe4a010 +0x04c ExceptionList : 0xffffffff _EXCEPTION_REGISTRATION_RECORD +0x050 SegFs : 0x30

It is good to know how to find a trap frame manually when the stack is corrupt or

WinDbg cannot find a trap frame automatically In this case we can take the advantage

of the fact that DS and ES segment registers have the same value in Windows flat

mem-ory model:

+0x034 SegEs : 0x23

+0x038 SegDs : 0x23

We need to find 2 consecutive 0×23 values on the stack There may be several

such places but usually the correct one comes between KiTrapXX address on the stack

and the initial processor trap frame shown below in bold This is because KiTrapXX

ob-viously calls other functions to further process an interrupt so its return address is saved

Subtracting the offset 0×38 from the address of the 00000023 value (f24f8aac)

and using dt command we can check _KTRAP_FRAME structure and apply trap

com-mand afterwards:

3: kd> dt _KTRAP_FRAME f24f8aac-38

+0x000 DbgEbp : 0xf24f8b18

+0x004 DbgEip : 0xde65190c

+0x008 DbgArgMark : 0xbadb0d00

+0x00c DbgArgPointer : 1

+0x010 TempSegCs : 0xb0501cd +0x014 TempEsp : 0xdcc01cd0 +0x018 Dr0 : 0xf24f8aa8 +0x01c Dr1 : 0xde46c90a +0x020 Dr2 : 0

+0x024 Dr3 : 0

+0x028 Dr6 : 0xdbe4a000 +0x02c Dr7 : 0

+0x030 SegGs : 0

+0x034 SegEs : 0x23

+0x038 SegDs : 0x23

+0x03c Edx : 1

+0x040 Ecx : 0xf24f8ac4 +0x044 Eax : 0xdbc128c0 +0x048 PreviousPreviousMode : 0xdbe4a010 +0x04c ExceptionList : 0xffffffff _EXCEPTION_REGISTRATION_RECORD +0x050 SegFs : 0x30

+0x054 Edi : 0

+0x058 Esi : 0x46525356

+0x05c Ebx : 0xdbe4a010 +0x060 Ebp : 0xf24f8b18 +0x064 ErrCode : 0

+0x068 Eip : 0xde65190c +0x06c SegCs : 8

+0x070 EFlags : 0x10206

+0x074 HardwareEsp : 0xdbc171b0 +0x078 HardwareSegSs : 0xde667677 +0x07c V86Es : 0xdbc128c0 +0x080 V86Ds : 0xdbc171c4 +0x084 V86Fs : 0xf24f8bc4 +0x088 V86Gs : 0

3: kd> ? f24f8aac-38

Evaluate expression: -229668236 = f24f8a74

In complete memory dumps we can see that _KTRAP_FRAME is saved system

ser-vices are called too:

f24f8d54 7c94ed54 nt!KiFastCallEntry+0xfc (TrapFrame @ f24f8d64)

0006e48c 77e34f1d ntdll!KiFastSystemCallRet

0006e53c 77e2f12f USER32!NtUserShowWindow+0xc

0006e570 77e2b0fe USER32!InternalDialogBox+0xa9

0006e590 77e29005 USER32!DialogBoxIndirectParamAorW+0×37

3: kd> kL

ChildEBP RetAddr

0006e48c 77e34f1d ntdll!KiFastSystemCallRet

0006e53c 77e2f12f USER32!NtUserShowWindow+0xc

0006e570 77e2b0fe USER32!InternalDialogBox+0xa9

0006e590 77e29005 USER32!DialogBoxIndirectParamAorW+0x37

0006e5b4 0103d569 USER32!DialogBoxParamW+0x3f

0006e5d8 0102d2f5 winlogon!Fusion_DialogBoxParam+0x24

TRAP COMMAND ON X64

Now I show how to simulate trap WinDbg command when we have x64

Win-dows kernel and complete memory dumps

When we have a fault an x64 processor saves some registers on the current

thread stack as explained in x64 Interrupts article (page 76) Then an interrupt handler

saves _KTRAP_FRAME on the stack:

Unfortunately the technique to use DS and ES pair to find the trap frame in x86

Windows crash dump doesn’t work here because KiPageFault interrupt handler doesn’t

save them as can be found by inspecting its disassembly Fortunately the registers that

an x64 processor pushes upon an interrupt are part of _KTRAP_FRAME shown in bold

above Fill1, Fill2, Fill3 and CodePatchCycle are just dummy values to fill 64-bit slots

be-cause CS and SS are 16-bit registers and in 64-bit RFLAGS only the first 32-bit EFLAGS

part is currently used Remember that a processor in 64-bit mode pushes 64-bit values

even if values occupy only 16 or 32-bit Therefore we can try to find CS and SS on the

stack because they have the following constant values:

Now we can calculate the trap frame address by subtracting SegSs offset in

_KTRAP_FRAME structure (0×188) from fffffadc`6e02cac8 address:

6: kd> ? fffffadc`6e02cac8-188

Evaluate expression: -5650331285184 = fffffadc`6e02c940

6: kd> trap fffffadc`6e02c940

NOTE: The trap frame does not contain all registers

Some register values may be zeroed

Our example shows how to find a trap frame manually in x64 kernel or complete

memory dump Usually WinDbg finds trap frames automatically (call arguments are

re-moved from verbose stack trace for clarity):

fffffadc`6e02c1c0 fffff800`0102e76f nt!KiDispatchException+0xd9

fffffadc`6e02c7c0 fffff800`0102d5e1 nt!KiExceptionExit

fffffadc`6e02c940 fffff97f`ff591ed3 nt!KiPageFault+0x1e1 (TrapFrame @

EXCEPTIONS IN USER MODE

Previous articles were dealing with exceptions in kernel mode Now I’m going to

investigate the flow of exception processing in user mode In x86 Interrupts article

(page 69) I mentioned that interrupts and exceptions generated when CPU is in user

mode require a processor to switch the current user mode stack to kernel mode

stack This can be seen when we have a user debugger attached and it gets an exception

notification called first chance exception Because of the stack switch we don’t see any

saved processor context on user mode thread stack when WinDbg breaks on

first-chance exception in TestDefaultDebugger64 (module name is renamed to TDD64 for

We see that there are no saved SS:RSP, RFLAGS, CS:RIP registers which we see on

a stack if an exception happens in kernel mode as shown in x64 Interrupt article (page

76) If we bugcheck our system using SystemDump tool to generate complete memory

dump at that time we can look later at the whole thread that experienced exception in

user mode and its user mode and kernel mode stacks:

kd> !process fffffadfe7055c20 2

PROCESS fffffadfe7055c20

SessionId: 0 Cid: 0c64 Peb: 7fffffd7000 ParentCid: 07b0

DirBase: 27e3d000 ObjectTable: fffffa800073a550 HandleCount: 55

Image: TDD64.exe

THREAD fffffadfe78f2bf0 Cid 0c64.0c68 Teb: 000007fffffde000

Win32Thread: fffff97ff4d71010 WAIT: (Unknown) KernelMode Non-Alertable

SuspendCount 1

fffffadfdf7b6fc0 SynchronizationEvent

THREAD fffffadfe734c3d0 Cid 0c64.0c88 Teb: 000007fffffdc000

Win32Thread: 0000000000000000 WAIT: (Unknown) KernelMode Non-Alertable

SuspendCount 1

FreezeCount 1

fffffadfe734c670 Semaphore Limit 0x2

kd> thread /r /p fffffadfe78f2bf0

Implicit thread is now fffffadf`e78f2bf0

Implicit process is now fffffadf`e7055c20

Loading User Symbols

Dumping kernel mode stack of our thread shows that the processor

saved registers there:

fffffadf`df7b7dc8 00000000`00000111 ; RBP saved by KiPageFault

fffffadf`df7b7dd0 00000000`00000006 ; Page-Fault Error Code

Error code 6 is 110 in binary and volume 3A of Intel manual tells us that “the fault

was caused by a non-present page” (bit 0 is cleared), “the access causing the fault was a

write” (bit 1 is set) and “the access causing the fault originated when the processor was

executing in user mode” (bit 2 is set)

HOW TO DISTINGUISH BETWEEN 1ST AND 2ND CHANCES

Sometimes we look for Early Crash Dump pattern (page 465) but information

about whether an exception was first-chance or second-chance is missing from a crash

dump file name or in a crash dump itself, for example:

This dump file has an exception of interest stored in it

The stored exception information can be accessed via ecxr

(1254.1124): Access violation - code c0000005 (first/second chance not

available)

TDD64!CTestDefaultDebuggerDlg::OnBnClickedButton1:

00000000`00401570 c704250000000000000000 mov dword ptr [0],0

ds:00000000`00000000=????????

If we recall that first-chance exceptions don’t leave any traces on user space

thread stacks (see Exceptions in User Mode, page 104, for details) then we won’t see

any exception codes on thread raw stack:

From raw stack data we can even tell when a crash dump was saved from a

debugger handling a second-chance exception or saved by a postmortem debugger

afterwards For example, on my Vista x64 I see the following difference:

Raw stack from a crash dump saved from WinDbg after receiving second-chance

Raw stack from a crash dump saved by CDB installed as a default postmortem

WHO CALLS THE POSTMORTEM DEBUGGER?

I was trying to understand who calls dwwin.exe (the part of Windows Error

Reporting on Windows XP) when a crash happens To find the answer I launched

TestDefaultDebugger application and after pushing its crash button the following

famil-iar WER dialog box appeared:

I repeated the same procedure while running ProcessHistory in the background

and from its log I found that the parent process for dwwin.exe and the postmortem

debugger (if we click on Debug button) was TestDefaultDebugger.exe In my case the

default postmortem debugger was drwtsn32.exe To dig further I attached WinDbg to

TestDefaultDebugger process when WER dialog box above was displayed and got the

following stack trace:

The combination of StartDWException and WaitForMultipleObjects suggests that

dwwin.exe process is started there Indeed, when I disassembled StartDWException

function I saw CreateProcess call just before the wait call:

694575a2 6a01 push 1

694575a4 50 push eax

694575a5 50 push eax

694575a6 ffb5a4f7ffff push dword ptr [ebp-85Ch] ; second parameter

694575ac 50 push eax ; first parameter

694575ad ff1558114569 call dword ptr [faultrep!_imp CreateProcessW

(69451158)]

…

…

…

The second parameter of CreateProcess, [ebp-85Ch], is the address of the

process command line and we know EBP value from the call stack above, 0012dd68, and

we get the command line straight away:

0:000> dpu 0012dd68-85Ch l1

0012d50c 0012d3ec "C:\WINDOWS\system32\dwwin.exe -x -s 208"

If we dismiss WER dialog by clicking on Debug button then a postmortem

debug-ger starts It also starts without WER dialog displayed if we rename faultrep.dll

before-hand Therefore the obvious place to look for postmortem debugger launch is

UnhandledExceptionFilter function Indeed, we see it there:

0:000> uf kernel32!UnhandledExceptionFilter

7c8636a8 8d850cfaffff lea eax,[ebp-5F4h]

7c8636ae 50 push eax

7c8636af 8d857cf9ffff lea eax,[ebp-684h]

7c8636b5 50 push eax

7c8636b6 33c0 xor eax,eax

7c8636be 50 push eax

7c8636bf 53 push ebx ; second parameter

7c8636c0 50 push eax ; first parameter

7c8636c1 e86cecf9ff call kernel32!CreateProcessW (7c802332)

…

…

…

Because this is the code that yet to be executed, we need to put a breakpoint at

7c8636c1 address, continue execution, and when the breakpoint is hit dump the second

parameter to CreateProcess that is the memory EBX points to:

If we look further at UnhandledExceptionFilter disassembled code we would see

that after creating a postmortem debugger process and waiting for it to finish saving the

process dump the function calls NtTerminateProcess

Therefore all error reporting, calling a postmortem debugger and final process

termination are done in the same process that had the exception The latter two

parts are also described in Matt Pietrek’s article:

http://www.microsoft.com/msj/0197/exception/exception.aspx

Starting from Windows XP UnhandledExceptionFilter function locates and loads

faultrep.dll which launches dwwin.exe to report an error:

We can also see that all processing is done using the same thread stack So if

something is wrong with that stack then you have silent process termination and no

error is reported In Vista there are some improvements that I’m going to cover in the

next article