Shellcode Hook Detection

Author: David Zimmer
Date: 05.15.12 - 7:43pm

Every now and then shellcode pops up that does hook detection.

Ok fine, scdbg doesnt use jmp hooks..but not so fine if the shellcode doesnt actually check for a hook, and just always replaces the api prolog and does a jmp api+5. This screws up scdbg because the hook address isnt hit and dll memory starts getting executed.

I pushed a test patch tonight to the github repository that aims to fix this. If it detects any api+5 address being executed, it assumes its a hook detection code running, and will pass execution back to a hookDetect call back which tries to restore the stack and allow the standard api logging code to run normally.

Tested it with several shellcodes and seems to be working fine.

A precompiled binary is available in the github archive you can download if you want to play with it. Of the thoughts I have had on how to handle this, this seems to be the best solution so far...

These types of hook evasion routines will also cause a crash in sclog. Initially i switched to a 7 byte push const ret hook so I wouldnt trigger the hook check, but if they jmp +5 regardless its a crash. For sclog, I guess what I should do is pad the beginning of the hook a bit so that jmp +5 lands them squarely on my jmp hookthunk. That would be the best of both worlds.

Update: I tried the 7 byte hook, it worked, but not all api have enough room to embed it. Its really best to have a prolog pad that is devoid of instructions like jumps and calls to slip in a hook. I could have it selectivly fall back to original style hooks, but in the end i just reverted to standard detours style hooks for simplicity and stability.

One additional tip, if you get a shellcode which starts executing dll opcodes and then crashs in scdbg. You can find the point of transfer to dll memory quite quickly with the following procedure:
  • run sample again -ba 7c800000. this will break immediately when it tries to execute any dll memory. Copy the step count displayed.
  • run sample again -bs [step count-x], this will break x steps before the dll memory was first executed showing you exactly how it got there.
You can also dump memory, disassemble and analyze in ida from there too if need be.

Additionally if the problem is when executing null memory, there is a special switch to break on that, replace step 1 with the -b0 option.

Couple examples for reference:

seg000:0000019F HookCheck       proc near                
seg000:0000019F                 cmp     byte ptr [eax], 0E8h ; F
seg000:000001A2                 jz      short loc_1AE
seg000:000001A4                 cmp     byte ptr [eax], 0E9h ; T
seg000:000001A7                 jz      short loc_1AE
seg000:000001A9                 cmp     byte ptr [eax], 0EBh ; d
seg000:000001AC                 jnz     short loc_1BF
seg000:000001AE loc_1AE:                                
seg000:000001AE                 cmp     dword ptr [eax+5], 90909090h
seg000:000001B5                 jz      short loc_1BF
seg000:000001B7                 mov     edi, edi
seg000:000001B9                 push    ebp
seg000:000001BA                 mov     ebp, esp
seg000:000001BC                 lea     eax, [eax+5]
seg000:000001BF loc_1BF:                                
seg000:000001BF                 jmp     eax
seg000:000001BF HookCheck       endp  

.text:00401050 JumpIntoLoadLibraryA .text:00401050 push ebx .text:00401051 push dword ptr [esi+34h] ;fake return address in k32 (scanned) .text:00401054 mov edi, edi ;implement its own prolog .text:00401056 push ebp .text:00401057 mov ebp, esp .text:00401059 mov edx, [esi] ; LoadLibraryA from api table .text:0040105B lea edx, [edx+5] ; +5 .text:0040105E jmp edx .text:0040105E JumpIntoLoadLibraryA endp

And the cleanup code: int HookDetector(char* fxName){ printf(" jmp %s+5 hook evasion code detected! trying to recover...\n", fxName); cpu->reg[esp] = cpu->reg[ebp]; cpu->reg[ebp] = popd(); return 1; }

Comments: (0)

Leave Comment:
Email: (not shown)
Message: (Required)
Math Question: 18 + 33 = ? followed by the letter: J 

About Me
More Blogs
Main Site
Yara WorkBench
vbdec dbg updates
vb6 PCode NOP
vb6 API and call backs
how pcode works Pt1
Reversing PCode Args
VB6 PCode Disassembly
VB6 PCode Debugger
UConnect Disable Cell Modem
IDA python over IPC
dns wildcard blocking
64bit IDA Plugins
anterior lines
misc news/updates
Decoders again
CDO.Message Breakpoints
SysAnalyzer Updates
SysAnalyzer and Site Updates
crazy decoder
ida js w/dbg
flash patching #2
JS Graphing
packet reassembly
Delphi IDA Plugin
scdbg IDA integration
API Hash Database
Winmerge plugin
IDACompare Updates
Guest Post @ hexblog
TCP Stream Reassembly
SysAnalyzer Updates
Apilogger Video
Shellcode2Exe trainer
scdbg updates
IDA Javascript w/IDE
Rop Analysis II
scdbg vrs ROP
flash patching
x64 Hooks
micro hook
jmp api+5 *2
SysAnalyzer Updates
InjDll runtime config
C# Asm/Dsm Library
Shellcode Hook Detection
Updates II
Java Hacking
Windows 8
Win7 x64
Graphing ideas
.Net Hacking
Old iDefense Releases
hll shellcode
ActionScript Tips
-patch fu
scdbg ordinal lookup
scdbg -api mode
Peb Module Lists
scdbg vrs Process Injection
GetProcAddress Scanner
scdbg fopen mode
scdbg findsc mode
scdbg MemMonitor
demo shellcodes
scdbg download
api hashs redux
Api hash gen
Retro XSS Chat Codes
Exe as DLL
Olly Plugins
Debugging Explorer
Attach to hidden process
JS Refactoring
Asm and Shellcode in CSharp
Fancy Return Address
PDF Stream Dumper
Malcode Call API by Hash
WinDbg Cheat Sheet
GPG Automation