64bit IDA Plugins

Author: David Zimmer
Date: 10.16.17 - 7:48am

So as we know IDA is going to primarily be a x64 binary now.

Bit of a hiccup for me since I use VB6 for my gui's. With IDAJscript the IDE/script engine actually run out of process and use interprocess communication (IPC) to talk to a small plugin stub, IDASRVR, that runs within IDA.

IDASRVR is going to be my escape path I suppose. I will have to convert IDACompare and others to run fully out of process which is fine. More stable for me to have my code insulated from API changes in IDA anyway.

So I started on the conversion of IDASRVR to first bring in 64bit p64 support which it didnt have yet (while keeping it a 32bit compile). This had some interesting bits.

My quickcall mechanism went almost completely out the window for x64. Using only sendmessage I was able to pass in two 32bit arguments and receive a 32bit return value. Well thats out for anything that takes a 64bit address. (If I was an x64 calling process it would probably still be ok but I am not)

Ultimately I want to get my IDAJScript running with the new IDA. So I have 3 layers to get working. First my IDASRVR plugin stub running in IDA, second my VB6 layer, and third its integration with the Duktape javascript engine.

So apparently Javascript does not support precise 64 bit numbers. You need to use a Bigint library for that. I didnt see that coming. Duktape was a perfect choice for a script engine with its nice tight size (1mb) and its debugger protocol. It took me like 3 mos of hobby time to get that debugger working the way I wanted and another 3 mos to get the Scintilla IDE portion where I wanted it. It was a big project to get just those components working.

VB6 doesnt support 64 bit numbers natively either so here is the plan as best I see it right now.

I will port IDASRVR to work with the 64bit IDA. It will support plw and p64 and run in process. All my other plugins, at least those worth porting (At least IDACOmpare and IDAJScript) will run fully out of process and only deal with IDA through the IDASRVR layer. Developing plugins in process is no fun anyway, out of process plugin interface is way better on so many levels. For lack of a better term I guess I will call these plugouts.

There will be slight differences between the 32bit and 64bit api use. Since neither vb6 or js support 64 bit numbers I will have to use a BigInt class to deal with them.

I could just convert fully and use only 64 bit addresses in the vb6 and js client classes...but using the BigInt class is going to add some annoying extra steps for quick scripts, and honestly 95% of the time I am working on 32bit disassemblies still.

So the plan is I am going to keep my 32bit IDAClient class unchanged (good already debugged and stable and wont break existing code). For working with 64bit disassemblies, I will add a IDAClient64 class whos exposed interface is almost identical, except that any arguments or return values which work with addresses now use a ULong64 class type. This vb6 class for BinInt support will be usable on the JS layer as well killing two birds with one stone.

Down side, 64bit and 32bit scripts will not be compatible, although you can write one script that works with both. This is annoying but I am dealing with multiple layers of incompatibilities so it kind of is what it is. Plus I dont do that much work with 64bit disassemblies anyway so.

64bit transition in general has been annoying for me. This also includes having to deal with Windows file system/registry redirection and COM registration concerns.

Aghh yes complexities abound and multiply. The poor human mind drowning on its current technological path. A couple relevant examples from todays news here, and here. It appears that today us is not smart enough for tomorrow us, or maybe tomorrow us just doesnt like today us that much :)

Comments: (2)

On 10.16.17 - 9:32am Dave wrote:
Some people might ask why i bother..I could just save myself a ton of work and use the built in python like everybody else.

I really want the syntax highlighting, intellisense and code debugging features of IDAJscript though.

I have read you can get WingWare IDE ($100 personal/ $250 professional license) for python to work as the remote debugger for IDA python, but never tried it myself yet.

For context, when I first started this project IDA only had IDC scripting support with what you typed into a flat textbox.

Times have changed, but I am already pretty invested and like what I have built. I also really like a fully open source stack.

IDASRVR opens the door for lots of other tool integration such as:
  • scdbg being able to -sync shellcode emulation
  • IDACompare able to pull in decompilation results on the fly or jump to functions
  • custom wingraph replacements able to navigate and rename functions
  • ability to work with IDA through arbitrary compiled languages
  • ability to debug/develop plugouts in your debugger of choice without having to continually open/close IDA
At a min at least IDASRVR must be updated and having a remote control interface for IDA integration has had proven utility.

On 10.22.17 - 1:49am Dave wrote:
If you want to see a diff for what had to change to get IDASRVR2 to compile for IDA7

Leave Comment:
Email: (not shown)
Message: (Required)
Math Question: 66 + 27 = ? followed by the letter: F 

About Me
More Blogs
Main Site
Yara WorkBench
vbdec dbg updates
vb6 PCode NOP
vb6 API and call backs
how pcode works Pt1
Reversing PCode Args
VB6 PCode Disassembly
VB6 PCode Debugger
UConnect Disable Cell Modem
IDA python over IPC
dns wildcard blocking
64bit IDA Plugins
anterior lines
misc news/updates
Decoders again
CDO.Message Breakpoints
SysAnalyzer Updates
SysAnalyzer and Site Updates
crazy decoder
ida js w/dbg
flash patching #2
JS Graphing
packet reassembly
Delphi IDA Plugin
scdbg IDA integration
API Hash Database
Winmerge plugin
IDACompare Updates
Guest Post @ hexblog
TCP Stream Reassembly
SysAnalyzer Updates
Apilogger Video
Shellcode2Exe trainer
scdbg updates
IDA Javascript w/IDE
Rop Analysis II
scdbg vrs ROP
flash patching
x64 Hooks
micro hook
jmp api+5 *2
SysAnalyzer Updates
InjDll runtime config
C# Asm/Dsm Library
Shellcode Hook Detection
Updates II
Java Hacking
Windows 8
Win7 x64
Graphing ideas
.Net Hacking
Old iDefense Releases
hll shellcode
ActionScript Tips
-patch fu
scdbg ordinal lookup
scdbg -api mode
Peb Module Lists
scdbg vrs Process Injection
GetProcAddress Scanner
scdbg fopen mode
scdbg findsc mode
scdbg MemMonitor
demo shellcodes
scdbg download
api hashs redux
Api hash gen
Retro XSS Chat Codes
Exe as DLL
Olly Plugins
Debugging Explorer
Attach to hidden process
JS Refactoring
Asm and Shellcode in CSharp
Fancy Return Address
PDF Stream Dumper
Malcode Call API by Hash
WinDbg Cheat Sheet
GPG Automation