64bit IDA Plugins
Author: David Zimmer
Date: 10.16.17 - 7:48am
So as we know IDA is going to primarily be a x64 binary now.
Bit of a hiccup for me since I use VB6 for my gui's. With IDAJscript the IDE/script engine actually run out of process and use interprocess communication (IPC) to talk to a small plugin stub, IDASRVR, that runs within IDA.
IDASRVR is going to be my escape path I suppose. I will have to convert IDACompare and others to run fully out of process which is fine. More stable for me to have my code insulated from API changes in IDA anyway.
So I started on the conversion of IDASRVR to first bring in 64bit p64 support which it didnt have yet (while keeping it a 32bit compile). This had some interesting bits.
My quickcall mechanism went almost completely out the window for x64. Using only sendmessage I was able to pass in two 32bit arguments and receive a 32bit return value. Well thats out for anything that takes a 64bit address. (If I was an x64 calling process it would probably still be ok but I am not)
VB6 doesnt support 64 bit numbers natively either so here is the plan as best I see it right now.
I will port IDASRVR to work with the 64bit IDA. It will support plw and p64 and run in process. All my other plugins, at least those worth porting (At least IDACOmpare and IDAJScript) will run fully out of process and only deal with IDA through the IDASRVR layer. Developing plugins in process is no fun anyway, out of process plugin interface is way better on so many levels. For lack of a better term I guess I will call these plugouts.
There will be slight differences between the 32bit and 64bit api use. Since neither vb6 or js support 64 bit numbers I will have to use a BigInt class to deal with them.
I could just convert fully and use only 64 bit addresses in the vb6 and js client classes...but using the BigInt class is going to add some annoying extra steps for quick scripts, and honestly 95% of the time I am working on 32bit disassemblies still.
So the plan is I am going to keep my 32bit IDAClient class unchanged (good already debugged and stable and wont break existing code). For working with 64bit disassemblies, I will add a IDAClient64 class whos exposed interface is almost identical, except that any arguments or return values which work with addresses now use a ULong64 class type. This vb6 class for BinInt support will be usable on the JS layer as well killing two birds with one stone.
Down side, 64bit and 32bit scripts will not be compatible, although you can write one script that works with both. This is annoying but I am dealing with multiple layers of incompatibilities so it kind of is what it is. Plus I dont do that much work with 64bit disassemblies anyway so.
64bit transition in general has been annoying for me. This also includes having to deal with Windows file system/registry redirection and COM registration concerns.
Aghh yes complexities abound and multiply. The poor human mind drowning on its current technological path. A couple relevant examples from todays news here, and here. It appears that today us is not smart enough for tomorrow us, or maybe tomorrow us just doesnt like today us that much :)
Comments: (2)On 10.16.17 - 9:32am Dave wrote:
On 10.22.17 - 1:49am Dave wrote: