64bit IDA Plugins

Author: David Zimmer
Date: 10.16.17 - 7:48am

So as we know IDA is going to primarily be a x64 binary now.

Bit of a hiccup for me since I use VB6 for my gui's. With IDAJscript the IDE/script engine actually run out of process and use interprocess communication (IPC) to talk to a small plugin stub, IDASRVR, that runs within IDA.

IDASRVR is going to be my escape path I suppose. I will have to convert IDACompare and others to run fully out of process which is fine. More stable for me to have my code insulated from API changes in IDA anyway.

So I started on the conversion of IDASRVR to first bring in 64bit p64 support which it didnt have yet (while keeping it a 32bit compile). This had some interesting bits.

My quickcall mechanism went almost completely out the window for x64. Using only sendmessage I was able to pass in two 32bit arguments and receive a 32bit return value. Well thats out for anything that takes a 64bit address. (If I was an x64 calling process it would probably still be ok but I am not)

Ultimately I want to get my IDAJScript running with the new IDA. So I have 3 layers to get working. First my IDASRVR plugin stub running in IDA, second my VB6 layer, and third its integration with the Duktape javascript engine.

So apparently Javascript does not support precise 64 bit numbers. You need to use a Bigint library for that. I didnt see that coming. Duktape was a perfect choice for a script engine with its nice tight size (1mb) and its debugger protocol. It took me like 3 mos of hobby time to get that debugger working the way I wanted and another 3 mos to get the Scintilla IDE portion where I wanted it. It was a big project to get just those components working.

VB6 doesnt support 64 bit numbers natively either so here is the plan as best I see it right now.

I will port IDASRVR to work with the 64bit IDA. It will support plw and p64 and run in process. All my other plugins, at least those worth porting (At least IDACOmpare and IDAJScript) will run fully out of process and only deal with IDA through the IDASRVR layer. Developing plugins in process is no fun anyway, out of process plugin interface is way better on so many levels. For lack of a better term I guess I will call these plugouts.

There will be slight differences between the 32bit and 64bit api use. Since neither vb6 or js support 64 bit numbers I will have to use a BigInt class to deal with them.

I could just convert fully and use only 64 bit addresses in the vb6 and js client classes...but using the BigInt class is going to add some annoying extra steps for quick scripts, and honestly 95% of the time I am working on 32bit disassemblies still.

So the plan is I am going to keep my 32bit IDAClient class unchanged (good already debugged and stable and wont break existing code). For working with 64bit disassemblies, I will add a IDAClient64 class whos exposed interface is almost identical, except that any arguments or return values which work with addresses now use a ULong64 class type. This vb6 class for BinInt support will be usable on the JS layer as well killing two birds with one stone.

Down side, 64bit and 32bit scripts will not be compatible, although you can write one script that works with both. This is annoying but I am dealing with multiple layers of incompatibilities so it kind of is what it is. Plus I dont do that much work with 64bit disassemblies anyway so.

64bit transition in general has been annoying for me. This also includes having to deal with Windows file system/registry redirection and COM registration concerns.

Aghh yes complexities abound and multiply. The poor human mind drowning on its current technological path. A couple relevant examples from todays news here, and here. It appears that today us is not smart enough for tomorrow us, or maybe tomorrow us just doesnt like today us that much :)

Comments: (2)

On 10.16.17 - 9:32am Dave wrote:
Some people might ask why i bother..I could just save myself a ton of work and use the built in python like everybody else.

I really want the syntax highlighting, intellisense and code debugging features of IDAJscript though.

I have read you can get WingWare IDE ($100 personal/ $250 professional license) for python to work as the remote debugger for IDA python, but never tried it myself yet.

For context, when I first started this project IDA only had IDC scripting support with what you typed into a flat textbox.

Times have changed, but I am already pretty invested and like what I have built. I also really like a fully open source stack.

IDASRVR opens the door for lots of other tool integration such as:
  • scdbg being able to -sync shellcode emulation
  • IDACompare able to pull in decompilation results on the fly or jump to functions
  • custom wingraph replacements able to navigate and rename functions
  • ability to work with IDA through arbitrary compiled languages
  • ability to debug/develop plugouts in your debugger of choice without having to continually open/close IDA
At a min at least IDASRVR must be updated and having a remote control interface for IDA integration has had proven utility.

On 10.22.17 - 1:49am Dave wrote:
If you want to see a diff for what had to change to get IDASRVR2 to compile for IDA7

Leave Comment:
Email: (not shown)
Message: (Required)
Math Question: 63 + 56 = ? followed by the letter: J 

About Me
More Blogs
Main Site
Posts: (year)
2024 (1)
     vbdec backstory
2023 (4)
     Yara Workbench Automation
     VS linker versions
     IDA decompiler comments
2022 (5)
     VB6 Implements
     VB6 Stubs BS
     VB6 TypeInfo
     VB6 VTable Layout
     Yara isPCode rule
2021 (2)
     VB6 Gosub
2020 (5)
     AutoIT versions
     IDA JScript 2
     Using VB6 Obj files from C
     Yara Corrupt Imports
     Yara Undefined values
2019 (6)
     Yara WorkBench
     vb6 API and call backs
     UConnect Disable Cell Modem
2017 (5)
     IDA python over IPC
     dns wildcard blocking
     64bit IDA Plugins
     anterior lines
     misc news/updates
2016 (4)
     KANAL Mod
     Decoders again
     CDO.Message Breakpoints
     SysAnalyzer Updates
2015 (5)
     SysAnalyzer and Site Updates
     crazy decoder
     ida js w/dbg
     flash patching #2
     JS Graphing
2014 (5)
     Delphi IDA Plugin
     scdbg IDA integration
     API Hash Database
     Winmerge plugin
     IDACompare Updates
2013 (9)
     Guest Post @ hexblog
     TCP Stream Reassembly
     SysAnalyzer Updates
     Apilogger Video
     Shellcode2Exe trainer
     scdbg updates
     IDA Javascript w/IDE
     Rop Analysis II
     scdbg vrs ROP
2012 (13)
     flash patching
     x64 Hooks
     micro hook
     jmp api+5 *2
     SysAnalyzer Updates
     InjDll runtime config
     C# Asm/Dsm Library
     Shellcode Hook Detection
     Updates II
     Java Hacking
     Windows 8
     Win7 x64
2011 (19)
     Graphing ideas
     .Net Hacking
     Old iDefense Releases
     hll shellcode
     ActionScript Tips
     -patch fu
     scdbg ordinal lookup
     scdbg -api mode
     Peb Module Lists
     scdbg vrs Process Injection
     GetProcAddress Scanner
     scdbg fopen mode
     scdbg findsc mode
     scdbg MemMonitor
     demo shellcodes
     scdbg download
     api hashs redux
     Api hash gen
2010 (11)
     Retro XSS Chat Codes
     Exe as DLL
     Olly Plugins
     Debugging Explorer
     Attach to hidden process
     JS Refactoring
     Asm and Shellcode in CSharp
     Fancy Return Address
     PDF Stream Dumper
     Malcode Call API by Hash
     WinDbg Cheat Sheet
2009 (1)
     GPG Automation