Inline Asm w VB6


Author: Dave
Date: 02.04.10 - 6:07am



Just a quick post for documentation sake of using inline asm with VB6 (or at least as close as we can get to it without an external c dll.

In terms of development of the asm to put in. You can use your C Compiler to generate it for you..here are the tips.

  • CallwindowsProc has 5 arguments max and can return a long The first argument is already used so your args start at [EBP+0x0C] this means use a dummy int arg first in your prototype to line up args

  • do not use any sub functions from your code do things in blocks in you have to.

  • once you generate your code, you can extract the opcodes from VC in debug mode viewing mixed mode disasm (develop as an exe usually although you may have to as a dll to use with vb as standard call dll)

  • you need to strip all the function prolog and epilog asm from the compiler generated block (or use naked declspec) your ret should be RETN 10h

  • keep a couple nops (&H90) in place at start in case you need room to add a breakpoint (&hCC) manually to stop on your asm in a debugger to debug it. yes you will have to use ollydbg to debug it in asm most likley.

  • you can twiddle with the CallWindowProc prototypes more based on what you are using it for..see last example.



Full example here:
http://sandsprite.com/CodeStuff/vb_cdecl.zip

Note: All single quotes for comments are stripped by my blog script
same as default CallWindowProc except
param 1 is now "ByRef lpBytes As Any"
or you can use the default like this: CallWindowProc( Varptr( asmBytes(0) ) ,
Private Declare Function CallAsm Lib "user32" 
    Alias "CallWindowProcA" _
    (ByRef lpBytes As Any, 
    ByVal hWnd As Long, 
    ByVal Msg As Long, 
    ByVal wParam As Long, 
    ByVal lParam As Long) As Long

Function Shl(x As Long) As Long
    8B45 0C        MOV EAX,DWORD PTR SS:[EBP+12]
    D1E0           SHL EAX,1
    C2 10 00       RETN 10h
    Dim o() As Byte
    Const sl As String = "8B 45 0C D1 E0 C2 10 00"
    o() = toBytes(sl)
    Shl = CallAsm(o(0), x, 0, 0, 0)
End Function

Function Shr(x As Long) As Long
    8B45 0C        MOV EAX,DWORD PTR SS:[EBP+12]
    D1E8           SHR EAX,1
    C2 10 00       RETN 10h
    Dim o() As Byte
    Const sr As String = "8B 45 0C D1 E8 C2 10 00"
    o() = toBytes(sr)
    Shr = CallAsm(o(0), x, 0, 0, 0)
End Function

Private Function ShlX(x As Long, shift As Byte) As Long
    8B45 0C        MOV EAX,DWORD PTR SS:[EBP+12]
    C1E0 12        SHL EAX,12
    C2 10 00       RETN 10h
    Dim o() As Byte
    Const sl As String = "8B 45 0C C1 E0 __ C2 10 00"
    o() = toBytes(Replace(sl, "__", Hex(shift)))
    Shl = CallAsm(o(0), x, 0, 0, 0)
End Function

Private Function ShrX(x As Long, shift As Byte) As Long
    8B45 0C        MOV EAX,DWORD PTR SS:[EBP+12]
    C1E8 12        SHR EAX,12
    C2 10 00       RETN 10h
    Dim o() As Byte
    Const sr As String = "8B 45 0C C1 E8 __ C2 10 00"
    o() = toBytes(Replace(sr, "__", Hex(shift)))
    ShrX = CallAsm(o(0), x, 0, 0, 0)
End Function

Function getESP() As Long

    Dim C As Currency 8 bytes, initially all 0s
    
    	his isnt a naked function, so the esp is actually local to this
    function frame, but since we only use it for relative size differences
    etween before and after and there are no stack mods in vb after prolog
    
ormally, this is fine as a standalone function
    
    8BC4           MOV EAX,ESP
    C2 10 00       retn 10h
    
    CopyMemory C, &H10C2C48B, 4
    getESP = CallAsm(C, 0, 0, 0, 0)

End Function

Function toBytes(x As String) As Byte()
    Dim tmp() As String
    Dim fx() As Byte
    Dim i As Long
    
    tmp = Split(x, " ")
    ReDim fx(UBound(tmp))
    
    For i = 0 To UBound(tmp)
        fx(i) = CInt("&h" & tmp(i))
    Next
    
    toBytes = fx()

End Function




Another example of working on a byte buffer in your asm Private Declare Function CallAsm2 Lib "user32" Alias "CallWindowProcA" _ (ByRef lpBytes As Any, ByRef chararray As Any, ByVal length As Long, ByVal unused1 As Long, ByVal unused2 As Long) As Long

Const opcodes As String = "909090C745F800000000EB098B4DF88" & _ "3C101894DF88B55F83B55107D258B45" & _ "0C0345F88A08884DFC8B45F833D28A5" & _ "5FC2AD08855FC8B550C0355F88A45FC" & _ "8802EBCA9090C21000" fx() = toBytes2(opcodes) CallAsm2 fx(0), _ byteBufferToWorkOn(0), _ UBound(byteBufferToWorkOn), _ 0, 0 opcodes are for the following C with the prolog and epilog stripped out.. void __stdcall fnDecode(int dummy, char* b, int len) { char x; for(int i=0; i len; i++){ x = b[i]; _asm{ //do stuff to x here } b[i] = x; //update vb byte buffer } } Function toBytes2(x As String, Optional debugit As Boolean = False) As Byte() Dim tmp() As String Dim fx() As Byte Dim i As Long Dim y ReDim fx(Len(x) / 2) For i = 1 To Len(x) Step 2 fx(y) = CByte(CLng("&h" & Mid(x, i, 2))) y = y + 1 Next If debugit Then fx(0) = &HCC toBytes2 = fx() End Function
- more C Source - Const asm_rol32 = "x55x8BxECx56x8Bx4Dx10x83xE1x1Fx8Bx45x0C" & _ "xD3xE0x8Bx4Dx10x83xE1x1FxBAx20x00x00x00" & _ "x2BxD1x83xE2x1Fx8Bx75x0Cx8BxCAxD3xEEx0B" & _ "xC6x5Ex5DxC2x10x00" Const asm_add32 = "x55x8BxECx8Bx45x0Cx03x45x10x5dxC2x10x00" Function rol32(base As Long, bits As Long) rol32 = CallWindowProc(VarPtr(asm(1)), 0, base, bits, 0) End Function Function add32(v1 As Long, v2 As Long) 'no overflow, allows wrap add32 = CallWindowProc(VarPtr(asm2(1)), 0, v1, v2, 0) End Function






Comments: (3)

On 09.04.11 - 4:17pm gheo wrote:
Golden code mate ! Thanks for this knowledge.

On 01.31.20 - 5:12pm Dave wrote:
another example of writing/debugging your asm in C thats compatible with the CallWindowProc prototype: http://sandsprite.com/blogs/index.php?uid=7&pid=144

On 02.05.20 - 7:45am Dave wrote:
Dep safe of the above:
https://github.com/dzzie/libs/blob/master/pe_lib3/modAsm.bas

 
Leave Comment:
Name:
Email: (not shown)
Message: (Required)
Math Question: 61 + 27 = ? followed by the letter: X 



About Me
More Blogs
Main Site
Posts: (year)
2024 (3)
     VB6 JSON
     ffmpeg voodoo
     RegJump Vb
2023 (9)
     VB6 Virtual Files
     File handles across dlls
     python abort / script timeout
     py4vb
     VB6 Python embed w/debugger
     python embedding
     VB6 IDE Enhancements
     No Sleep
     A2W no ATL
2022 (4)
     More VB6 - C data passing
     Vb6 Asm listing
     Byte Array C to VB6
     Planet Source Code DVDs
2021 (2)
     Obscure VB
     VB6 IDE SP6
2020 (4)
     NTFileSize
     BSTR from C Dll to VB
     Cpp Memory Manipulation
     ActiveX Binary Compatability
2019 (5)
     Console tricks
     FireFox temp dir
     OCX License
     Extract substring
     VB6 Console Apps
2018 (6)
     VB6 UDTs
     VB6 Debugger View As Hex tooltips
     VB6 - C Share registry data
     VB6 Addin Missing Menus
     VB6 Class Init Params
     VB6 isIn function
2017 (6)
     Python and VB6
     Python pros and cons
     download web Dir
     vc rand in python
     VB6 Language Enhancement
     Register .NET as COM
2016 (22)
     VB6 CDECL
     UDT Tricks pt2
     Remote Data Extraction
     Collection Extender
     VB6 FindResource
     CDO.Message
     DirList Single Click
     Reset CheckPoint VPN Policy
     VB6 BSTR Oddities Explained
     SafeArrays in C
     BSTR and Variant in C++
     Property let optional args
     Misc Libs
     Enum Named Pipes
     Vb6 Collection in C++
     VB6 Overloaded Methods
     EXPORT FUNCDNAME Warning
     VB6 Syncronous Socket
     Simple IPC
     VB6 Auto Resize Form Elements
     Mach3 Automation
     Exit For in While
2015 (15)
     C# self register ocx
     VB6 Class Method Pointers
     Duktape Debug Protocol
     QtScript 4 VB
     Vb6 Named Args
     vb6 Addin Part 2
     VB6 Addin vrs Toolbars
     OpenFile Dialog MultiSelect
     Duktape Example
     DukTape JS
     VB6 Unsigned
     .Net version
     TitleBar Height
     .NET again
     VB6 Self Register OCXs
2014 (25)
     Query Last 12 Mos
     Progid from Interface ID
     VB6 to C Array Examples
     Human Readable Variant Type
     ScriptBasic COM Integration
     CodeView Addin
     ScriptBasic - Part 2
     Script Env
     MSCOMCTL Win7 Error
     printf override
     History Combo
     Disable IE
     API Hooking in VB6
     Addin Hook Events
     FastBuild Addin
     VB6 MemoryWindow
     Link C Obj Files into VB6
     Vb6 Standard Dlls
     CStr for Pascal
     Lazarus Review
     asprintf for VS
     VB6 GlobalMultiUse
     Scintilla in VB6
     Dynamic Highlight
     WinVerifyTrust, CryptMsgGetParam VB6
2013 (4)
     MS GLEE Graphing
     printf for VB6
     C# App Config
     Tero DES C# Test
2012 (10)
     VC 2008 Bit Fields
     Speed trap
     C# Db Class Generator
     VB6 vrs .NET (again)
     FireFox Whois Extension
     git and vb6
     Code Additions
     Compiled date to string
     C# ListView Sorter
     VB6 Wish List
2011 (7)
     C# Process Injection
     CAPTCHA Bots
     C# PE Offset Calculator
     VB6 Async Download
     Show Desktop
     coding philosophy
     Code release
2010 (11)
     Dll Not Found in IDE
     Advanced MSScript Control
     random tip
     Clipart / Vector Art
     VB6 Callback from C#
     Binary data from VB6 to C#
     CSharp and MsScriptControl
     HexDumper functions
     Js Beautify From VB6 or C#
     vb6 FormPos
     Inline Asm w VB6
2009 (3)
     The .NET Fiasco
     One rub on computers
     Universal extractor