VB6 Gosub

Author: David Zimmer
Date: 04.24.21 - 3:29pm

So it was recently noted that the VB6 gosub is actually slower than a function call out to a sub function. This seems weird right? shouldnt code still inline to the local function be faster than calling out to another with arguments?

So this made me curious..we start with the following vb code:

Private Sub Form_Load()
    Dim i As Long

    i = 0
    GoSub mysub
    GoSub mysub2
    GoSub mysub3
    MsgBox i
    
Exit Sub
mysub:
    i = i + 1
    Return
    
mysub2:
    i = i + 2
    Return
    
mysub3:
    i = i + 3
    Return
End Sub

This gives us the following PCode (the native code follows the same code flow)

4017B8 Form1.Form_Load:
4017B8    F5 00000000           LitI4 0x0
4017BD    71 78FF               FStR4 [var_88]     ;i variable memory address
4017C0    FD0A 3900             Gosub loc_4017F1          
4017C4    FD0A 4700             Gosub loc_4017FF
4017C8    FD0A 5500             Gosub loc_40180D
4017CC    27 04FF               LitVar_Missing var_FC
4017CF    27 24FF               LitVar_Missing var_DC
4017D2    27 44FF               LitVar_Missing var_BC
4017D5    F5 00000000           LitI4 0x0
4017DA    04 78FF               FLdRfVar var_88
4017DD    4D 64FF0340           CVarRef var_9C  0x4003 (I4 | ByRef)
4017E2    0A 00001400           ImpAdCallFPR4 rtcMsgBox
4017E7    36 0600 [6 bytes]     FFreeVar var_BC var_DC var_FC 
4017F0    13                    ExitProcHresult 
4017F1    6C 78FF               ILdRf [var_88]
4017F4    F5 01000000           LitI4 0x1
4017F9    AA                    AddI4
4017FA    71 78FF               FStR4 [var_88]
4017FD    FCC9                  Return
4017FF    6C 78FF               ILdRf [var_88]
401802    F5 02000000           LitI4 0x2
401807    AA                    AddI4
401808    71 78FF               FStR4 [var_88]
40180B    FCC9                  Return
40180D    6C 78FF               ILdRf [var_88]
401810    F5 03000000           LitI4 0x3
401815    AA                    AddI4
401816    71 78FF               FStR4 [var_88]
401819    FCC9                  Return
40181B    13                    ExitProcHresult

So we see that the gosub routines themselves are actually in the body of the main function, this way they can share all of the local variables etc and be in the same stack frame.

So where is the slow down occurring? Looking back at the native code..we see the following:

.text:00401985                 lea     edx, [ebp+var_8C]
.text:0040198B                 push    edx
.text:0040198C                 call    ds:__vbaGosub
.text:00401992                 test    eax, eax
.text:00401994                 jnz     short loc_40199B
.text:00401996                 jmp     loc_401A2F      ; jmp to actual inline gosub handler

.text:00401A2F loc_401A2F:                                    ;mysub label                       
.text:00401A2F                 mov     ecx, [ebp+var_18]
.text:00401A32                 add     ecx, 1
.text:00401A35                 jo      loc_401AD1
.text:00401A3B                 mov     [ebp+var_18], ecx
.text:00401A3E                 lea     edx, [ebp+var_8C]
.text:00401A44                 push    edx
.text:00401A45                 call    ds:__vbaGosubReturn

so the vbaGosub actually does an 8 byte alloc where it stores the return address from the vbagosub call
execution then returns to 0401992 in the parent function,
it is the jmp @0401996 which executes the inline mysub in the context of the parent function.
vbaGosubReturn then uses the alloc that was created in vbaGosub to then modify its own stack return address
then frees the 8 byte alloc
when the ret inside vbaGosubReturn happens, it then returns the next address after the calling vbaGosub call.

this means that the code immediately after the vbagosub actually runs twice!

.text:0040198C                 call    ds:__vbaGosub
.text:00401992                 test    eax, eax
.text:00401994                 jnz     short loc_40199B
.text:00401996                 jmp     loc_401A2F      ; jmp to actual inline gosub handler

vbaGosub sets eax to 0 so when it returns the jnz short loc_40199B will not trigger.
vbaGosubReturn sets eax to 1, so when execution again gets back to 0401992, this time the jnz will trigger
which skips the jmp handler.

so the reason gosub is slow is because there is an alloc/free behind the scenes in every gosub call and also two calls into the runtime for the vbaGosub/vbaGosubReturn pair.

I havent bothered to explicitly confirm it, but I suspect the second value saved to the alloc is to create a one way linked list to any previous gosub call allocs so they can be nested.

The actual native handlers are

ENGINE:6610D2E8 ___vbaGosub@4   proc near              
ENGINE:6610D2E8
ENGINE:6610D2E8 arg_0           = dword ptr  8
ENGINE:6610D2E8
ENGINE:6610D2E8                 push    ebp
ENGINE:6610D2E9                 mov     ebp, esp
ENGINE:6610D2EB                 push    ebx
ENGINE:6610D2EC                 push    esi
ENGINE:6610D2ED                 push    edi
ENGINE:6610D2EE                 push    8
ENGINE:6610D2F0                 call    _ProfMemAlloc@4 ; 8 byte alloc to store the current return addr and a var
ENGINE:6610D2F5                 or      eax, eax
ENGINE:6610D2F7                 jz      short OutOfMemory
ENGINE:6610D2F9                 mov     ebx, [ebp+arg_0]
ENGINE:6610D2FC                 mov     ecx, [ebx]
ENGINE:6610D2FE                 mov     [eax], ecx
ENGINE:6610D300                 mov     ecx, [ebp+4]
ENGINE:6610D303                 mov     [eax+4], ecx
ENGINE:6610D306                 mov     [ebx], eax
ENGINE:6610D308                 xor     eax, eax    <-- 0 eax make sure we dont trigger the jnz and hit the jmp
ENGINE:6610D30A                 pop     edi
ENGINE:6610D30B                 pop     esi
ENGINE:6610D30C                 pop     ebx
ENGINE:6610D30D                 leave
ENGINE:6610D30E                 retn    4
ENGINE:6610D30E ___vbaGosub@4   endp

ENGINE:6610D311 ___vbaGosubReturn@4 proc near          
ENGINE:6610D311
ENGINE:6610D311 arg_0           = dword ptr  8
ENGINE:6610D311
ENGINE:6610D311                 push    ebp
ENGINE:6610D312                 mov     ebp, esp
ENGINE:6610D314                 push    ebx
ENGINE:6610D315                 push    esi
ENGINE:6610D316                 push    edi
ENGINE:6610D317                 mov     ebx, [ebp+arg_0]
ENGINE:6610D31A                 mov     esi, [ebx]
ENGINE:6610D31C                 or      esi, esi
ENGINE:6610D31E                 jz      short ReturnWOGoSub
ENGINE:6610D320                 mov     ecx, [esi]      ; esi = alloced mem from gosub
ENGINE:6610D322                 mov     [ebx], ecx
ENGINE:6610D324                 mov     ecx, [esi+4]
ENGINE:6610D327                 mov     [ebp+4], ecx    ; overwrite return address with one saved from alloc
ENGINE:6610D32A                 push    esi
ENGINE:6610D32B                 call    _ProfMemFree@4  ; free alloc
ENGINE:6610D330                 mov     eax, 1 <--- triggers the jnz this time
ENGINE:6610D335                 pop     edi
ENGINE:6610D336                 pop     esi
ENGINE:6610D337                 pop     ebx
ENGINE:6610D338                 leave
ENGINE:6610D339                 retn    4               ; now we return to the address right after the original vbagosub call
ENGINE:6610D339 ___vbaGosubReturn@4 endp

The api decl is like this: (do not run in IDE, compiled only)

Private Type goSubAlloc
    backLinkToPrevAlloc As Long
    calledFromRetAddr As Long
End Type

Private Declare Function vbagosub Lib "msvbvm60" Alias "__vbaGosub" (ByRef pAlloc As Long) As Long
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (pDest As Any, pSrc As Any, ByVal ByteLen As Long)

Private Sub Form_Load()

    Dim p As Long
    Dim gsa As goSubAlloc
    Dim r As Long
   
    r = vbagosub(p)
    MsgBox "vbagosub(p) = " & r & " p now = " & Hex(p)

    CopyMemory gsa, ByVal p, 8
    
    MsgBox "backLinkToPrevAlloc: " & Hex(gsa.backLinkToPrevAlloc)
    MsgBox "calledFromRetAddr: " & Hex(gsa.calledFromRetAddr)

end sub