What’s wrong with virtual methods called through an interface

Calling a virtual method through an interface always was a lot slower than calling a static method through an interface. But why is that? Sure, the virtual method call costs some time, but comparing it with the difference of a normal static and virtual method call shows that the timings diverge too much.

Let’s assume we have this declaration:

type
  IMyInterface = interface
    procedure Test(A, B: Integer);
  end;
  TTest = class(TInterfacedObject, IMyInterface)
  public
    procedure Test(A, B: Integer); virtual;
  end;

The compiler will generate a helper function for the interface method “Test”. This helper converts the “MyIntf” interface reference in the call “MyIntf.Test()” to the object reference behind the interface and then jumps to the virtual method.

add eax,-$0C   // convert the interface reference to the object reference
push eax       // save the object reference on the stack
mov eax,[eax]  // access the VMT
mov eax,[eax]  // get the “Test” VMT method address
xchg [esp],eax // swap the object ref on the stack with the method address
ret            // do the jump to the method address

This is very slow as you can see in the table above. If you know the “XCHG mem,reg” instruction, then you also know that it has an implicit “CPU LOCK” that slows down the method call a lot. But why is it using the XCHG instruction in the first place? Well, we are in between a method call. All the parameters are already loaded in to EAX, EDX and ECX. So we can’t use those to do the swap. The only way is to use the stack as temporary variable, and XCHG seemed to be the choice of the compiler engineer at the time interfaces were introduced to Delphi.

Let’s change that code to not use XCHG.

add eax,-$0 C      // convert the interface reference to the object reference
push eax           // reserve space for the method address used by RET
push eax           // save the object reference on the stack
mov eax,[eax]      // access the VMT
mov eax,[eax]      // get the “Test” VMT method address
mov [esp+04],eax   // write the method address to the reserved space
pop eax            // restore the object reference
ret                // do the jump to the method address

This is a lot faster, but still slow compared to the “Instance call”. The helper has a lot of memory accesses, but they shouldn’t slow it that much down, especially not in a tight loop when everything comes from the CPU’s cache.

So where does the code spend the time? Well, modern CPUs (after P1) have a feature called “return stack buffer”. The CPU puts the return address on the “return stack buffer” for every CALL instruction so it can predict where the RETinstruction will jump to. This requires that every CALL is matched by a RET. But wait, the helper uses a RET for an indirect jump. We have the CALL from the interface method call, the RET in the helper and the RET in the actual method. That doesn’t match up. In other words, this helper renders the “return stack buffer” invalid what comes with a performance hit because the CPU can’t predict where to jump.

Let’s see what happens if we replace the RET with a JMP.

add eax,-$0C        // convert the interface reference to the object reference
push eax            // save the object reference on the stack
mov eax,[eax]       // access the VMT
push DWORD PTR [eax]// save the “Test” VMT entry method address on the stack
add esp,$04         // skip the method address stack entry
pop eax             // restore the object reference
jmp [esp-$08]       // jump to the method address

UPDATE: As fast as this implementation may be, it has a problem. As Allen and Mark pointed out, it accesses memory on the stack that is treated as free memory from the system. So if a hardware interrupt is triggered between the “add esp,$04” and the “jmp [esp-$08]”, the data on the stack is overwritten and the jump will end somewhere but not where it should be.

UPDATE 2: Thorsten Engler sent me an e-mail that invalidates the “hardware interrupt problem”. All interrupts are handled in kernel mode and kernel mode code doesn’t touch the user stack. The CPU itself switches the SS:ESP before invoking the interrupt handler.

Based on AMD64 Architecture Programmer’s Manual Volume 2 – System Programming Rev.3.22 Section 8.7.3 Interrupt To Higher Privilege:

When a control transfer to an exception or interrupt handler running at a higher privilege occurs (numerically lower CPL value), the processor performs a stack switch using the following steps:

1. The target CPL is read by the processor from the target code-segment DPL and used as an index into the TSS for selecting the new stack pointer (SS:ESP). For example, if the target CPL is 1, the processor selects the SS:ESP for privilege-level 1 from the TSS.
2. Pushes the return stack pointer (old SS:ESP) onto the new stack. The SS value is padded with two bytes to form a doubleword.

20 thoughts on “What’s wrong with virtual methods called through an interface”

1. Ian BarkerMay 31, 2016

   I’m starting to think you might be a witch or at the very least you’ve promised one or more body parts to the evil demons of The Dark Arts.

   This is an INCREDIBLE find, a huge performance improvement for virtual methods – seriously, look at the figures!

   Good catch, again. 

    
2. BasMay 31, 2016

   Nasty problem…. what a shame you didn’t have a solution for us delphi developers.
   Let’s hope the compiler builders pick it up quickly

    
3. Allen BauerMay 31, 2016

   While your code is certainly faster, it does reference memory *above* the stack pointer. If you consider the stack as a mark-and-release memory allocator, the memory above the stack pointer is *freed* memory. This code is referencing freed memory. For the CPU stack this is not considered universally safe at all.

    
   1. Andreas Hausladen Post authorMay 31, 2016

      At least the freed stack slot was accessed before it is read, so the memory page is already committed.

       
      1. Mark GriffithsJune 1, 2016

         I could well be wrong on this, but I think that the issue that Allen is alluding to is that between: add esp,$04 and the jump, if a CPU interrupt occurs, the contents of the stack that you’re going to use later will be overwritten – leading to unpredictable crashes…

          
         1. Andreas Hausladen Post authorJune 1, 2016

            Ah, the interrupts. Yes that could be a problem here.

             
      2. Allen BauerJune 1, 2016

         That isn’t the problem. As Mark pointed out, interrupts can scribble over the area above the stack pointer, rendering that data invalid.

         while there are valid reasons to make the implementing method virtual, but for the most part it isn’t necessary. By definition interfaces already are virtual.

         Instances where it is valid is where you are adding interface implementations to an existing hierarchy of classes and still want to preserve the normal inheritance and virtual override functionality. Sometimes you want to “hide” the notion of an interface since it’s only used internal to the class hierarchy.

         If you think the virtual method thunk is expensive, try it with a dynamic method 

          
4. Hallvard VassbotnJune 1, 2016

   Well, people should not be mixing interface methods and virtual methods anyway – it is a kind of two-level indirection that doesn’t really provide anything useful – other than slowing down the code, of course… 

    
   1. Stefan GlienkeJune 1, 2016

      If I have an abstract base class that implements a given number of interfaces and I build a class hierarchy on top of that whats wrong with that?

       
      1. Dave NottageJune 2, 2016

         Stole the words from my mouth, and I have at least one situation that does it. Having said that, the calls are not made often, so the performance hit is not much of a concern.

          
      2. Hallvard VassbotnJune 2, 2016

         Performance. And mixed abstraction models.

         To “override” a method you should consciously know that you are modifying the behavior of a contract (the interface), thus you should include the interface in the sub-class and re-implement the method. No virtual needed.

          
5. Hallvard VassbotnJune 1, 2016

   But a very nice find and write-up! 

    
6. RafaelJune 2, 2016

   When I test it under 32 bits,
   I get similar results but if I test it on 64 bit, I do not get such slowdowns. I would say the 64 Bit compilers works in this case more intelligent 

    
7. RafaelJune 2, 2016

   I tested this case under Delphi XE 7 and get same results when I test it under 32 bits, but if I test it on 64 bit, I do not get such slowdowns. I would say in this case the 64-bit compiler operates smarter.

    
8. Andreas Hausladen Post authorJune 2, 2016

   The Win64 compiler has more CPU registers available and doesn’t need to use the stack to jump to the virtual method.

    
9. egJune 2, 2016

   Does this issue impact iOS implemetation?

    
   1. Andreas Hausladen Post authorJune 2, 2016

      Did I show any ARM assembler code? This is Win32 only.

       
10. Mark GriffithsJune 3, 2016

    I’ve been trying to think of a possible solution to this problem.

    One thought that I had was to use self modifying code – i.e. instead of storing the destination address on the stack, you could update the address in memory for an unconditional jump. The downside to this would be that the code then wouldn’t be thread safe.

    My next thought was to have thread specific memory allocated so that each thread had it’s own section of memory for the self modifying code – the problem with this is that you’re then back to having to determine an address and jump to it without having enough registers or being able to use the stack.

    A possible solution then would be to store the address on the stack, jump to the self modifying code which first adjusts the stack pointer to remove the temporary address and then do the unconditional jump.

    I think that this would work and be safe, but it does increase the overheads and introduces possible problems from using self modifying code – either or which could end up causing more problems than we started with…

     
    1. Andreas Hausladen Post authorJune 3, 2016

       I tried to use a “CALL” that jumps to the next line, so that the CALL+RET match, but modifying the return address on the stack already kills the return address prediction.

       What the compiler could do is, if the method has only one parameter, the ECX register would be free to use. The XCHG can be removed in all cases.

       The only good solution (for Win32) is to not use virtual methods in interfaces at all if this is an actual bottleneck in the application. In most cases it isn’t because the other code takes much more time.

        
11. Arthur HoornwegJune 3, 2016

    Hi Andreas,
    I think the problem can be easily overcome by using explicit interface method resolution. Let the interface point to a non-virtual method that chains into the virtual method.

    Regards,
    Arthur Hoornweg
    ———————-

    Type imyinterface=interface
    Procedure Dosomething;
    end;

    Type tMyclass=class(tInterfacedObject, iMyInterface)
    Procedure DoSomething; VIRTUAL; {virtual/slow}
    Procedure DoSomethingFast; {fast}
    Procedure iMyInterface.DoSomething =DoSomethingFast;
    End;

    Procedure tMyclass.DoSomethingFast;
    begin
    DoSomething; //chain into virtual method
    end;