<-- Visual C++ 7.1 PDB Handle Leak Bug - Overview / Visual C++ 7.1 (2003) PDB Handle Leak Bug - Patch #2 Details 
1
Visual C++ 7.1 (2003) PDB Handle Leak Bug - Patch #2 Details

Date: Apr 29, 2018

DETAIL SUMMARY:
The circumstances for the second patch are almost identical to to Patch #1, however I've had at least one report that the original patch does not work on Windows 10, so that is why patch #2 exists. On Windows 10 (and possibly Windows 8.x as well), the PDB file remains locked, despite decrementing the refount an extra time. Patch #2 is a more aggressive variation of the original that repeatedly decrements the refcount UNTIL it becomes 1, which is the ultimate goal to unlock the file. The fix was slightly smaller code and more elegant because it doesn't rely on a hardcoded refcount.

I suspect that versions of the operating system greater than Windows 7, generate more Debugging API events as a result of new features or may generate these events in a different order. Ultimately the refcount is wrong by a larger amount than supported by the original patch.

Although the second patch should address any problems with the refcount of the object regardless of the version of Windows being used, I decided to keep the first patch available because it had already been tested to work on several machines at the time both became available.

Although both patches have been shown to work on some Windows 7 (64-bit) machines, I don't have any personal experience testing them on Windows 10. My only experience with Windows 10 is finding where Microsoft moved the shutdown button and then downgrading back to Windows 7. With that said, others have reported that it does indeed work.

Download Patch #2 with instructions (327 k)

To ensure this patch is compatible with your installation, please ensure that you have already installed the Visual Studio.NET 7.1 (2003) Service Pack 1 and that the original NatDbgDE.dll has the following attributes:

size  708,608 bytes (692 k)
date03/19/2003 02:59 AM
version7.10.3077.0
md587f2402cae3d54478c63fe8fc6f8f72f

The patched DLL (Patch #2) has the following attributes:

size  708,608 bytes (692 k)
md54dd400b81f60ceeb8686c95124d61d4d

MANUAL PATCHING:
If you'd like to perform the patch manually on your own version of NatDbgDE.dll, download the bytepatch tool and use the following commands to patch the two locations necessary for the fix:
bytepatch -pa 0x5473DA94 natdbgde.dll E907DB050090
bytepatch -pa 0x5479B5A0 natdbgde.dll 5589E583EC08894DFC8B018945F88B45F88B4DFC51FF50089C83F80174039DEBED9D89EC5DC3
NOTE: The opcodes used in the patch are described in detail below.

After manually patching, you'll probably want to update the PE checksum with peupdate using "peupdate -f NATDBGDE.DLL" or using your favorite checksum calculation/replacement tool. Since the patched bytes indirectly alter the checksum of the DLL, future versions of Windows may abort loading this DLL. Windows 7 doesn't care about a checksum mismatch so this step isn't technically necessary. If you choose to download the patched DLL, the checksum has already been updated so none of these steps are necessary.

PATCH DETAILS:
The original patch only invoked an extra Release() if the first Release() resulted in a refcount of 2, bringing the refcount to the desired count of 1. In contrast, Patch #2 (below) invokes Release() in a loop until the desired refcount of 1 is reached. As described in the original patch details, since this is called from the final object cleanup function, releasing all of the extraneous references is safe thing to do at this particular spot in the code. If you are running Windows 10, this is the patch you need because the original function is usually entered with a refcount higher than 2 under this version of Windows.

5479B5A0 55 PUSH EBP ;function is passed ECX which is "this" pointer to refcounted object 5479B5A1 89E5 MOV EBP, ESP ;create function stack frame 5479B5A3 83EC 08 SUB ESP, 8 ; to store 2 local vars 5479B5A6 894D FC MOV DWORD PTR [EBP-4], ECX ;1st local, [EBP-4] stores "this" pointer 5479B5A9 8B01 MOV EAX, DWORD PTR [ECX] 5479B5AB 8945 F8 MOV DWORD PTR [EBP-8], EAX ;2nd local, [EBP-8] stores object's function vtable 5479B5AE 8B45 F8 MOV EAX, DWORD PTR [EBP-8] ;prepare for call to Release(), EAX = vtable 5479B5B1 8B4D FC MOV ECX, DWORD PTR [EBP-4] ; ECX = "this" pointer 5479B5B4 51 PUSH ECX ;pass "this pointer" as argument to function 5479B5B5 FF50 08 CALL DWORD PTR [EAX+8] ;call Release() - 3rd entry in vtable 5479B5B8 9C PUSHFD ;save post-function flags because caller might depend on them as CMP below destroys them 5479B5B9 83F8 01 CMP EAX, 1 ;is refcount now 1? 5479B5BC 74 03 JE SHORT NatDbgDE.5479B5C1 ;if so, jump to exit 5479B5BE 9D POPFD ;clean up post-function flags which we don't need yet 5479B5BF EB ED JMP SHORT NatDbgDE.5479B5AE ;LOOP back to invoke Release() again 5479B5C1 9D POPFD ;EXIT: restore post-function flags to simulate unpatched behavior JUST IN CASE 5479B5C2 89EC MOV ESP, EBP ;cleanup stack frame 5479B5C4 5D POP EBP 5479B5C5 C3 RETN ;done!

Please inform me if you have problems or questions.

 1:1