How do you prevent the linker from discarding a function you want to make available for debugging?

Date:	January 2, 2015 / year-entry #2
Tags:	code
Orig Link:	https://blogs.msdn.microsoft.com/oldnewthing/20150102-00/?p=43233
Comments:	22
Summary:	We saw some time ago that you can ask the Windows symbolic debugger engine to call a function directly from the debugger. To do this, of course, the function needs to exist. But what if you want a function for the sole purpose of debugging? It never gets called from the main program, so the...

We saw some time ago that you can ask the Windows symbolic debugger engine to call a function directly from the debugger. To do this, of course, the function needs to exist.

But what if you want a function for the sole purpose of debugging? It never gets called from the main program, so the linker will declare the code dead and remove it.

One sledgehammer solution is to disable discarding of unused functions. This the global solution to a local problem, since you are now preventing the discard of any unused function, even though all you care about is one specific function.

If you are comfortable hard-coding function decorations for specific architectures, you can use the /INCLUDE directive.

#if defined(_X86_)
#define DecorateCdeclFunctionName(fn) "_" #fn
#elif defined(_AMD64_)
#define DecorateCdeclFunctionName(fn) #fn
#elif defined(_IA64_)
#define DecorateCdeclFunctionName(fn) "." #fn
#elif defined(_ALPHA_)
#define DecorateCdeclFunctionName(fn) #fn
#elif defined(_MIPS_)
#define DecorateCdeclFunctionName(fn) #fn
#elif defined(_PPC_)
#define DecorateCdeclFunctionName(fn) ".." #fn
#else
#error Unknown architecture - don't know how it decorates cdecl.
#endif
#pragma comment(linker, "/include:" DecoratedCdeclFunctionName(TestMe))
EXTERN_C void __cdecl TestMe(int x, int y)
{
    ...
}

If you are not comfortable with that (and I don't blame you), you can create a false reference to the debugging function that cannot be optimized out. You do this by passing a pointer to the debugging function to a helper function outside your module that doesn't do anything interesting. Since the helper function is not in your module, the compiler doesn't know that the helper function doesn't do anything, so it cannot optimize out the debugging function.

struct ForceFunctionToBeLinked
{
  ForceFunctionToBeLinked(const void *p) { SetLastError(PtrToInt(p)); }
};

ForceFunctionToBeLinked forceTestMe(TestMe);

The call to SetLastError merely updates the thread's last-error code, but since this is not called at a time where anybody cares about the last-error code, it is has no meaningful effect. The compiler doesn't know that, though, so it has to generate the code, and that forces the function to be linked.

The nice thing about this technique is that the optimizer sees that this class has no data members, so no data gets generated into the module's data segment. The not-nice thing about this technique is that it is kind of opaque.

Comments (22)

skSdnW says:

January 2, 2015 at 7:31 am

I sometimes do "if (!GetCurrentThreadId()) DoDebugThing(whatever);". This has more run-time overhead but also allows you to selectively execute the function.

If you only care about VC:

EXTERN_C void __cdecl ThisSeemsToWorkInVC(int x, int y)

{

#pragma comment(linker, "/include:"__FUNCDNAME__)

}
Joshua says:

January 2, 2015 at 7:46 am

I just export the function. EXE images can export functions. I'm told DLL images can refer to them but I have no idea how this works and as far as I know only Cygwin uses this so they may have hacked the loader.
Ben Voigt says:

January 2, 2015 at 8:18 am

@Joshua: DLLs importing from the any other module, including the executable image, is supported by the OS itself (and not a recently added feature either). Nothing cygwin-specific there.
Matt says:

January 2, 2015 at 11:42 am

Eww. Don't send it to a function that sets state and has a meaningful type like SetLastError. It's a type error at best and an app-compat fail waiting to happen. One day sending the pointer to SetLastError will do something crazy you didn't expect (like a shim hooking SetLastError to watching for a WINERROR range will happen to fire because ASLR put your function somewhere where it overlaps with that error-code range and then your app will explode and install malware and take your credit-card information).

At best pass it to something that doesn't change state and which expects a PVOID like Ntdll!EncodePointer, rather than something that statefully sets a meaningful non-PVOID parameter like SetLastError.
poizan42 says:

January 2, 2015 at 11:44 am

@Ben Voigt, I'm not sure about how ASLR fit into this, but exe images used to not be relocateable (unless forcing the linker to generate a relocateable image ofc.), in which case loading it into another process will fail if there is already somthing allocated at the base address.
Joshua says:

January 2, 2015 at 12:26 pm

Hmmm. Matt is correct. Too bad NtDll!EncodePointer is undocumented.
Gabe says:

January 2, 2015 at 12:53 pm

Oh, are we trying to figure out the best Win32 function that you can pass a pointer to while minimizing any side effects?

I'm going to nominate GetProcessId(). It will almost certainly fail, causing the last error to be set to something, but at least it will be something fairly predictable. It takes a HANDLE, but it's no problem to convert a pointer to a HANDLE.

[This is a bad idea, because it will generate a STATUS_INVALID_HANDLE exception. -Raymond]
skSdnW says:

January 2, 2015 at 1:11 pm

@Gabe: Why gamble when you can use "IsBadReadPtr(myptr,0)"? kernel32!EncodePointer is documented if you want to use that.

[IsBadReadPtr is a bad idea because it will trigger security warnings in code analysis tools. -Raymond]
Matt says:

January 2, 2015 at 1:17 pm

@Joshua: It isn't undocumented: msdn.microsoft.com/…/bb432254%28v=vs.85%29.aspx
Joshua says:

January 2, 2015 at 2:21 pm

@Matt: Ah there we go that's kernel32!EncodePointer which is fine if you run on anything newer than Windows 2000.
Chris Crowther says:

January 2, 2015 at 3:36 pm

At least the not-nice thing can be resolved with comments.
Anonymous says:

January 2, 2015 at 10:30 pm

Ignoring for a moment that many of these architectures (alpha, mips, ppc) are not currently supported by NT-based Windows (and ia64 is not supported in the latest version), why not combine some of those (undecorated) architectures with an || in the preprocessor directive? This makes it much less verbose:

#if defined(_X86_)

#define DecorateCdeclFunctionName(fn) "_" #fn

#elif defined(_AMD64_) || defined(_ALPHA_) || defined(_MIPS_)

#define DecorateCdeclFunctionName(fn) #fn

#elif defined(_IA64_)

#define DecorateCdeclFunctionName(fn) "." #fn

#elif defined(_PPC_)

#define DecorateCdeclFunctionName(fn) ".." #fn

#else

#error Unknown architecture

#endif

You could make it even shorter by putting the "_", ".", or ".." into another variable, maybe with "prefix" in the name… Seeing DecorateCdeclFunctionName(fn) so verbosely and so many times is off-putting.

[It's less verbose, but it's harder to manage when new architectures are added. -Raymond]
Alex says:

January 3, 2015 at 3:53 am

@poizan42: this feature may be useful for in-process DLLs, too.
Craven Weasel says:

January 3, 2015 at 8:07 am

I can handle the syntactical ambiguities of C for the most part. But while glancing over your post I stared at the line "ForceFunctionToBeLinked forceTestMe(TestMe);" for about two minutes, trying to figure out how you could keep a function from being linked out simply by the return type, until it occurred to me that you are defining a variable, not a function.

Regarding the opaque-ness of this technique, you could always define a macro to hide the variable declaration behind a meaningful name. Perhaps play a trick with __LINE__ to generate a sufficiently unique identifier.
Ben Voigt says:

January 3, 2015 at 1:49 pm

@poizan42 That creates difficulty in loading an image compiled as a main application into a process that has another main application. It presents no difficulty whatsoever for modules importing functions from the first non-relocatable image in their process, which is the topic being discussed.

Imports don't necessarily require loading a DLL; they are perfectly happy binding to modules already present, including the application module.
voo says:

January 4, 2015 at 3:20 am

@Matt: You are right, but we can still use the call with a simple change, since: "Bit 29 is reserved for application-defined error codes; no system error code has this bit set"

So just or the result with (1 << 28) and we're golden.
killer{r} says:

January 4, 2015 at 5:53 am

if (!&GetLastError) SetLastError(PtrToInt(p));

would be less intrusive, if only GetLastError will not become intrinsic one day
Killer{R} says:

January 4, 2015 at 7:37 am

Also a bit surprising that

volatile void *pTestMe = &TestMe;

is not helpful – no TestMe and no p in resulting module.
Sven2 says:

January 5, 2015 at 4:57 am

Could also do it like this:

if (GetLastError() == PtrToInt(p)) SetLastError(PtrToInt(p));

It would never change the error code and even the call to SetLastError is very unlikely to happen.
mikeb says:

January 5, 2015 at 9:41 am

It seems to me that Joshua's idea of simply exporting the function would be the best choice – there's no 'opaqueness' about it. You're explicitly declaring that you want to make the function callable from 'something else'. In this case the debugger is that something else.

Is there a downside to that technique that's causing people to continue to look for another method that doesn't have some oddball side-effect possibility?

[The downside of exporting it is that it becomes externally visible, and now you have an undocumented API that you have to preserve for compatibility reasons. It also means that there are now two files you need to keep in sync (the DEF file and the implementation). And you can't put #ifdef in a DEF file, so now you have the debugging function availble even in the non-debug build. -Raymond]
alegr1 says:

January 5, 2015 at 9:45 am

The worse thing the linker does in the debug build is identical COMDAT elimination (ICE). This causes all little stub functions to be merged. Then you can't set a breakpoint in just one of them, and it will mess any stack trace and disassembly which might refer to such functions.
James Curran says:

January 5, 2015 at 2:20 pm

Also, couldn't we reduce this to just

ForceFunctionToBeLinked(TestMe);

(Constructing, but not storing the object). This is even more obvious what it's doing (if less obvious how it doing it).

[You have to put the function call inside a function, and you have to make sure that enclosing function is itself not optimized out by the linker, so you're back to the original problem. -Raymond]

Comments are closed.

*DISCLAIMER: I DO NOT OWN THIS CONTENT. If you are the owner and would like it removed, please contact me. The content herein is an archived reproduction of entries from Raymond Chen's "Old New Thing" Blog (most recent link is here). It may have slight formatting modifications for consistency and to improve readability.

WHY DID I DUPLICATE THIS CONTENT HERE? Let me first say this site has never had anything to sell and has never shown ads of any kind. I have nothing monetarily to gain by duplicating content here. Because I had made my own local copy of this content throughout the years, for ease of using tools like grep, I decided to put it online after I discovered some of the original content previously and publicly available, had disappeared approximately early to mid 2019. At the same time, I present the content in an easily accessible theme-agnostic way.

The information provided by Raymond's blog is, for all practical purposes, more authoritative on Windows Development than Microsoft's own MSDN documentation and should be considered supplemental reading to that documentation. The wealth of missing details provided by this blog that Microsoft could not or did not document about Windows over the years is vital enough, many would agree an online "backup" of these details is a necessary endeavor. Specifics include:

A "redesign" after 2019 erased thousands of user's comments from previous years. As many have stated, the comments are nearly as important as the postings themselves. The archived copies of the postings contained here retain the original comments.
The blog has changed domains many times and the urls have otherwise been under constant change since 2003. Even when proper redirection has been set up for those links, redirection only works for a limited period of time. For example, all of the internal blog links that were valid in early 2019, were broken by 2020 without proper redirection.
The blog has been under constant re-design and re-theming since its inception. It is downright irritating to deal with a bogged-down site experience as the result of the latest visual themes designed for cell-phone browsers. As of this writing, it is cumbersome to navigate titles with only 10 entries per page. While it is nice that the official site has a search feature, searching using this index (with all titles on a single page) is much quicker (CTRL-F in most browsers).

<-- Back to Old New Thing Archive Index