Pidls and monikers do roughly the same thing, just backwards

Date:July 12, 2006 / year-entry #232
Tags:other
Orig Link:https://blogs.msdn.microsoft.com/oldnewthing/20060712-17/?p=30543
Comments:    11
Summary:When operating with the Windows shell, you will almost certainly find yourself at some point working with a pointer to an item ID list, known also as a "pidl" (rhymes with "middle"). On the other hand, when working with OLE you may find yourself having do deal with monikers. In a sense, they both do...

When operating with the Windows shell, you will almost certainly find yourself at some point working with a pointer to an item ID list, known also as a "pidl" (rhymes with "middle"). On the other hand, when working with OLE you may find yourself having do deal with monikers. In a sense, they both do the same thing. They let you refer to some sort of object inside a namespace (to which you can bind or otherwise operate on), they have a hierarchical structure, you can persist them, and so on.

Why, then, did the Windows shell team invent pidls when monikers do the same thing?

The fundamental difference between pidls and monikers is not in what you can do with them, but rather in their "bias". A moniker is a COM object. This means that its primary existence is in the form of a COM interface (IMoniker), which means that the code behind the object is loaded and ready. You can convert it to a persistence format, but that's not how monikers really spend their lives. If you have an array of monikers, all the code behind those objects has been loaded and initialized.

A pidl, on the other hand, spends most of its life in its persistence format. Only when you bind to it does a live COM object come out. Consider, for example, the case where you enumerated the contents of a shell folder. This produces a pidl for each item in the folder, but producing and retaining that pidl doesn't require that the code for each item be loaded and initialized. A folder with a thousand items produces a thousand little chunks of data, not a thousand COM objects. Consider, for example, a folder that contains a dozen Excel spreadsheets. If you enumerate the contents of the folder, you don't want Excel to fire up and load each of those spreadsheets to give you a dozen live, running spreadsheet objects. At this stage, you are just talking about that spreadsheet file. It's not until the user double-clicks on other otherwise tries to activate that spreadsheet that you want Excel to start up.

Since the shell spends nearly all its time talking about things and comparatively rarely talks to them, an object that spends most of its time "dead" was more appropriate. You might say that the difference between pidls and monikers is a matter of life and death.


Comments (11)
  1. PatriotB says:

    Plus the fact that using monikers instead of PIDLs would’ve been a nightmare given Windows 95’s requirement to fit in 4 MB of RAM.

  2. Igor Tandetnik says:

    You appear to misunderstand the idea of monikers. You seem to believe that, as soon as you have a moniker to Excel file, Excel itself is started. This is not the case. A separate step – a call to IMoniker::BindToObject – is necessary to actually run the underlying server. This is equivalent to ShellExecuteEx call on a PIDL. Until then, the moniker is just like PIDL – a generalized notion of a file path if you will.

  3. BryanK says:

    Excel isn’t started, but the code for the IMoniker implementation must be loaded.

    I don’t know how much code that is (less than a page?), and it’s nowhere near Excel’s footprint.  But I think it is more than the space required for the code behind a PIDL, because AFAIK there is no code behind a PIDL (it’s just plain-old-data).  Could be wrong on that, though.

  4. mattd says:

    Igor,

    Q. What do you think is implementing IMoniker?

    A. A COM object.

    Hence, you would have 1000 COM objects as pointed out.

  5. S. Ganesh says:

    Igor’s point is that you don’t get “a dozen live, running spreadsheet objects”. You do get a dozen file monikers, the code for which is shared. The data within each instance of the file moniker is probably is just plain-old-data.

    [Even if it’s not live running spreadsheet objects, it’s still code that has to be loaded off the disk. And all that just to show a list of files? The principle of “pay for play” applies here. -Raymond]
  6. Igor Tandetnik says:

    You have to load an external DLL to load the moniker class implementation

    Not if the moniker is implemented by shell32.dll, or shlwapi.dll – whoever now implements PIDL manipulation API. It does not have to be a file moniker implemented in ole32.dll – anybody can implement a COM interface. It can very well be an IMoniker implementation wrapping a PIDL internally.

    > And what if there’s an Excel file but you don’t have Excel installed? Does that mean no moniker?

    A moniker does not know nor care which server a particular file belongs to – until you call BindToObject. Until then, it just carries around a file path or whatever information is necessary to locate a file – just like PIDL does. When you call BindToObject and no server can be found to handle the file, it fails – just like ShellExecuteEx does.

    Face it – monikers and PIDLs are two manifestations of the same fundamental concept. One is exposed via COM-based API while the other via C style API, and that’s pretty much the only difference.

    [True, but the COM object is “live” – it’s hard to marshal, it’s hard to copy into a block of memory, good luck sending it to another computer. A pidl is “dead”; it’s just a bunch of bytes. You can copy it around with impunity. It’s the difference between FindFirstFile returning a bunch of HANDLEs and returning a bunch of file names. -Raymond]
  7. Igor Tandetnik says:

    >Hence, you would have 1000 COM objects as pointed out.

    So? Why do you think it matters if you have 1000 PIDLs or 1000 COM
    pointers each wrapping a PIDL? Do you also believe that a 1000
    instances of C++ class are more expensive than a 1000 instances of
    plain old C struct with the same fields? COM is not black magic, a COM
    object is little more than a C++ class instance.

    I admit a moniker would be slightly more expensive (by one vtable
    pointer) than raw PIDL, but not vastly, outlandishly expensive as
    Raymond makes it sound.

    > AFAIK there is no code behind a PIDL

    What do you mean, no code? How do you manipulate a PIDL in your
    application then? What’s really the difference between calling
    SHGetPathFromIDList and calling IMoniker::GetDisplayName?

    Consider:

    struct X {

     int n;

    };

    void DoSomething(X* p);

    class CX {

     int n;

    public:

     void DoSomething();

    };

    You seem to imply that there is “no code” behind X, but there is
    “some code” behind CX. I fail to see the fundamental difference between

    X x;

    DoSomething(&x);

    and

    CX cx;

    cx.DoSomething();

    In fact, the latter is internally implemented very similarly to the former: the compiler generates something like

    void CX_DoSomething(CX* this);

    // cx.DoSomething();

    CX_DoSomething(&cx);

    That’s where ‘this’ pointer inside a C++ method comes from.

    [You have to load an external DLL to load the moniker class implementation. You don’t have to load an external DLL to load the pidl implementation. And what if there’s an Excel file but you don’t have Excel installed? Does that mean no moniker? -Raymond]
  8. Igor Tandetnik says:

    Marshalling could have been implemented with Marshal-By-Value, transferring PIDL to the target process or machine and wrapping it in another moniker instance once there (all transparently behind the scenes).

    IMoniker is derived from IPersistStream – that’s how you store the underlying data in a block of memory.

    Note, I’m not arguing that using PIDLs was a bad decision and that the shell should have used monikers instead. In fact, the issue didn’t occur to me until you raised it. I’m just arguing against your claim that monikers are somehow fundamentally different and completely unsuitable for the task.

    [Oh, you’re talking about using monikers in addition to pidls? That’s just wasteful. You have only 4MB of memory. -Raymond]
  9. Igor Tandetnik says:

    Your article makes a number of false claims about monikers, quite unrelated to whether or not Shell could or should use them. Somebody reading it might get wrong ideas. I’m just trying to set the record straight, is all.

    Shell can happily continue to use PIDLs as far as I’m concerned. I’m not advocating adding moniker support. I’m arguing in principle that your claim about monikers being fundamentally unsuitable, and a few other claims, are wrong. That doesn’t mean somebody has to rush and implement monikers in the shell just because it’s possible.

  10. The other day, Raymond Chen posted Pidls and monikers do roughly the same thing, just backwards.

    And…

  11. The whole discussion misses the real difference between a PIDL and a Moniker, which is, that a PIDL not only contains an identification of an object (i. e. a file name, object pointer etc.) but may contain as much additional data as you please. In practice, this is used to store attributes of the objects the PIDLs refer to in the PIDL structure. In the Windows Shell, you can display files’ attributes or virtual folders’ attributes given the PIDL without "binding" to the target object. In case you have a large list of objects which you want to display in the Windows Explorer Details View, it’s much faster to find the attributes (columns in details view) in the PIDL than to open the objects one by one.

    And of course, every Shell Extension needs a PIDL-Manager Module, which might be a pain to program.

Comments are closed.


*DISCLAIMER: I DO NOT OWN THIS CONTENT. If you are the owner and would like it removed, please contact me. The content herein is an archived reproduction of entries from Raymond Chen's "Old New Thing" Blog (most recent link is here). It may have slight formatting modifications for consistency and to improve readability.

WHY DID I DUPLICATE THIS CONTENT HERE? Let me first say this site has never had anything to sell and has never shown ads of any kind. I have nothing monetarily to gain by duplicating content here. Because I had made my own local copy of this content throughout the years, for ease of using tools like grep, I decided to put it online after I discovered some of the original content previously and publicly available, had disappeared approximately early to mid 2019. At the same time, I present the content in an easily accessible theme-agnostic way.

The information provided by Raymond's blog is, for all practical purposes, more authoritative on Windows Development than Microsoft's own MSDN documentation and should be considered supplemental reading to that documentation. The wealth of missing details provided by this blog that Microsoft could not or did not document about Windows over the years is vital enough, many would agree an online "backup" of these details is a necessary endeavor. Specifics include:

<-- Back to Old New Thing Archive Index