ShellExecuteEx on Windows 7 with default image / image viewer

Question

We have a desktop application running on Windows XP, Windows Vista, Windows 7, etc. (it is written in cobol, but I think that is not relevant in this case).

In our source code, we call ShellExecuteEx with open .
After that - to wait for the called program to end - let's call WaitForSingleObject with infinite .

It works without problems, only on windows-7-machines with images and in the default image view mode we have problems.

If ShellExecuteEx is called there for a jpg or tif file, and on the computer there is only a default image / photo viewer, an error occurs. WaitForSingeObject returns WAIT_FAILED and GetLastError() says: INVALID_HANDLE .

This DOES NOT PASS on Windows 7 when there is another image viewer, for example, Picasa Photo Viewer or Evince version for Windows or JPEGView.

I do not know where the problem is with the default image viewer in Windows 7 and ShellExecuteEx . ShellExecuteEx does NOT return an error code!

Thanks in advance.

Update: Thanks for your answers.

@ David Heffernan: As you said, in the case when it does NOT work, ShellExecuteEx has a return code (hInstApp) of 42 (ok!), And hProcess is NULL! (Only a new process has begun - dllhost.exe.)

But I tested all this on another Windows 7 machine. In this case, I did the following:
- On this computer, Picasa was the default viewer.
- I switched to standard (Windows Photo Viewer).
- It worked!
- Then I did NOT close the photo viewer and clicked "show" in our application AGAIN. - It also worked (a new instance of the viewer appeared), even when Windows Photo Viewer was already running!

Answer 1

The default handler for the verb open on images in modern versions of Windows cannot invoke a new process. It can simply show the image in an already running shell process. And when that happens, the return handle to the process is NULL . This is what happens here, and so the WaitForSingleObject call does not work the way you describe.

The documentation for the SHELLEXECUTEINFO describes this:

Even if fMask is set to SEE_MASK_NOCLOSEPROCESS, hProcess will be NULL if the process has not been started. For example, if the document to be launched is a URL and an instance of Internet Explorer is already running, it will display the document. No new process starts and hProcess will be NULL.

All this means that the design of this part of your program is based on an erroneous assumption. Namely, the assumption that a call to ShellExecuteEx will always give a handle to the process where you can wait for completion. You will need to find another way to solve your problem.