If you choose static linking to the MSVCRT library (C Runtime), you will get the description that you are describing. The same thing happens if your EXE and DLL are dynamically linked to the MSVCRT DLL but use different versions. Or, if they match the same version, but one uses DEBUG and the other uses RETAIL. In other words, memory is used as well as the lifetime of the MSVCRTxxx.dll file used for allocation. I just saw your update to your question - yes, mixing and matching CRT between VS 2008 and 2010 is the exact cause of the failure.
If your DLLs and EXEs are dynamically linked to the same version of the MSVCRT DLL, you are sharing a lot of memory and you are avoiding the problems you have.
The standard practice is this: if the exported DLL function returns everything that needs to be "released" or "released" later, then the standard practice is to provide an extra function exported from the DLL to handle de-allocation.
You can configure the EXE and C Runtime DLL connections from the code page for the C / C ++ project settings in your project.
Image here: http://imgur.com/uld4KYF.png

source share