Delphi Win API CreateTimerQueueTimer streams and thread safe shifts FormatDateTime

Question

This is a little long question, but here we go. There is a version of FormatDateTime called thread safe because you use

GetLocaleFormatSettings(3081, FormatSettings); 

to get the value, and then you can use it like this;

 FormatDateTime('yyyy', 0, FormatSettings); 

Now imagine two timers, one of which uses TTimer (interval, say, 1000 ms), and then the other timer created in this way (interval of 10 ms);

 CreateTimerQueueTimer ( FQueueTimer, 0, TimerCallback, nil, 10, 10, WT_EXECUTEINTIMERTHREAD ); 

Now the Narly bit, if in the callback, as well as a timer event, you have the following code:

 for i := 1 to 10000 do begin FormatDateTime('yyyy', 0, FormatSettings); end; 

Please note that the appointment is missing. This leads to access violations almost instantly, sometimes after 20 minutes, regardless of whether in arbitrary places. Now, if you write this code in C ++ Builder, it never fails. The header conversion we are using is JEDI JwaXXXX. Even if we put locks in the Delphi version around the code, this only delays the inevitable. We looked at the source C header files, and it all looks good, is there any other way that C ++ uses the Delphi runtime? The streaming version of FormatDatTime is re-looking. Any ideas or thoughts from those who may have seen this before.

UPDATE:

To narrow it down a bit, FormatSettings is passed as const, so what does it matter if they use the same copy (as it turns out, passing the local version inside the call yeilds function the same problem)? Also, the version of FormatDateTime that accepts FormatSettings does not call GetThreadLocale, because it already has Locale information in the FormatSettings structure (I double check it through the code).

I mentioned the lack of a destination so that it is clear that access to shared memory is not performed, so no lock is required.

WT_EXECUTEINTIMERTHREAD is used to simplify the problem. I got the impression that you should use it only for very short tasks, because it can mean that it will skip the next interval if it starts something for a long time?

If you use plain old TThread, the problem does not occur. I suppose using TThread or TTimer works, but using a thread created outside of VCL is not, so I asked if there is a difference in how C ++ Builder uses VCL / Delphi RTL.

As an aside, this code, as mentioned earlier, also fails (but takes longer), after some time CS: = TCriticalSection.Create;

  CS.Acquire; for i := 1 to LoopCount do begin FormatDateTime('yyyy', 0, FormatSettings); end; CS.Release; 

And now for a bit that I really don't understand, I wrote it as suggested;

 function ReturnAString: string; begin Result := 'Test'; UniqueString(Result); end; 

and then inside each type of timer there is a code:

  for i := 1 to 10000 do begin ReturnAString; end; 

This leads to the same failures as I said before the error will never be in the same place inside the CPU window, etc. Sometimes this is a violation of access rights, and sometimes it can be a wrong pointer operation. I am using Delphi 2009 btw.

UPDATE 2:

Roddy (below) points to the Ontimer event (and, unfortunately, Winsock, i.e. TClientSocket), uses a Windows message pump (as a side, it would be nice to have some nice Winsock2 components using IOCP and Overlapping IO), therefore, push to get away from him. However, does anyone know how to determine which local thread store is configured on CreateQueueTimerQueue?

Thank you for taking the time to think and answer this problem.

Answer 1

I’m not sure if this is a good form for posting “Answer” to my own question, but it seemed logical, let me know if it’s unscrupulous.

I think I found the problem, the idea of ​​local thread storage prompted me to follow heaps of potential customers, and I found this magic line;

IsMultiThread: = True;

From the reference;

"IsMultiThread is true to indicate that the memory manager should support multiple threads. IsMultiThread is set to true on BeginThread and class factories.

This, of course, is not set using a single Main VCL thread using TTimer, but it is configured for you when you use TThread. If I install it manually, the problem will disappear.

In C ++ Builder, I do not use TThread, but it appears with the following code:

 if (IsMultiThread) { ShowMessage("IsMultiThread is True!"); } 

which is set for you somewhere automatically.

I am very happy for the contribution of people, so that I could find this, and I hope this can help someone else.

Answer 2

Since a DateTimeToString that uses FormatDateTime uses GetThreadLocale, you can try to have a local variable FormatSettings for each thread, maybe even set FormatSettings in a local variable before the loop.

It can also be the WT_EXECUTEINTIMERTHREAD parameter that causes this. Please note that it indicates that it should only be used for very short tasks.

If the problem persists, the problem may be in another place, and this was my first guess when I saw it, but I don’t have enough information about what the code really defines.

Details of where the access violation occurs can help.

Answer 3

Are you sure this really relates to FormatDateTime ? You noted that there is no assignment statement; is an important aspect of your question? What happens if you call some other string return function? (Make sure this is not a constant string. Write your own function that calls UniqueString(Result) before returning.)

Is the FormatSettings variable a variable with a stream? This indicates the presence of an additional parameter for FormatDateTime , so each thread has its own private copy, which will not be changed by any other thread while the function is active.

Is the timer queue important for this issue? Or do you get the same results when you use the plain old TThread and run your loop in the Execute method?

You warned that this is a long question, but there seem to be a few things you could do to reduce it, to narrow down the scope of the problem.

Answer 4

I wonder if the RTL / VCL calls you make will wait for access to some streaming storage variables (TLS) that are incorrectly configured when you call your code through a timer queue?

This is not the answer to your problem, but did you know that TTimer OnTimer events are fired as part of the normal message loop in the main VCL stream?

Answer 5

You found your answer - IsMultiThread. This must be used at any time to return to using the API and create threads. From MSDN: CreateTimerQueueTimer creates a thread pool to handle this function, so you have an external thread working with the main VCL thread without protection. (Note: your CS.acquire / release does nothing at all if other parts of the code do not respect this lock.)

Answer 6

Re. your last question about Winsock and overlapping I / O: you should carefully study Indy .

Indy uses blocking I / O and is a great choice if you want high-performance network I / O no matter what the main stream does. Now you are faced with the threading problem, you just have to create another thread (or more) to use indy to handle I / O.

Answer 7

The problem with Indy is that if you need a lot of connections, this is not effective. It requires one thread for each connection (blocking I / O), which does not scale at all, therefore the advantage of IOCP and Overlapping IO is pretty much the only scalable way on Windows.

Answer 8

For update2:

There are free IOCP socket components: http://www.torry.net/authorsmore.php?id=7131 (including source code)

"Naberezhny Sergey N. High-performance socket server based on the Windows completion port and using the Windows Socket Extension. IPv6 is supported."

I found it while looking at the best components / library to rebuild my small instant messaging server. I have not tried it yet, but it looks good, like a first impression.