I think this argument is important and deserves some place here.
Consider the most common design of input / output I / O ports in C / C ++, having a structure (or class) that abstract the HANDLE and some of its properties, for example:
class Stream { enum { Open = 1, Closed = 0 }; // Dtor virtual ~Stream() { if (m_read_packet != 0) delete_packet(m_read_packet); // the same for write packet } // Functions: bool read(...) { if (m_read_packet != 0) m_read_packet = allocate_packet(); ReadFile(m_handle ...); } bool write(...); bool close() { m_state = Closed; if (!CloseHandle(m_handle)) return false; else return true; } // Callbacks: virtual void onRead(...); virtual void onWrite(...); virtual void onEof(); virtual void onClose(); // Other stuff .... // Properties: HANDLE m_handle; int m_state; int m_pending_reads; int m_pending_writes; IoPacket * m_read_packet; IoPacket * m_write_packet; };
It's pretty simple, you have this class that abstracts the HANDLE, and the IO I / O Port Streams trigger callbacks when I / O is completed. In addition, you can cache and reuse the OVERLAPPED structures (obtained by IoPacket) stored in these pointers.
You use this class by highlighting the Stream object yourself or simply letting other classes allocate it after the connection arrives (for example, AcceptEx () or the named pipe server). The Stream object will automatically begin to perform I / O based on the implementation of its callbacks.
Now in both cases you choose [1) allocate Stream yourself (client) or 2) from other objects (servers)] there will be a common problem: you do not know exactly when you can safely free the Stream object from memory. And this is true even if you use atomic ref counts for IoPacket objects.
Let's see why: when doing i / o, the pointer to the Stream object will be contained in IoPacket objects, which, in turn, are processed and used by other threads in threadpool, which, in turn, uses this pointer to call callbacks [onRead (), onWrite (), etc ....]. You also have the m_pending_reads / write variables to see how many are waiting to read / write them. Thus, the pointer to the Stream object will be shared by the streams.
At this point, consider that you no longer need this connection, and you want to close and release the associated Stream object.
how to make it safe?
I mean, if you just do:
stream->close(); delete stream;
you will get into some crashes because another stream can still use the stream pointer. A safe way would be the following:
stream->close(); while ((stream->m_pending_reads != 0) && (stream->m_pending_writes != 0)); delete stream;
But this is completely inefficient, since it blocks the executeg path, blocks-expects other threads to complete their work with the thread object. (And besides, I think I’ll add some kind of volatile or barrier mechanism?) The best way is to do this asynchronously, with another thread terminating the LAST IoPacket for this Stream object and calling onClose (), which in turn will :
void onClose() { delete this; }
So my question is: how do I recognize that I am processing (in any thread) LAST IoPacket (or OVERLAPPED) for a given descriptor, so after I was able to safely call onClose (), which removes Stream objects, which dtor in turn removes cached IoPackets used for working with i / o.
Now I use this method: if (to read the terminations) `GetQueuedCompletionStatus () 'returns ERROR_OPERATION_ABORTED [so that means CloseHandle (m_handle); was called] OR bytes_read == 0 [this means EOF] OR if (state! = Open) THEN this means that it was the last pending read batch and then calls onClose (); after processing one. Record packets will ignore onClose ().
I am not sure if this is the right method, because even if the read packets are those that will always wait until the i / o event occurs, it may happen that onClose () will be called by some thread scheduled before another thread, which is ready to process the last recording packet.
What about these derived Stream objects (like TcpClient) that also have other IoPackets? for example one for ConnectEx ().
A possible solution would be to have AtomicInt m_total_pending_requests; <- and when this atomic integer reaches 0, the termination thread calls onClose ();
But is this last solution ineffective? Why do I use atomic integers to accomplish such a basic thing as closing a connection and freeing its structures?
I may be completely mistaken in the design for IOCP with these Stream objects, but I think this is a fairly common design for abstracting HANDLE, so I wanted to hear the opinion of the musical IOCPs. Should I change everything, or could this project of Stream objects work quite solidly, safely and, most importantly, quickly? I ask about this because of the difficulty that arose when creating such a basic thing as closing + free memory.