Multithreaded Asynchronous I/O & I/O Completion Ports | Dr Dobb's (drdobbs.com)
I/O completion ports provide an elegant solution to the problem of writing scalable server applications that use multithreading and asynchronous I/O.
When developing server applications, it is important to consider scalability, which usually boils down to two issues. First, work must be distributed across threads or processes to take advantage of today's multiprocessor hosts. Second, I/O operations must be scheduled efficiently to maximize responsiveness and throughput. In this article, I examine I/O completion ports—an elegant innovation available on Windows that helps you accomplish both of these goals.
I/O completion ports provide a mechanism that facilitates efficient handling of multiple asynchronous I/O requests in a program. The basic steps for using them are:
Multiple threads may monitor a single I/O completion port and retrieve completion events—the operating system effectively manages the thread pool, ensuring that the completion events are distributed efficiently across threads in the pool.
A new I/O completion port is created with the CreateIoCompletionPort API. The same function, when called in a slightly different way, is used to associate file descriptors with an existing completion port. The prototype for the function looks like this:
**HANDLE** CreateIoCompletionPort(
**HANDLE** FileHandle,
HANDLEExistingCompletionPort,
**ULONG_PTR** CompletionKey,
**DWORD** NumberOfConcurrentThreads);
When creating a new port object, the caller simply passes INVALID_HANDLE_VALUE for the first parameter, NULL for the second and third parameters, and either zero or a positive number for the ConcurrentThreads parameter. The last parameter specifies the maximum number of threads Windows schedules to concurrently process I/O completion events. Passing zero tells the operating system to allow at least as many threads as processors, which is a reasonable default. For a discussion of why you might want to schedule more threads than available processors, see Programming Server-Side Applications for Windows 2000 by Jeffrey Richter and Jason D. Clark.
Once a port has been created, file descriptors opened with the FILE_FLAG_OVERLAPPED (or WSA_FLAG_OVERLAPPED for sockets) may be associated with the port via another call to the same function. To associate an open file descriptor (or socket) with an I/O completion port, the caller passes the descriptor as the first parameter, the handle of the existing completion port as the second parameter, and a value to be used as the "completion key" for the third parameter. The completion key value is passed back when removing completed I/O requests from the port. The fourth parameter is ignored when associating files to completion ports; a good idea is to set this to zero.
Once a descriptor is associated with a port, (and you may associate many file descriptors with a single I/O Completion Port), an asynchronous I/O operation on any of the descriptor(s) results in a completion event being posted to the port by the operating system. The same Windows APIs that let callers perform standard synchronous I/O have a provision for issuing asynchronous I/O requests. This is accomplished by passing a valid OVERLAPPED pointer to one of the standard functions. For example, take a look at ReadFile:
**BOOL** ReadFile(
**HANDLE** File,
**LPVOID** pBuffer,
**DWORD** NumberOfBytesToRead,
**LPDWORD** pNumberOfBytesRead,
LPOVERLAPPED pOverlapped);
For typical (synchronous) I/O operations, you've always passed NULL for the last parameter, but when doing asynchronous I/O, you need to pass the address of an OVERLAPPED structure in order to specify certain parameters as well as to receive the results of the operation. Asynchronous calls to ReadFile are likely to return FALSE, but GetLastError returns ERROR_IO_PENDING, indicating to the caller that the operation is expected to complete in the future.
A common mistake when using OVERLAPPED structures is to pass the address of an OVERLAPPED structure declared on the stack: