Is the CPU keeping on checking some status to see if the I/O call is returned?
I am asking this question just because I want to understand why async mode server (like nodejs) can handle more concurrent request.
If the CPU is just idle when a synchronized I/O call is not returned, then it would be fine that the server can start a new thread for handing new request as it still have enough CPU resources.
If you do synchronous I/O, then your process (or thread) is blocked until the operation completes. That's why it's called synchronous. What is the CPU doing during that time? Servicing other processes. The same way most modern operating systems "schedule" work from multiple processes so none of them is completely deprived even if there are more jobs than processors, scheduling happens when I/O causes a process to no longer be runnable. If no runnable process is available, the processor can just idle for a while.
Asynchronous (aka non-blocking, evented) I/O is different. There, within a single process you have the ability to do more work while waiting for I/O events. It's like you're scheduling work within your own process, rather than depending on the OS to choose what work to do while you wait for I/O. Node.js can handle many concurrent connections because all I/O is done in non-blocking mode, so a single Node.js process can achieve a high level of utilization of a single processor core, and is never stuck waiting for anything (except truly CPU-bound computation, of course, but most Node.js programs don't have terribly much of that).