Modern CPUs are superscalar, i.e., they can process multiple instructions in a single cycle, e.g., the add instruction has a throughput of 0.25 on recent Intel CPUs which means that up to four additions can be processed in a single cycle. It is usually stated in cycles-per-instruction (CPI).

This is called the throughput of an instruction. This is called the latency of an instruction.Īfter the first instruction, every following instruction only needs one additional cycle to finish as the fetch and exec stages are already done in parallel to the previous instruction. One instruction still requires three cycles to be processed. Now, the processor only needs five cycles instead of nine to process the sequence of instructions. Hence, the CPU executes parts of multiple instructions in parallel leading to a reduced execution time of the whole sequence. In the next cycle, the exec stage exec 1 and, additionally, the fetch stage of the next instruction fetch 2 can start processing. Such a simple pipeline with the three stages fetch, exec and retire is illustrated in the following timeline diagram.Ġ 1 2 3 4 5 fetch 1 fetch 2 fetch 3 exec 1 exec 2 exec 3 retire 1 retire 2 retire 3Īfter the first cycle, the fetch stage of the first instruction fetch 1 is done. It consists of multiple smaller steps, e.g., fetching operands, executing the operation on the operands and retiring the instruction by writing the result to the destination. The main insight pipelining is based upon is that one instruction is not a single block of work.

The processor needs nine cycles to finish the sequence. The following timeline diagram illustrates how the sequence is processed.Ġ 1 2 3 4 5 6 7 8 9 instruction 1 instruction 2 instruction 3 Let’s assume we have a sequence of three instructions, each taking three cycles to be processed. Pipeliningīefore we stare in wonder at the output of analysis tools, let us briefly recap some basic concepts in processor design, starting with pipelining. Static machine code analyzers let us take a look at how our code is executed by modeling the various execution stages of a CPU. They can execute multiple instructions in a single cycle if there are no dependencies. They reorder instructions in an ever-increasing instruction window and speculatively execute following iterations of a loop by predicting the branch of the loop condition.īoth features are meant to extract as much instruction parallelism from the program code as possible to feed superscalar CPUs with enough work.

Intel c compiler __sync_fetch_and_add