Buffers/Cache Memory

Your system memory is a key component in storing data as processing goes on. But, it is WAY too slow to keep up with the internal calculations being done by your processor. So, a processor contains its own internal cache and multiple buffers to aid it in storing data. Buffers are simply temporary storage areas for instructions that either are incomplete or have not been written to memory. Buffers are used throughout the processor in a variety of purposes and settings. Since the actual calculation units do only one thing at a time, despite the speed at which they do it, a memory buffer is needed to store queued instructions, processed instuctions, etc.

In addition to the various memory buffers built-in to the processor circuitry, there is cache on the processor to serve as temporary storage areas. Cache is broken up into two categories: the Level One cache runs at the processor’s own clock speed, so is very fast. This cache is split up into two sections, one for data, the other for instructions. The more level one cache you have, the faster the chip. Computers usually come with secondary, or Level Two, cache. This cache used to reside on the motherboard itself and was thus confined to the speed of the system bus. To get around this limitation, the Pentium Pro and Pentium II and later processors have their own built in level two cache. The Pentium II and later chips house the level two cache along with the CPU, therefore bypassing the motherboard altogether. The Pentium II has its L2 cache on the daightercard with the processor, and it ran at half the speed of the processor. But, later chips from both AMD and Intel now have the L2 cache built right into the die of the processor, running at full clock speed.

A Bit on Prediction

As I said above, the actual speed of a processor depends on a lot more than its clock speed. One of the factors that determine overall speed is the ability of a processor to predict instructions, to predict what is coming next. Branch prediction is one such methodology used by processors today, and is almost a necessity with today’s highly pipelined chips. The terms refers to the ability of the processor to predict, based on past instructions, what the coming instruction will be in a set. To illustrate the need for this, take a simple IF, THEN statement in programming.

IF A=B THEN
   DO C
ELSE
   DO D
ENDIF

In a pipeline, where multiple instructions are being set up for processing at a time, a statement like this would be a problem. The processor would decode the IF statement by comparing A and B, but it would not know whether it was going to be doing C or D at this point, so it could not put that instruction into the pipeline for rapid execution. These types of statements are very common in programming, and would cause a problem for pipelining. This problem is handled by giving the processor the ability to predict which way the branch will go, thereby allowing it to put that insturction into the pipeline and not degrade performance. This is called branch prediction. A slightly less advanced processor would simply do what is called speculative execution, which would simply be to do both C and D at the same time and then throw out whichever one it doesn’t need when it knows if A is equal to B. With branch prediction, it will make use of a buffer to store the results of a branch when it is processed. This data is then used later to predict the results of future branches in the programming code.