In practice, processors tend to access memory in a very patterned way. For example,
in the absence of branches, the PC in MARIE is incremented by one after
each instruction fetch. Thus, if memory location X is accessed at time t, there is a
high probability that memory location X + 1 will also be accessed in the near
future. This clustering of memory references into groups is an example of locality
of reference. This locality can be exploited by implementing the memory as a
hierarchy; when a miss is processed, instead of simply transferring the requested
data to a higher level, the entire block containing the data is transferred. Because
of locality of reference, it is likely that the additional data in the block will be
needed in the near future, and if so, this data can be loaded quickly from the
faster memory.
There are three basic forms of locality:
• Temporal locality—Recently accessed items tend to be accessed again in the
near future.
• Spatial locality—Accesses tend to be clustered in the address space (for example,
as in arrays or loops).
• Sequential locality—Instructions tend to be accessed sequentially.
The locality principle provides the opportunity for a system to use a small amount
of very fast memory to effectively accelerate the majority of memory accesses.
Typically, only a small amount of the entire memory space is being accessed at any
given time, and values in that space are being accessed repeatedly. Therefore, we
can copy those values from a slower memory to a smaller but faster memory that
resides higher in the hierarchy. This results in a memory system that can store a
large amount of information in a large but low-cost memory, yet provide nearly the
same access speeds that would result from using very fast but expensive memory.