Memory Hierarchy: Performance Optimization

Memory hierarchy. Optimization involves the memory hierarchy.

We see an example memory hierarchy in a fairly modern computer. This guides optimization of computer programs. It also influences how compilers, operating systems and databases work.

Virtual Memory: Disk

> 2GB (Typical size)
3-15 ms (Access time)

Physical Memory

256MB-2GB
100-150 ns

2nd-Level Cache

128KB-4MB
40-60 ns

1st-Level Cache

16-64KB
5-10 ns

Registers: Processor

32 Words
1 ns

Intro. The above table shows the critical performance aspects of a somewhat recent memory hierarchy on a personal computer. You can see that reading a file from the disk requires about 3-15 milliseconds.

Compilers: Principles, Techniques and Tools

So: Reading data from memory requires 100-150 nanoseconds. And reading data from the processor caches requires 1-60 nanoseconds.

How much faster is physical memory than the disk? Here we use the table shown above to compute how much faster you can load a small data object from memory instead of the disk. Parsing time is not included.

Please recall that one millisecond is equal to one million nanoseconds. Let us say that the computer is busy and the disk requires 15 milliseconds, and the memory requires 150 nanoseconds.

Disk time

15 ms * 1000000 = 15000000 ns

Memory time

150 ns

Memory performance

Memory is 100,000x (one-hundred thousand) times faster.

Uses. There is more complexity to computers than this memory hierarchy. All modern operating systems use a file cache, which puts disk files into memory, essentially making accesses one-hundred thousand times faster.

However: This is not configurable in many cases and it can be better to use custom file caches that will not be expired.

File Caching

Summary. We explored the memory hierarchy using a table from the book Compilers: Principles, Techniques and Tools. The memory hierarchy shows us that disk accesses are far more expensive than in-memory reads with similar complexity.

Often: Avoiding accessing virtual memory (disk) is an important optimization to make.

Related Links