What is a cache hit and a cache miss? Why would context-switching cause cache miss?

74
votes

From the 11th Chapter(Performance and Scalability) and the section named Context Switching of the JCIP book:

When a new thread is switched in, the data it needs is unlikely to be in the local processor cache, so a context-switch causes a flurry of cache misses, and thus threads run a little more slowly when they are first scheduled.

  1. Can someone explain in an easy to understand way the concept of cache miss and its probable opposite (cache hit)?
  2. Why context-switching would cause a lot of cache miss?
5
concurrencylanguage-agnosticcpucpu-architecturecpu-cache
Cache hit - found in cache, without having to go "further" (into memory, disk, etc). There isn't much else to it. – Anirudh Ramanathan
See also What Every Programmer Should Know About Memory. It's old but still relevant, except for the part about prefetch threads and so much software prefetching. That part mostly only applies to Pentium4, but everything else about multi-level caches and laying out your data for locality and sequential access still applies. – Peter Cordes

5 Answers

110
votes

Can someone explain in an easy to understand way the concept of cache miss and its probable opposite (cache hit)?

A cache miss, generally, is when something is looked up in the cache and is not found – the cache did not contain the item being looked up. The cache hit is when you look something up in a cache and it was storing the item and is able to satisfy the query.

Why context-switching would cause a lot of cache miss?

In terms of memory, each processor has a memory cache – a high speed copy of small portions of main memory. When a new thread is context switched into a processor, the local cache memory is empty or it doesn't correspond to the data needed for the thread. This means that all (or most) memory lookups made by that new thread result in cache misses because the data that it needs is not stored in the local memory cache. The hardware has to then make a number of requests to main memory to fill up the local memory cache which causes the thread to initially run slower.

4
votes

Whenever the processor wants to fetch data from main memory, first it will look at the cache buffer to see whether the corresponding address is present in the buffer. If it is there, it will perform the operation by using the cache; no need to fetch from the main memory. This is called a "Cache hit".

If the address is not present in the cache, it is called a "Cache miss". If a cache miss has occurred, that means the processor has go to main memory to fetch the address and it takes some more time.

1
votes

You should also observer that if a context switch causes a return of a previously run thread to active state on a processor with access to the cached data, there is a chance that the required "working set" is still in the cache. The probability of this being true depends on the cache size (and structure). It also depends on the workload: How much demand for cache was there during the threads idle or waiting period, and how long the idle or waiting period lasts.

0
votes

If the processor finds that the memory location is in the cache, we say that a cache hit otherwise we speak of a cache miss.

-3
votes

If the desired data is in L1, then it's a cache hit. And if the desired data is in another cache memory level then it's a cache miss.