Computer Architecture - Lecture 9: Memory Latency (Fall 2023)

Lecture 9: Memory Latency
Date: October 26, 2023

Lecture 9: Memory Latency

Recommended Reading:
====================
Intelligent Architectures for Intelligent Computing Systems

A Modern Primer on Processing in Memory

RowHammer: A Retrospective

RECOMMENDED LECTURE VIDEOS & PLAYLISTS:
========================================
Computer Architecture Fall 2021 Lectures Playlist:

Digital Design and Computer Architecture Spring 2022 Livestream Lectures Playlist:

Featured Lectures:

Interview with Professor Onur Mutlu:

The Story of RowHammer Lecture:

Accelerating Genome Analysis Lecture:

Memory-Centric Computing Systems Tutorial at IEDM 2021:

Intelligent Architectures for Intelligent Machines Lecture:

Computer Architecture Fall 2020 Lectures Playlist:

Digital Design and Computer Architecture Spring 2020 Lectures Playlist:

Public Lectures by Onur Mutlu, Playlist:

Computer Architecture at Carnegie Mellon Spring 2015 Lectures Playlist:

Rethinking Memory System Design Lecture @stanfordonline :

[joechang]

lack of appreciate of the role of memory latency is the saddest part of modern computing.
first we should acknowledge that latency at the DRAM interface is one thing, but the ultimate metric is latency to the core, including delays in checking cache. the slide shows mid-50ns latency, which is at the DRAM interface?
Desktop system might show 65-75ns latency to the core?
High-core count processors in a multi-socket system might have 90ns to local node and 120ns+ to one-hop remote.
Latency increases at high load, all cores active?
with the latency contribution outside the DRAM, lowering latency at the DRAM chip has muted impact. First, we must acknowledge that the era of multi-socket systems must end. With very high core count processor, there is no true need for multi-socket, especially when it comes with negative impact of increased memory latency.
Once the systems of concern are single socket, it makes sense to lobby for low latency DRAM. It might also be good to give up socketed DRAM and have the DRAM chips in the processor package, for which we must make the permanent decision on processor & memory at time of purchase forgoing upgrade possibility.

up to the 90's, low price to enable affordability of adequate memory capacity was the really the primary objective.
afterwards, people continue to regurgitate that this was still the objective.
I recall in the early 2000's, DRAM vendors raised the question of whether lower latency would justify higher cost. Too many people said no, but based on old obsolete rules, rather than current quantitative data.