This blog aims to assist you in understanding the medial concepts of Chapter:8 [Buffer Manager] from the book The Internals of PostgreSQL.
Note: Ensure that you have a thorough understanding of
Chapter 8 Part-1 and basics of PostreSQL before we proceed to Chapter 8 Part-2, as it forms the foundation for our exploration.
So, Let's Continue:
Buffer Descriptors Layer
A collection of buffer descriptors forms an array. In this document, the array is referred to as the buffer descriptors layer.
When the PostgreSQL server starts, the state of all buffer descriptors is empty. In PostgreSQL, those descriptors comprise a linked list called freelist.
Buffer manager initial state in PostgreSQL is depicted in figure below:
Loading the first page in PostgreSQL is depicted in figure below:
Buffer Pool
The buffer pool is a simple array that stores data file pages, such as tables and indexes. Indices of the buffer pool array are referred to as buffer_ids.
The buffer pool slot size is 8 KB, which is equal to the size of a page. Thus, each slot can store an entire page.
Buffer manager locks
Buffer Table Locks
BufMappingLock protects the data integrity of the entire buffer table.
It is a light-weight lock that can be used in both shared and exclusive modes. When searching an entry in the buffer table, a backend process holds a shared BufMappingLock.
The BufMappingLock is split into partitions to reduce the contention in the buffer table (the default is 128 partitions). Each BufMappingLock partition guards the portion of the corresponding hash bucket slots.
Two processes simultaneously acquire the respective partitions of BufMappingLock in exclusive mode to insert new data entries in PostgreSQL is depicted in figure below:
Locks for Each Buffer Descriptor
- Each buffer descriptor uses two light-weight locks, content_lock and io_in_progress_lock, to control access to the stored page in the corresponding buffer pool slot. When the values of own fields are checked or changed, a spinlock is used.
Content_lock
The content_lock is a typical lock that enforces access limits. It can be used in shared and exclusive modes.
When reading a page, a backend process acquires a shared content_lock of the buffer descriptor that stores the page.
Exclusive content_lock is acquired in the following scenarios:
- Inserting rows (tuples) into the stored page.
- Changing the t_xmin/t_xmax fields of tuples within the stored page.
- Removing tuples physically or compacting free space on the stored page.
- Freezing tuples within the stored page.
io_in_progress_lock
The io_in_progress lock is used to wait for I/O on a buffer to complete.
When a PostgreSQL process loads/writes page data from/to storage, the process holds an exclusive io_in_progress lock of the corresponding descriptor while accessing the storage.
Spinlock
When the flags or other fields (e.g. refcount and usage_count) are checked or changed, a spinlock is used.
Two specific examples of spinlock usage are given below:
1. The following shows how to pin the buffer descriptor:
- Acquire a spinlock of the buffer descriptor.
- Increase the values of its refcount and usage_count by 1.
- Release the spinlock.
2. The following shows how to set the dirty bit to '1':
- Acquire a spinlock of the buffer descriptor.
- Set the dirty bit to '1' using a bitwise operation.
- Release the spinlock.
How the buffer manager works
When a backend process wants to access a desired page, it calls the ReadBufferExtended function.
The behavior of the ReadBufferExtended function depends on three logical cases.
1. Accessing a Page Stored in the Buffer Pool
First, the simplest case is described, i.e. the desired page is already stored in the buffer pool.
In this case, the buffer manager performs the following steps:
(1) Create the buffer_tag of the desired page (in this example, the buffer_tag is 'Tag_C') and compute the hash bucket slot, which contains the associated entry of the created buffer_tag, using the hash function.
(2) Acquire the BufMappingLock partition that covers the obtained hash bucket slot in shared mode (this lock will be released in step (5)).
(3) Look up the entry whose tag is 'Tag_C' and obtain the buffer_id from the entry. In this example, the buffer_id is 2.
(4) Pin the buffer descriptor for buffer_id 2, i.e. the refcount and usage_count of the descriptor are increased by 1.
(5) Release the BufMappingLock.
(6) Access the buffer pool slot with buffer_id 2.
Accessing a page stored in the buffer pool in PostgreSQL is depicted in figure below:
When reading rows from the page in the buffer pool slot, the PostgreSQL process acquires the shared content_lock of the corresponding buffer descriptor. Thus, buffer pool slots can be read by multiple processes simultaneously.
When inserting (and updating or deleting) rows to the page, a Postgres process acquires the exclusive content_lock of the corresponding buffer descriptor (note that the dirty bit of the page must be set to '1').
After accessing the pages, the refcount values of the corresponding buffer descriptors are decreased by 1.
2. Loading a Page from Storage to Empty Slot
In this second case, assume that the desired page is not in the buffer pool and the freelist has free elements (empty descriptors).
In this case, the buffer manager performs the following steps:
(1) Look up the buffer table (we assume it is not found).
- Create the buffer_tag of the desired page (in this example, the buffer_tag is 'Tag_E') and compute the hash bucket slot.
- Acquire the BufMappingLock partition in shared mode.
- Look up the buffer table (not found according to the assumption).
- Release the BufMappingLock.
(2) Obtain the empty buffer descriptor from the freelist, and pin it. In this example, the buffer_id of the obtained descriptor is 4.
(3) Acquire the BufMappingLock partition in exclusive mode (this lock will be released in step (6)).
(4) Create a new data entry that comprises the buffer_tag 'Tag_E' and buffer_id 4; insert the created entry to the buffer table.
(5) Load the desired page data from storage to the buffer pool slot with buffer_id 4 as follows:
Acquire the exclusive io_in_progress_lock of the corresponding descriptor.
Set the io_in_progress bit of the corresponding descriptor to '1 to prevent access by other processes.
Load the desired page data from storage to the buffer pool slot.
Change the states of the corresponding descriptor; the io_in_progress bit is set to '0', and the valid bit is set to '1'.
Release the io_in_progress_lock.
- (6) Release the BufMappingLock.
- (7) Access the buffer pool slot with buffer_id 4.
Loading a page from storage to an empty slot in PostgreSQL is depicted in figure below:
I hope, this blog has helped you in understanding the medial concepts of Buffer Manager in PostreSQL.
Check out summary of Chapter : 8 Part-3
If you want to understand PostgreSQL In-Depth.
Top comments (0)