Saurabh Singhal's profilesaurabh singhal's spacePhotosBlogListsMore ![]() | Help |
|
March 07 Sql Server memory 101Recently a customer wanted to know some details on how SqlServer manges memory, sort of the basics. Presented below is some of the starter resources for learning about it, I would be happy to hear any specific questions/comments that anyone has after going thru the material below.
thanks
Saurabh
The following blog from Slava is a very good starting point that explains the basics of memory management in Sql Server 2005: http://blogs.msdn.com/slavao/archive/2005/02/11/371063.aspx
So in effect: SQL Server 2005 dynamically acquires and frees memory as required and typically, an administrator does not have to specify how much memory should be allocated to SQL Server. Nonetheless SQL Server provides two memory settings (max and min server memory BOL link) that the user can control using sp_configure. These control the size of buffer pool, which is the preferable provider for all dynamic allocations inside Sql Server. Besides that Sql Server also reacts to VAS pressure on the box or any external physical memory pressure signaled by underlying Windows OS on the box (details of which you can find at http://blogs.msdn.com/slavao/archive/2005/02/19/376714.aspx)
More from BOL: “When SQL Server starts, it computes the size of virtual address space for the buffer pool based on a number of parameters such as amount of physical memory on the system, number of server threads and various startup parameters. SQL Server reserves the computed amount of its process virtual address space for the buffer pool, but it acquires (commits) only the required amount of physical memory for the current load. The instance then continues to acquire memory as needed to support the workload. As more users connect and run queries, SQL Server acquires the additional physical memory on demand. A SQL Server instance continues to acquire physical memory until it either reaches its max server memory allocation target or Windows indicates there is no longer an excess of free memory; it frees memory when it has more than the min server memory setting, and Windows indicates that there is a shortage of free memory. As other applications are started on a computer running an instance of SQL Server, they consume memory and the amount of free physical memory drops below the SQL Server target. The instance of SQL Server adjusts its memory consumption. If another application is stopped and more memory becomes available, the instance of SQL Server increases the size of its memory allocation. SQL Server can free and acquire several megabytes of memory each second, allowing it to quickly adjust to memory allocation changes.” For further details, you can refer to following article: http://msdn2.microsoft.com/en-us/library/aa337525.aspx February 08 NUMA and SqlserverRecently there was an issue in our labs related to NUMA and what is treated as NUMA by SqlServer2005. Here are some findings:
“every x64 machine is a NUMA machines” - False
“all dualcore+ AMD boxes are treated as NUMA”- False
Then what is considered as NUMA by Sql Server? 1) Any real “NUMA hardware”** · Low-end NUMA check: SQL Server by default ignores NUMA configuration when hardware NUMA has four or less CPUs*** and at least one node has only one CPU***. · All AMD64 machines use NUMA design so any m/c satisfying low end NUMA check above would be considered a NUMA box from Sql Server perspective · To override the low-end NUMA check use T8021
2) Any soft NUMA configuration: · You usually configure soft-NUMA when you have many CPUs and do not have real hardware NUMA (Done thru registry settings) · You can also use soft-NUMA to subdivide hardware NUMA nodes into smaller groups.
And here's a useful bit from Jason:
How do I know if the hardware I am running on is NUMA?
o You can look at hardware specs. Note, your hardware may be NUMA but may be configured to use interleaved memory. In that case Windows will not recognize it as NUMA and therefore SQL Server. One quick way to check this is that if you have only one memory node, then the hardware is configured as non-NUMA. You can run the following query to find the number of memory nodes
Select distinct memory_node_id
From sys.dm_os_memory_clerks
Please contact your hardware vendor to find out how to enable NUMA at hardware level.
o During startup SQL Server will configure itself based on underlying OS and hardware configuration. It will create the environment inside of itself to mimic the actual hardware - SQL Server will create a software abstraction, for purpose of our discussion we will call it a Node, around every NUMA node and its memory.
You can look at errorlog to see if it has been booted with multiple NUMA nodes. Here is an example of the NUMA specific information with two nodes and with 1 CPU per node.
2005-09-13 16:08:49.23 Server Multinode configuration: node 0: CPU mask: 0x00000002 Active CPU mask: 0x00000002. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.
2005-09-13 16:08:49.23 Server Multinode configuration: node 1: CPU mask: 0x00000001 Active CPU mask: 0x00000001. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required
o Not all versions of SQL Server are NUMA aware. If you don't have SQL2000 post SP3 QFE that has NUMA optimizations you shouldn't be running SQL Server on the pure NUMA configuration. You may want to configure your system for interleaved memory. In general, SQL2000 has very limited set of NUMA optimizations and we recommend you to use SQL2005 to take full advantage of NUMA configuration.
o Note that the NUMA Node id assigned by SQL may not match the Node id assigned by Windows. SQL normally tries to boot on the non-default node (a NUMA Node other than Node 0).
**NUMA hardware has more than one system bus, each serving a small set of processors. Each group of processors has its own memory and possibly its own I/O channels, but each CPU can access memory associated with other groups in a coherent way. Each group is called a NUMA node ***CPU being the logical representation so [1 socket with 1 core and no HT will be 1 CPU] and [1 socket with 2 cores and no HT will be 2 CPUs] and [1 socket with 1 core and HT enabled will be 2 CPUs] and so forth November 10 The basics of basicsOne of most basic component in SQLOS is memory object. Every introduction to memory object (refer http://blogs.msdn.com/slavao/archive/2005/02/11/371063.aspx) starts with the fact that "SQLOS's memory object is nothing but a heap". And then goes on to explain the how SQLOS supports three types of memory objects (fixed size, variable size, and incremental type) and how its related to another component in Sqlos space called memory clerk.
So far so good, but one thing that was slightly hazy was why do we need something like memory object (or heap) when we can use virtual memory functions to manage our VAS (virtual address space).
The short answer is that the granularity of smallest allocable chunk of memory using virtual memory management functions is 64K. So if you need to have a smaller granularity than that, heap is your answer. You can allocate any amount of memory even if it’s just a few handful of bytes and heap manager will satisfy that request (internally committing additional pages of memory as needed)
There is an old (but kind of relevant) article on MSDN that explains in detail the heap memory management in Win32 (http://msdn2.microsoft.com/en-us/library/ms810603.aspx)
Coming back to Sql Server and Sqlos, "memory objects are nothing but a heap" :-). But in addition to providing the allocation granularity, it also provides things like type of allocation (fixed/variable/incremental), debugging support, caching support or support for synchronization by wrapping existing memory object into a sort of proxy that has the required additional support. November 03 Hello Live Blog!My very first attempt at blogging....will try to keep things simple here. Will use this space for dumping things I learn everyday at work - primarily related to (but not limited to!) Sql Server, and the core of it (better known as SQLOS or SOS for short). Hopefully someday I can reflect back and find this useful :-)
|
|
|