In real world memory performance issues plays a major role in system performance. Unix system memory contains 2 types , one is physical memory which is attached to the DIMM modules of the hardware and second is swap space which is a dedicated space in the disk which is treated as a memory by OS ( since the disk I/O is much slower than the I/O to the memory in generic way we will prefer to use swap space less frequently as possible).
tivity
Swap space is only used when the physical memory is too small to accommodate system memory requirements . At that time space is freed in physical memory by paging (moving) it out to swap space ( also keep it in mind that if the increase in paging to swap space will degrade system CPU performance )
vmstat command
***********
bash-3.2# swap -s
total: 267032k bytes allocated + 86184k reserved = 353216k used, 964544k available
tivity
Swap space is only used when the physical memory is too small to accommodate system memory requirements . At that time space is freed in physical memory by paging (moving) it out to swap space ( also keep it in mind that if the increase in paging to swap space will degrade system CPU performance )
vmstat command
***********
vmstat reports virtual memory statics regarding kernel thread, virtual memory,disk,thread,cpu activity etc. Also note that in multiple CPU systems this will show the average of the number of CPU output.
The details of the command is given below
kthr - This indicates the kernel threads details in 3 states .
r - The number of kernel threads in run queue
b - The number of blocked threads which are waiting for I/O paging
w - Number of swapped out lightweight processes (LWP - processes which is running under same kernel thread and shares its system resources and addresses with other lwp's) that are waiting for resources to finish
memory - Report the usage of real and virtual memory
swap - available swap space ( in kb)
free - size of the free list (in kb)
page - Report about the page faults and paging activity . Details of this section is given below
re - page reclaims
mf - minor faults
pi - kilobytes paged in
po - kilobytes paged out
fr - kilibytes freed
de - anticipated short term memory shortfall ( in KB)
sr- pages scanned by clock algorithm
disk - Reports the number of disk operations per second . There are slots up to 4 disks with a letter and number ( letter indicates the disk type like scsi.ide ) and number is the logical number
faults - Reports trap/interrupt rates
in - interrupts
sy - system calls
cs- cpu context switches
cpu - Breakdown usage of the cpu time. In multi processor systems it will be average of all the CPU's
us - user time
sys - system time
id - idle time
swap usage analysis
****************
If you concentrate in swap analysis we need to use below mentioned two commands .
total: 267032k bytes allocated + 86184k reserved = 353216k used, 964544k available
bash-3.2# swap -l
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 181,1 8 2097144 2097144
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 181,1 8 2097144 2097144
But there is major difference between these two commands , in the first one we are using 353216k of (964544 + 353216)k which means of 26% in use. In the second one you can see as all the 2097144 is free means 0% is used. In the first command (swap -s ) includes the portion of physical memory also which is using as swap. The major difference in usage of these two commands are in generic if you are checking the swap usage over time you can use swap -s. (if the system performance is good). But if the system performance is degraded you need to concentrate more about the change in swap usage and what causes that change ( also keep it in mind that swap -l displays output in 512 bytes and swap -s displays in 1024 byte blocks) .
If the system run's out of swap space it will show the error messages given below and we might think about expanding the same using creating the swap file. In general while creating the swap you have to provide size as half of the system physical memory
for example if the system memory is 8GB the ideal swap size should be 4GB
***********************************************************
application is out of memory
malloc error O
messages.1:Sep 21 20:52:11 mars genunix: [ID 470503 kern.warning]
WARNING: Sorry, no swap space to grow stack for pid 100295 (myprog)
***********************************************************
If the system run's out of swap space it will show the error messages given below and we might think about expanding the same using creating the swap file. In general while creating the swap you have to provide size as half of the system physical memory
for example if the system memory is 8GB the ideal swap size should be 4GB
***********************************************************
application is out of memory
malloc error O
messages.1:Sep 21 20:52:11 mars genunix: [ID 470503 kern.warning]
WARNING: Sorry, no swap space to grow stack for pid 100295 (myprog)
***********************************************************
Creating the swap file
1.Login as super user
2. Create the swap file using mkfile <name> <size in k/m/g> filename
3. Activate the swap file using /usr/sbin/swap -a /path/filename
4. Add the entry at /etc/vfstab
/path/filename - - swap - no -
5. Verify the swap file using /usr/bin/swap -l
As a nutshell while configuring the swap , please keep it in below points
- Never allocate swap with size less then 30% of RAM.
- Determine whether large applications (such as compilers and databases) will be using the /tmp directory. If one or several of your application have a huge demand for swap space, use the swap -s command to monitor swap resources on a similar existing system tro get estimate of the actual requirements.
Cache
If we check the free -m command in a unix box we can see major portion of the memory is in cached column. So what is mean by that cache, is it currently used by system?
[root@testserver ~]# free -m
total used free shared buffers cached
Mem: 15976 15195 781 0 167 9153
-/+ buffers/cache: 5874 10102
Swap: 2000 0 1999
In this case you can see 9GB is cached . These caches are called page caches / dirty caches which will act as a temporary memory for read and write process. During the write process the contents of these dirty cache will be periodically transferred to the system storage . Till 2.6.31 version , the process called pdflush will ensure that the data is transferring to system storage and clearing the dirty pages periodically. But after this kernel version there will be a thread for each device ( like sda/sdb) will monitor this mechanism
root@pc:~# ls -l /dev/sda brw-rw---- 1 root disk 8, 0 2011-09-01 10:36 /dev/sda root@pc:~# ls -l /dev/sdb brw-rw---- 1 root disk 8, 16 2011-09-01 10:36 /dev/sdb root@pc:~# ps -eaf | grep -i flush root 935 2 0 10:36 ? 00:00:00 [flush-8:0] root 936 2 0 10:36 ? 00:00:00 [flush-8:16]
This same mechanism is applicable for reading also, file blocks will be transferred from disk to page cache for reading . For example if you access 100MB file twice , in second time the access will be faster as it is fetching from the cache. If Linux needs more memory for normal applications than is currently available, areas of the Page Cache that are no longer in use will be automatically deleted.
Mostly log files or database dump file ( data files) are mostly accumulated by page cache as it is accessed continuously . So configuring perfect log rotate or zipping it periodically will release the page cache when it will be really needed for system performance .
