User Tools

Site Tools


tutorials:troubleshooting:memory

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
tutorials:troubleshooting:memory [2015/09/04 00:47]
miconda [Memory Manager Debugging]
tutorials:troubleshooting:memory [2019/06/12 13:26]
qxork updated link to logicmonitor blog article
Line 46: Line 46:
 <code> <code>
 kamctl stats shmem kamctl stats shmem
 +kamcmd mod.stats all shm
 </code> </code>
  
Line 52: Line 53:
 <code> <code>
 kamcmd pkg.stats kamcmd pkg.stats
 +kamcmd mod.stats all pkg
 </code> </code>
  
 Notice that for SHM only one group of statistics is printed, being one zone of memory, while for PKG you get a list with many groups of statistics, each specific for a Kamailio process (child). Notice that for SHM only one group of statistics is printed, being one zone of memory, while for PKG you get a list with many groups of statistics, each specific for a Kamailio process (child).
 +
 +In order to merge the free memory fragments one should enable memory join. Default is disabled (mem_join=0).
 +<code>
 +mem_join=1
 +</code>
 +
  
 ===== Analysis of Memory Incidents ===== ===== Analysis of Memory Incidents =====
Line 77: Line 85:
  
 <code> <code>
-MEMMNG=1 MEMDBG=1 make cfg ...+MEMDBG=1 make cfg ...
 </code> </code>
  
Line 98: Line 106:
  
 Then restart and wait a bit for getting some traffic processed. Then restart and wait a bit for getting some traffic processed.
 +
 +MEMMNG=0/1/2 to select from different memory allocation algorithms (fm, qm, tlsf) is deprecated. Use "-x" parameter when running kamailio instead; see kamailio -h for more details.
  
 To get the list of chunks from memory manager, there are two ways: To get the list of chunks from memory manager, there are two ways:
Line 157: Line 167:
 20(4082) NOTICE: fm_status:  count=     1 size=       704 bytes from <core>: cfg/cfg_struct.c: cfg_shmize(217) 20(4082) NOTICE: fm_status:  count=     1 size=       704 bytes from <core>: cfg/cfg_struct.c: cfg_shmize(217)
 20(4082) NOTICE: fm_status:  count=     3 size=        64 bytes from usrloc: udomain.c: build_stat_name(51) 20(4082) NOTICE: fm_status:  count=     3 size=        64 bytes from usrloc: udomain.c: build_stat_name(51)
 +</code>
 +
 +If you dumped the status with qm_malloc, you can extract the logs from syslog and count the unique allocations with next commands:
 +
 +<code>
 +grep qm_status /var/log/syslog >qm_status.txt
 +
 +# or:
 +# grep qm_status /var/log/messages >qm_status.txt
  
 +grep alloc qm_status.txt | awk '{ print substr( $0, 16, length($0) ) }' | sort | uniq -c | sort -k1n
 </code> </code>
 ===== Using GDB ===== ===== Using GDB =====
Line 248: Line 268:
 Here is the article that presents better the situation: Here is the article that presents better the situation:
  
-   http://blog.logicmonitor.com/2014/10/09/more-linux-memory-free-memory-that-is-not-free-nor-cache/+   https://www.logicmonitor.com/blog/more-linux-memory-free-memory-that-is-not-free-nor-cache/
  
 An relevant excerpt from the blog article: An relevant excerpt from the blog article:
tutorials/troubleshooting/memory.txt ยท Last modified: 2021/06/01 20:44 by giavac