Troubleshooting High CPU load issue

  • Filter
  • Time
  • Show
Clear All
new posts

    Troubleshooting High CPU load issue

    CPU load : It is measure of CPU usage. It depend on number of processes
    running and waiting for CPU. When its average value is less than 1 then it says to be normal.

    There can be many causes, including DoS attacks, spam attacks, poorly designed php scripts which consume large amounts of memory, web spiders that crawl sites too aggressively, hardware issues, massive amounts of disk writes to a user's MySQL database, and much, much more.

    Note : Normal load value depend on number of CPU cores available on system.
    # cat /proc/cpuinfo | grep processor | wc -l
    Above command gives number of CPU cores and then divide the current CPU load [use top command to get current CPU load] value by number of CPU-core. If the result is above 1, then system is say to be under the load.

    System is under load when there are excessive use of memory, disk I/O and CPU.
    To investigate the issue you need to check current and historical statistics for these parameters.

    A ] CPU usage

    Current : you can check current CPU usage using top command
    # top -c

    Historical :
    Check %ideal column values
    # sar -p

    sar program is used to track CPU load. This program is available in sysstat package.

    1. Check today's CPU load average statistics
    # sar -q | less
    It shows load summary for each 10 minutes interval.

    2. Check load for specific date :
    # cd /var/log/sa
    # sar -q -f sa01 | less
    It shows load statistics for date 01 of the month. This program keep history of load average
    for last ten days.

    3. Look load average statistics for specific time period
    # sar -q -s 22:00:00 -e 00:00:00 -f /var/log/sa/sa10
    input the start[s] and end[e] time in 24 hr format.

    B] Memory: Check swap and physical memory. If system used whole physical memory then this is not
    big issue. When both swap and physical memory get 100 % used then need to reboot the server
    to get access.

    Current :
    # free -m

    # top -c
    and press M it gives process which consume more memory.

    Historical :
    Physical memory usage
    #sar -r

    swap memory usage
    # sar -S
    Note : Check "%memused" and "%swpused":
    C ] Disk I/O :

    This gives statistics for last 10 entries with 1 min intervals. Check %util column

    # iostat -x 1 10

    Historical :
    # sar -d

    Then proceed for finding out the processes which cause CPU load

    1> Check running process
    # ps -auxf
    take pid of process which consume more CPU and find out files related to this process as.
    # cd /proc/pid

    2> Check for mysql processes
    # mysqladmin processlist

    3> run dmesg to find out hardware issue.

    4> Check number of connections using netstat command.

    5> tail the various log files to investigate the active issue.

    Here are some logs to check:

    syslogs: /var/log/messages, /var/log/secure

    SMTP logs: /var/log/exim_mainlog, /var/log/exim_rejectlog, /var/log/exim_paniclog

    POP3/IMAP logs: /var/log/maillog

    Apache logs: /usr/local/apache/logs/access_log, /usr/local/apache/logs/error_log, /usr/local/apache/logs/suexec_log, /usr/local/apache/logs/suphp_log

    Website logs: /usr/local/apache/domlogs/ (use this to find sites with traffic in the last 60 seconds: find -maxdepth 1 -type f -mmin -1 | egrep -v 'offset|_log$')

    cron logs: /var/log/cron

    BEST LUCK................