Ranter
Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Comments
-
@linuxxx I am working in a hosting company's. My task is what I explained above. It is very helpful when You guide me through this
-
@manikandanm Do you mean load as in CPU load or as in loading the OS at boot time? (Ex-hosting employee)
-
@linuxxx Server load like Which service is taking the resources...
How to resolve the issues...
Like Mysql, SMTP, apache etc.., -
@manikandanm If its CPU load, just look at what software is causing this (through top or htop for example) and investigate what that software is doing.
What often happens when a server is low on resources is that it goes OOM (Out Of Memory) which kills the biggest process and then the load can skyrocket (I've seen 20 core servers go towards a 1200 load due to this).
But it really depends on the case. Maybe PHP is busy or crashed which can have many reasons, same for Apache, nginx, mysql and so on, this is impossible to say without knowing more details. -
@manikandanm Had it once where the permissions of a huge site weren't correct and for some reason that made that I think both apache and php were statting the entire (huge ass) application at every request (sometimes hundreds a second). Took our most senior engineer more than an hour to figure out and about nothing pointed towards that at first 😄
-
@manikandanm server load is too ambiguous. Narrow it down please.
Load average?
Cpu usage?
Network load?
San writes/reads?
Mem%?
Processes' recycles?
Limits' exhaustion?
Saying it's 'server load' is like saying 'events in the country'. Be more speciffic -
C0D4681465yInstall htop
#yum install htop
And see what's the obvious issue.
Otherwise be more precise in what you mean by load. -
@manikandanm mailq and spamq are not linux metrics. These are application metrics. Though there is mailq that could give you a hint what's happening and what's flooding.
Cpu% - that's easy. Just run top or htop and sort by cpu%. You might be tempted to use ps aux, but there are pots on this path - ps aggregates processes' cpu% through all the process lifetime [to be fair it's linux that aggregates it.. Ps only shows you that aggregate]. Top/htop on the other hand are polling for this metric every second or so and diffs it between polls. Hence you see how aggressive the processes were between two last polls.
I use top for live view and either sar -u 1 3 or vmstat 1 3 for scripts
Cpu queue [load avg] - if you were dealing with unix - it would most likely correlate with cpu%. Linux excludes cpu% from load avg and from my xp 99.9% of the cases cpu q is usually filled with slow i/o. See sar -q 1 3 for current queue length. To troubleshoot i/o issues first monitor top for a while -- see the wa% [io wait] piece. Normally it should not be >3, most of the time it should be 0. If it's high - see iotop to find devices that are lagging and see ps aux output for any processes in D state. Iowait almost always means your storage is lagging. Hba fiber issues? Suboptimal cloud storage config? Throttled io? That's for you to figure out.
Swap - first learn more about linux mem mgmt. Processes are never ever able to allocate swap. Only linux kernel can do that and it only does that when mem% is too high. Look for hi mem usage. Currently or in the past. It's likely some process used 90% of ram 2 hours ago, linux swapped out idle processes and left them in there. When mem% droped down to normal linux DOES NOT free swap by itself, unless idle processes ask for memory blocks that were swapped out.
Look for HI mem%
Ex-Linux and unix sysadmin, oriented to performance issues -
Extra tip: ALWAYS discard the first line of results in vmstat and sar outputs. The same reason why ps is tricky with cpu%
-
@manikandanm bear in mind that swap is storage-based. It's unlikely for servers [more likely for cobtainers] but possible that processes go wild in cpu%, consume mem%, kernel will start swapping memory out to storage that is in another aws AZ and is throttled. Effectively your load avg will sky-rocket.
That's an unlikely scenario for bare servers but is likely for docket hosts. That's why swap should be disabled for containers
Related Rants
What are types of load issues occurs in a Linux (Cent os) server and it is troubleshooting methods???
question
centos server
server