Cloud Performance Observations

[danyboyj]

... and it's dead again


EDIT:
... and it's back :S

[ejsolutions]

For others who may be interested/monitoring progress:

[The crew at eUKHost have been working tirelessly to make the eNlight platform into a system that many of us have grown to expect from them. This is new bleeding edge technology, so teething is unfortunately a necessary evil. For me, however, unless it develops to allow a smaller footprint, then I'll need to look elsewhere for a small Cloud.]

Latest observations:
For Centos 5.7 cPanel/WHM on the current eNlight VM template.
The key performance issue is swap - anything else is just dross.
768Mb is absolute bottom limit - 1Gb is fine but you MUST contain memory usage by pairing back Apache settings and tuning of mySQL.
AFTER you have done this, at a shell prompt

echo "3" > /proc/sys/vm/drop_caches
swapoff -av
swapon -av

You may need to temporarily stop some processes, before being able to do a swapoff, to free off memory. Clamd, mySQL and Apache are the most obvious candidates.

If you use Directmin, some other control panel, or none at all, then you will very likely get better response from a low end installation.

At this point, I'd like to see some development on reducing the swapiness (yes, it is a *nix word), thus reducing pressure on swap. Unfortunately my time spent on eNlight has left me short of time & energy to get on with client-side work.

Hope the above helps some of you.

EJ

[eUK-Rishi]

Thanks EJ for keeping with us and giving your precious time.

The new template for CPanel is available and we can see the results. It is improved a lot for smaller sites.

[email protected] [~]# free -m
             total       used       free     shared    buffers     cached
Mem:           524        289        234          0          2         80
-/+ buffers/cache:        207        316
Swap:          511          0        511

Its running small site CPanel now under 300M including cache and 0 swap.
There are few improvements at the infra side as well to serve them better.

Will keep posted for more.

[ejsolutions]

The new template for CPanel is available and we can see the results. It is improved a lot for smaller sites.

Yahoo! Gimme, gimme

[ejsolutions]

...The new template for CPanel is available...

Raised a ticket yesterday, to try to get this actioned on my existing VM but has caused more issues. Snr. support didn't know what I was talking about (fair enough). The new ticket system won't allow me to add updates, so couldn't even refer them to this thread.

If a new VM is now created using the dropdown selection of cPanel, does it now use the above mentioned template?

[eUK-Rishi]

Raised a ticket yesterday,

Give me the ticket ID and I'll patch in some one.

If a new VM is now created using the dropdown selection of cPanel, does it now use the above mentioned template?

Yes, It uses the new template having 512M Swap.

[ejsolutions]

Give me the ticket ID and I'll patch in some one.

SUPPORT #JNI-146-31145

Yes, It uses the new template having 512M Swap.

Might as well use up the rest of my 'bonus' credit Though I suspect a tweak to swapiness is a distinct possibility.

[eUK-Rishi]

You will have to create a VM and then update about the migrations in ticket. Sorry.
Or you may ask them to create a VM for you in your account and migrate the data.

Don't worry about the credit, we'll refill it for migration period

[ejsolutions]

Thanks - much appreciated.
I've closed the ticket and will handle the migration myself: I'm pretty seasoned at cpmoves. Though I may have a requirement to transfer the cPanel license to the new IP - time will tell.


Normal service will resume shortly...

[THOMAS-UK]

The swappiness of your VM is not actually calculated by Enlight. It's a kernel feature since 2003. You can change it on the fly by using:

sysctl -w vm.swappiness=60

Where 60 is the default swappiness and the higher it is; the quicker the OS will start to use its swap. Lowering it should enforce less swap to be used.

To make the change permanent, add it to /etc/sysctl.conf.

vm.swappiness = 60

[ejsolutions]

Thanks, Thomas.
That saved me looking it up.

[ejsolutions]

On a newly created VM, with the 'fresh' template....

• Apache recompiled with sensible settings
• CSF installed, along with the other useful "freebies" - numerous services disabled
• mySQL adjust for a low footprint
• clamd added for Windoze clients
• Bind chosen for its' flexibility (as usual)
• Three domains migrated over, though 'inactive' currently

             total       used       free     shared    buffers     cached
Mem:        524288     468744      55544          0        448      18388
-/+ buffers/cache:     449908      74380
Swap:       524280     322884     201396
Total:     1048568     791628     256940

Responsiveness: a slug chasing a blackbird - not the other way around.

For comparison:
Swappiness changed to 40 on original eNlight unfortunately appears to make little difference.

             total       used       free     shared    buffers     cached
Mem:        524288     457512      66776          0        364      20280
-/+ buffers/cache:     436868      87420
Swap:       524280     313308     210972
Total:     1048568     770820     277748

Both VMs burst to a Load of over 40 at the slightest thing eg. opening WHM.

As a reminder, here's the similar VM that has been running for a few months, with Load consistently low (<1).

             total       used       free     shared    buffers     cached
Mem:        524288     520960       3328          0        908      78228
-/+ buffers/cache:     441824      82464
Swap:       522104      57752     464352
Total:     1046392     578712     467680

And the services running on it (reformatted to compress it a bit)...

clamd
cpanellogd   cpdavd 	cpsrvd   
exim 	exim-4.69-30_cpanel_maildir
ftpd
httpd 	2.2.21 (Unix) 
imap 	ipaliases   lfd 
mysql 	5.1.56 
named 	9.3.6-P1-RedHat-9.3.6-16.P1.el5_7.1	
queueprocd 	
spamd 	3.3.1
sshd	syslogd

[eUK-Rishi]

Hello EJ,

We have been working on this particular IO wait issue and have made significant changes on the storage for everyones benefit. Less swap & optimized storage have reduced the IO problem so far. Can you check in the "top" which may indicate the reason of sluggishness ?

I checked in your VM graphs and it shows that load was max 2 in past couple of days after our changes. Hows its health now for you?

The server which you gave actually have less %user than the enlight one. I suspect its due to less processing load but there can be other factors too.

[danyboyj]

I've had Pingdom checking my site for the last week - for the most part its been ok - until just recently

See attached screenshot:

[ejsolutions]

Whilst there has been improvements, I'm reserving judgement until I bring the RAM allocation back down to 512Mb, from 768Mb (matching the existing Cloud #2 VM).

As I type this, I'm unable to monitor the status of eNlight, as I am getting no response from the server whatsoever - not good. I'll try a forced restart from the eUKHost client area.
(It's not my poxy broadband dropping issues, as other WHM and sites are responding fine.)
Except I can't:
"Error : Under Maintenance."