Based on the information you sent, if you haven't already try looking at your network hardware. Anything from errors on the interfaces to the speed on the network switch port.
In a job many moons ago we had a similar problem with a different system. After about 50 users or so the system would slow to a halt and be unusable, but all pieces of hardware that was involved was essentially idle. After much finger pointing and such with tech support it turned out that the ethernet interface on the server was auto negotiating it's speed to 10 instead of 100. We had a lot of identical hardware but only that server decided it was going to run at a lower speed.
If that fails look for anything that could be doing some kind of rate limiting.
Damion
----- Original Message -----
From: "Dmitry S. Makovey" <dmitry@athabascau.ca>
To: "Steve Hillman" <hillman@sfu.ca>
Cc: "zimbra-hied-admins" <zimbra-hied-admins@sfu.ca>
Sent: Tuesday, February 17, 2009 6:28:47 PM GMT -05:00 US/Canada Eastern
Subject: Re: High-volume server settings
On February 17, 2009, Steve Hillman wrote:
> We have 4 mailbox servers with about 12,000 users on each so far. It wasn't
> until we did our student migration over Christmas (which took us from
> 10,000 total users to 42,000) that we had to tweak our thread counts. The
> HTTP threads weren't even really obvious (and I'm still not 100% sure we
> were running out), but it *seemed* to make a difference to web snappyness
> when I doubled them to 500.
this is good to hear. Because we started to get worried about thread counts
with only 300 active users that need to be trippled, which doesn't seem to be
a scalable approach. What you're saying makes sense to me.
> If your servers are sluggish or unresponsive, the problem is probably one
> of two things - memory or disk. More memory == less disk reads (doesn't
> help writes much). We run with 32gb on each mailbox server and give MySQL
> 14gb. Right now, that's theoretically enough for the entire mysql database
> to fit in RAM
>
> We have a shelf (14 drives) of fibrechannel disks dedicated to each mailbox
> server for db and index storage. The 'store' space runs fine off of SATA
> drives
thanks to your posts and other information from this ML we're doing exactly
the same :) It doesn't look though that we're hitting hardware issue. Not at
first sight anyway (environments are pretty much idle from io and os load
perspective).
--
Dmitry Makovey
Web Systems Administrator
Athabasca University
(780) 675-6245