Re: Large Zimbra deployments on VM

KSU has been using Zimbra since early 2008, mostly virtual. Some Background ...

Kennesaw State University
3300+ accounts, VMware 3.5 and 4, Red Hat Linux OS.
IBM 3950 physical hardware underneath.
IBM nSeries storage.
Web client, mostly. Some Mac, Outlook, Blackberry, iPhone, Thunderbird, etc.

Lessons learned over the last six months, some specific to VMs:

- Go 64-bit, do not compromise on any 32-bit platform, guest OS, etc. We started with 32-bits, and this has resulted in some significant growth pains.
- Read the performance tuning guide (even for non-VMs, this is a must-read): http://wiki.zimbra.com/index.php?title=Performance_Tuning_Guidelines_for_Large_Deployments. This is some thick reading, but worth the time.
- RAM sizing: Don't do less than 16GB on a mail store.
- Swapping is bad for Zimbra, the OS, I/O wait, performance ...... See the last item. There are at least two good stories regarding swap/Linux swappiness here. Just make sure you've got enough guest RAM so that swap does not begin to enter your mind.
- Five or six mail stores seems to be sufficient for our environment, including an evaluation mail store (21 accounts on actual hardware), ~800-900 accounts per VM. We may be able to extend this account count. We have not yet pushed the 64-bit platforms, but will soon.
- If you are expecting heavy non-web use, be careful to balance our IMAP, Mac and mobile accounts across the mail stores.
- Pay careful attention to the distance from the mail stores to the storage, physically and logically. I/O wait consistently over 1-2% is bad for performance. If you are swapping on top of, say, a long-distance iSCSI situation, then don't expect good performance. Backups are also still problematic from a performance perspective, an issue we are looking into currently.
- I've had several tickets open with Zimbra support regarding upgrading to Zimbra v6 from our 5.0.19NE, and we've not yet received a clear recommendation - especially (1) Since VMware just bought Zimbra, and (2) Zimbra v6.x apparently has specific issues within VMware. I would expect to see some specific tuning recommendations in the tuning wiki - specific to VMware - soon.

This is not so much in the way of questions to ask, but may prompt some from you. Since you are already running Zimbra, you may know many of the questions already.

I'd be happy to discuss further ...

A. Daniel King, IT System Support Professional V
Kennesaw State University
1000 Chastain Road, Mailstop 3503
Kennesaw, GA 30101
770-423-6902 (Office)

----- Original Message -----
From: "Tim Ross" <tross@calpoly.edu>
To: zimbra-hied-admins@sfu.ca
Sent: Wednesday, February 17, 2010 7:35:42 PM GMT -05:00 US/Canada Eastern
Subject: Large Zimbra deployments on VM

We are currently running ZCS 5.0.19 NE on Red Hat 5 on hardware. Our current servers are coming up on the end of their service contracts later in the year and our Central Server group is encouraging us to consider going to a VM setup instead. We are currently on the path to upgrade to 6.0.5 by summer, so that is where we would be when we would switch to VM if we decide to go that way. I have read several forum posts about VM setups working pretty well with ZCS, but I haven't seen anyone with more than around 1000 users talk about using VM. We have VMWare here on campus, so that would be our VM environment. We have around 30,000 accounts (around 22,000 student accounts, many of which are lightly or not used).

Are any of you running a large Production system in a VM environment? Any suggestions on questions to ask our Central Server guys about our current VM environment to determine if it can support our ZCS setup?

Thanks,

Tim Ross
Application Administrator
Collaboration Support Group