#1 Mail Server problem: Storage is Cheap, Indexing is Not
Category Administration
This weekend I was at a customer site and was surprised to find a new high score for large, individual, mail files: 16 G. There is just no justification for a 16 G mail file, and, of course, the administrator agrees but is without any authority to enforce a reasonable policy.
Large email files are not a technical problem, but a management breakdown. Email storage, on any system, is not an cheap substitute for a document management solution.
So, what are the alternatives? How do we get back to letting the mail files handle mail, rather than become huge document repositories? Here’s a short list of my recommendations.
-
·
- Document and chart the progress of the problem.
In an ideal setting, a correlation needs to be documented between storage resources and server performance that can be used to establish proof for management. We need to demonstrate that server performance is lagging because of bloated storage requirements. The simplest way is to be running your statistics collector, enable performance trapping, and create historical charts from your Statistics and Reporting database. If you don’t know how to do this, wait a little bit and I’ll post a write-up on it.
What management needs to see is that storage works like tanker cars on a railroad. A moderate engine can pull a very large load, and it seems easy to just add more tankers as needed. After all, most of the terrain is flat. This is reasoning that justifies large SAN outlays, without touching the necessities of server processing. What happens when the train needs to manage a steep incline? Everything slows down. Worse, by far, is what happens when the train must descend rapidly.
In measuring server performance regarding storage we have two perspectives to keep in balance to prevent catastrophes, free space and indexing power. The server needs a percentage of free space, which is recommended at 30 percent. I’ve seen mail servers running less than 5 percent and the management is puzzled why the systems are failing—after all, 5 G free space out of a 100 G sub-system seems like a lot for someone used to desktop systems. Unfortunately, that 5 G vanishes under heavy indexing and then the stability of the server is questionable.
The mail store is a database file, which provides indexing to quickly identify the appropriate records. Indexing requires RAM and CPU processing (there are additionally I/O needs, as well, but RAM and CPU are the most sensitive). Increasing the storage capability without attending to RAM and CPU requirements is the second technique for building unstable mail servers. Insufficient RAM will allow indexes to corrupt and insufficient CPU just backlogs all the indexing so that the systems become increasing slower.
Failure of a mail server is a natural consequence when available storage is low and resources are minimal. Then, the administrator is just waiting for the "perfect storm" of a torrential mail load. In addition, mail administrators are increasingly reporting that unusually large mail files present a disproportionate load on mail servers. Documentation is the first step to establishing a responsible criteria for upgrading essential mail systems. - Shared mail (Single Copy Object Store or SCOS) can help to minimize storage requirements, and ND6 has continued to improve on its capabilities. Of course, if the shared mail file becomes corrupt, then instead of having one or two irate users, the entire set of users on that mail server are without functioning mail (hopefully, the mail servers are clustered).
- Document management integration is the real answer to these large mail files. The client does not want to lose the data, but has no other database in which to store the documents.
- Quotas would be terrific, if possible. Sometimes, after management reads the reports and understands the expense of an unstable infrastructure, they’ll accept quotas.
- Archiving allows the data to be moved to (a) a local file on the desktop (or mapped to a file server), (b) a sub-directory on the same mail server, (c) transferred to a second Domino server. The advantage of archiving is that it can be automatically scheduled, controlled by the administrator, and the client user only has to hit the archive icon in their mail file to be immediately redirected to their old mail. By moving old mail out of the active mail file, indexing resources are dramatically relaxed.
- Centralize large mail files on a server dedicated to these users. In this way, the large files are, in effect, quarantined. This can be an effective détente, where the client can still persist with perpetuating an oversize mail file, but the general population in the organization is not punished with slow services.
- 
Comments
Where can I find real, technical documentation as to the official (or eve n best practice) recommendation and why it is at least 15-20% ?? I am one of those Domino admins trying to explain to management that an email server is not a file server; and they continue to ask "what specifically uses all that space and why?"
Of course I've tried to open tickets with Lotus but all to no avail. We're finally going to start looking at an archive solution but in the meantime we're adding disk.
Is there anything you have that could help?
Posted by eric At 04:40:58 AM On 02/14/2006 | - Website - |
Posted by Nathan T. Freeman At 12:56:50 AM On 02/25/2004 | - Website - |