TechOpsGuys.com Diggin' technology every day

I read slashdot quite often, mostly for the comments, I post (as AC) maybe once every 3 years, but find the discussions interesting on occasion.

One such discussion was here, where someone was asking for advice as to how best to migrate off of gmail onto their own hosted platform. To me it seemed simple enough, but honestly I could not believe the negative response towards running your own mail server.

First off I’ll say I haven’t run a “corporate” mail server for almost a decade now, I have run several mail relays for companies for applications and stuff. I have been running my own mail server for my own personal (and some family) use for more than a decade, and I run another mail server that has maybe a dozen people on it, left overs from when I ran a small ISP in Washington.

So nothing major. I didn’t get the impression that the poster on slashdot was asking for anything major. But I was seeing people talk about massive headaches with blacklisting, anti spam, having to worry about disaster recovery, data replication, and the constant hand holding and patching of the system to keep it running.

I just didn’t get it. I mean sure it took some effort to set up the system I have which is pretty basic, it really requires minimal maintenance, I have never been blacklisted, really have minimal spam problems (very manageable anyways for me).

My setup is basic as I mentioned:

• Postfix for SMTP – I setup quite a bit of anti spam stuff many, many years ago but really haven’t touched it much at all since.
• SpamAssassin – for – duh – spam. I took some time to integrate this into incoming postfix email and it flags messages as **** SPAM ****  in the subject when something hits the spam filter, I have server side mail filters that move that to a dedicated folder. In all my years I have never noticed a false positive and have never had anyone complain that they can’t email me for a reason related to Spam Assassin flagging their email as Spam. My biggest potential issue with Spam Assasin is I probably get 150 spam (that get past the filters) for every real email I get (I don’t get a lot of email at home excluding mailing lists that I occasionally participate in). So I don’t have much “ham” to train SA with. I haven’t recently tried to determine how much spam is blocked at the various levels but last time I did (many years ago) it was quite a bit.
• Anomy Sanitizer – this does quite a few things such as stripping HTML email, stripping bad attachments etc. I’m sure it goes overboard in a lot of cases, and most users probably wouldn’t like it, stripping HTML email probably causes the most usability issues for me as some emails don’t come in with plain text as well as HTML, so some times I  get email that says “Hey click on this link to unsubscribe (or do some other action – e.g. rate Netflix quality back when I used their stuff)” only to find Sanitizer stripped the html so there is no link to click on, and no url I can copy/paste to the browser. But IMO at least it’s a small price to pay
• Cyrus IMAP 2.1 for IMAP – I started using Cyrus back in 2000 when I migrated a company off of UW IMAP onto Cyrus because it had some more advanced functionality vs Courier at the time (don’t remember what). I’ve stuck to it because it seems to work for me. I create different email addresses for pretty much every organization I deal with and have those go to dedicated IMAP folders (server side – not using filtering, postfix delivers directly to the mailbox), so most of the time I am unsubscribed to 85% of the IMAP “user accounts”, and only subscribe when I need to, email collects silently in the background in the meantime. User accounts is in quotes because I use a single account to access all of the other IMAP accounts (which can be problematic for some email clients because they make certain assumptions that don’t apply to me)
• Squirrelmail for webmail – The UI is basic, hasn’t really changed much since – you guessed it – about 2000 when I first started using it, I have, on occasion looked for alternatives but have not found one (until this slashdot discussion that is) that looked interesting enough to try. One big feature I like about squirrelmail is the ability to have many, many “From” email addresses, and select them from a drop down box. I have upwards of 100 different email accounts(view from the outside world – from my view it’s a single account), I maybe need to send mail “From” from maybe 10-20. So when I compose an email I select which email address to send “From”. Squirrelmail is smart enough if I hit reply on an email someone sent, say to my linkedin@ address, the mail client will automatically select the right “From” email address to use without me having to think about it. Since Sanitizer strips out html from emails I don’t believe I have to worry about XSS bugs in Squirrelmail because it’s all stripped out (but I could be wrong I’m not sure). As a result I haven’t upgraded in eons..

The last time I did major changes to my system was probably 5-6 years ago, those were introducing Spam Assassin to the system, and the more painful process of upgrading from Cyrus 1.x to 2.x (it was an ugly upgrade process).

I don’t use anti virus, never needed it(I integrated anti virus with Sendmail at the one company that I did manage the corporate email servers for back in 2000-2002). A lot of my habits and practices were set up a long time ago and there may very well be better ways to go about things these days(one thing would be to investigate using spamd for anti spam instead of spawning a separate SA process for each message), but what I have works, it doesn’t cause issues, I just don’t understand what some of those people were complaining about when running their own personal mail server.

I also don’t do any sort of calendaring system – never really needed it for personal use.

Sure it requires some setup, and you need to be smart (forward & reverse DNS, you want clean IPs that aren’t blacklisted (easy ways to check that are out there). I saw one guy say the person should get a block of 30-40 IPs and put the mail server in the middle of the block! I mean are you kidding me?

Even back when I ran an ISP with maybe 50-60 users (yes it was a small ISP – back in late 90s mostly), we never had blacklisting or spam problems. Maybe we were lucky I don’t know.

I just couldn’t believe the experiences some of these people were posting. Sure I can understand having those kinds of issues if your running a big mail system for a lot of users, but the impression I got was the original poster was looking to run a setup sort of like mine – a small number of users – or maybe just themselves.

I’ve even gone through the process of migrating my mail system from:

• Office with dedicated T1 lines to..
• Home with DSL connection to ..
• Temporary office with T1 lines while my DSL is moved to ..
• Home with DSL connection to ..
• Co-location to..
• Terremark Cloud (where it runs now) and this weekend to..
• Co-location

And all of the DNS changes and stuff associated with it, for the most part the migrations are pretty painless.

A couple of my users were wondering how I went about moving all of their data and stuff from Washington to Miami in a matter of minutes without them noticing anything, it takes some moderate planning but in the grand scheme of things it’s not that difficult from a technical perspective. Next weekend I’ll be migrating them again from Miami to California – their data resides on a different system from mine.

I just wanted to write about my experience – sure it does take some work – depending on how much functionality  you want  – to initially set  up the system. But in my experience once it is setup, the amount of work to maintain it is minimal.

I like the privacy and control I get with running my own stuff. I sure as hell don’t trust google with my data, they could pay me $100/mo for hosting my email with them and I wouldn’t do it. But for others it may be a good option.

I did use Gmail at the last company I worked at, their corp email was Gmail. I really didn’t like it – but what surprised me the most was how slow the search was. I was expecting anything I searched for would be returned in a fraction of a second but it took much longer (not forever but 10-20x longer than I was originally expecting). I never got used to how they organize their mail, with the tags and stuff. Even after using it for ~8 months I never warmed up to it, probably because there was mini demons in the back of my head screaming at me not to like it because I don’t trust Google, I’m sure that had something to do with it.

But for others maybe it’s the best way, or hotmail, or yahoo, or whatever.. I’ve been hosting my own email for so long I never really used anything else.

The most annoying problem I think I ever came across running my own mail system was not granting Spam Assassin enough memory – sometimes it would puke causing the email to get garbled – it would happen maybe once every 150,000 emails or something (which means maybe 2-3 times per year), I ignored it for a few years finally decided to look into it and found that SA was running out of memory so I gave it more(I think I went from something like granting it 32MB to 128MB), hasn’t happened since.

Do I have disaster recovery? No – don’t need it. If my server goes down for a few hours or a day or whatever, mail is likely queued at the other end (mail is often queued for up to what – 5 days before being dropped from the queue?). What happens if I lose an email? Really not the end of the world. I have had a few times when people say my system rejected their message – and sometimes it does, I wrote (again a long time ago) a lot of regular expression checks to try to detect spam, and sometimes it gets a false positive, so I fix it and move on, it’s rare though(again a few times a year at most). If the email is THAT CRITICAL then if they really can’t get through to me they’ll call. And if I don’t answer(or don’t return the call) and it’s THAT CRITICAL – they’ll call again later.

Of course my co-location/cloud stuff doesn’t run just email – it runs this blog, my basic web sites, DNS, and my new co-location serves as my off site backup with ~3.5TB of usable disk space on the system, I brought the system home last weekend and sync’d up 1.7TB of data to it.

Moral of the story is – if you really want to run your own mail system, don’t be afraid – it’s not THAT hard.