The Artifex: hack of 1/15/04

Ben and Jack are the folks best equipped to explain the intrusion we had two weeks ago. However I'll explain what I understand as best I can.

On Monday January 15th, we started having a large amount of traffic going through our mail server, or at least trying to do so. By the following day, terra.artifex.org was doing a number of other things that left us and our colocation facility no choice but to shut down the machine. As best as we can tell, the intrusion occurred through one of a number of pieces of unupdated software -- the most notable ones being a recent update to the Linux kernel, and an update to a more secure e-mail server, or Mail Transfer Agent.

Ben was very quick to set to work on the server. Unfortunately for us, our colocation facility had failed to get him onto the Access Control List -- we lost several days because Ben couldn't get into the building. Finally last Monday, Ben made it in, took the server home, and reinstalled it. We updated to a more secure version of the kernel, a safer mail transfer agent, and also closed down as many insecure or unencrypted protocols that we could. Since we back up our data nightly, no user data was lost. However, some mail messages did bounce during the time that the main server was down. Although we have several secondary servers, they all sit on internet connections that don't allow incoming e-mail traffic.

Needless to say security is not a one-time sort of issue; it's a way of life and should be cultured as such. The past two weeks we've spent cleaning up the server to be configured as best as it can be. Our task for the coming years is to keep it that way. For more information on what we should all be trying to do about this in the future, see the note on future security plans.

Thanks everybody for hanging on the past two weeks. If you have any questions at all, don't hesitate to get a hold of me, Ben, or Jack.

one monkey don't stop the show,
Hunter