If you have only a single computer, then it's possible for you to spend your days giving it careful manual scrutiny for mischiefs and problems. Perhaps not entirely desirable, but possible. But in the real world we need good tools to monitor and warn us of mischiefs, so we can actually go outside and have a life every so often. Intrusion detection is one of those gnarly jobs that can make you paranoid and nervous — it seems the more you study it, the more difficult, scary, and unreliable it appears. But it's really not that bad, and Linux admins have a number of powerful tools to choose from. The best tactic is a layered approach that combines the oldies but goodies, like Snort and iptables, add some newfangled tools like psad and AppArmor or SELinux, throw in some nice analysis tools, and you're darn near state-of-the-art.
The oldtime notion of intrusion detection was to be alerted when an intruder successfully gained root access. But in these modern times, and actually in olden times too, any user account on the machine could be used for mischief. I think it's a long-standing and chronic weakness in Linux and Unix security to place all the importance on protecting root, as though user accounts were unimportant. Think about it — a simple reinstall replaces compromised system files, but what about data files? That's where the real gold is, and any intrusion has the potential to inflict a lot of damage. You don't need root access to spew spam, copy sensitive files, serve up bogus music and movies, launch attacks on other systems, and so forth.
psad (port scan attack detector) is a great new-ish tool that works in conjunction with iptables and Snort to give you a complete picture of all the nasties trying to ooze into your network. This is my first choice for a Linux IDS. psad uses a number of Snort tools, and combining it with fwsnortand iptables logs means you can even dig into the application layer and perform a bit of content analysis. It performs Nmap-style analysis of packet headers, sends you alerts, and can even be configured to automatically block suspicious IP addresses.
Its forensic abilities are valuable; in fact a key aspect of any intrusion-detection system is capturing and analyzing mass quantities of data. If you don't do this you're flying blindly, and not able to tune your IDS efficiently. You can export psad data to AfterGlow and Gnuplot to get excellent visual representations of what's hitting your firewall. Nothing beats seeing full-color representations of mass worms assaulting your network.
There is an excellent book that I think all serious IDS admins should have, and that is "Linux Firewalls: Attack Detection and Response" by Michael Rash. This is a clear, thorough guide with a lot of nearly copy-and-paste examples, and good understandable explanations of how things work and why they are done a certain way.
Take away the omnipotence of root, and you de-fang a significant percentage of exploits. Take it a step further and fence in all users and processes so that they have only enough permissions to do their jobs and no more, and you have a good tight system that will vex and frustrate even a successful intruder. The key is replacing the Unix system of Discretionary Access Controls (DAC) with Mandatory Access Control (MAC.) We've covered SELinux before. AppArmor and GRSecurity apply the same principles, but are easier to use than SELinux. Whichever one you choose, they all provide potent protection.
Good Old Snort
Snort is a reliable old standby that gets better with age. It's lightweight, reliable, and easy to use. You can run it standalone, or together with psad and iptables. You should be able to install it from your Linux distribution's package repositories, which is a great improvement over the old days of source installations. Keeping a set of current rules should be equally simple, because oinkmaster, Snort's rule updater and manager, is also in most distro repos.
Snort is easy to administer, though it does require a bit of care and feeding. To start with, the default configuration is not suitable for most shops because it includes rules for everything under the sun. So your first job is to get rid of everything that's not pertinent, because leaving all that lard in will hurt performance, and will generate false alarms.
Another important tactic is to run Snort in stealth mode, which means it listens to a network interface that does not have an IP address. On Linux bring the interface up without assigning it an IP address, for example ifconfig eth0 up, and then run Snort with the -i option, like snort -i eth0. It may be that if Network Manager is on your system, it will "helpfully" bring up unconfigured interfaces anyway, so it's probably safer to remove Network Manager.
Snort can collect an overwhelming of data, so add BASE (Basic Analysis and Security Engine) to the brew to get a good visual analysis tool. It is based on the older ACID (Analysis Console for Intrusion Databases).
Quick and Easy
You're probably familiar with the old rootkit detectors, Chkrootkit and Rootkit Hunter. Obviously these are more trustworthy when run from a non-writable external device, such as a CD or write-protected USB device. I like SD cards because they have the little write-protect switches- I adore physical on/off switches. These search for known rootkits, backdoors and local exploits, and have limited abilities to sniff out suspicious activities. You need to use these on running systems because they look at /proc, ps, and other significant activities on live filesystems. These aren't for network protection, but for fast scans on individual computers.
I know some folks who think Tripwire is a great application and they swear by it. Me, I swear at it. The enterprise versions are well-maintained and have all the bells and whistles a hardworking network admin could want. The open source version is a neglected orphan with a sparse feature set and sparser documentation. I must have been smarter in my younger days, because back then I didn't think Tripwire was a horridly overcomplicated, headache-inducing beast. But now I do. I mention it only because if I don't I'll get email asking why I didn't.