A hidden jewel -- or pain in the rear, depending on your perspective -- is Linux PAM (Pluggable Authentication Module). Linux oldtimers remember the ancient days when PAM was simple and used but a single configuration file. It didn't do much, and life was easy. The modern PAM is more complex and flexible, which can be trying for new sysadmins. But it has a number of significant advantages.
A Quick History Lesson
Back in the Linux stone age, passwords were encrypted by the venerable crypt and the resulting hash was kept in /etc/passwd. /etc/passwd has to be world-readable, so anyone who could glom a copy of it could then crack the passwords at leisure. So shadow passwords were invented; the hashed password is kept in /etc/shadow, which only root can read.
But life was still nasty and brutish. Applications that required an authentication mechanism had to be modified to support both old-style and shadow passwords, or provide their own authentication mechanisms, which was very inconvenient and meant that users had to recompile their existing programs to update them. So PAM was invented. PAM separates authentication from applications that use it. Supporting a new application means building it to be PAM-aware and writing a PAM module for it, which is a whole lot simpler. Sysadmins benefit too -- access can be finely tuned, and user authentication information can be stored anywhere, like in LDAP databases, rather than being limited to Linux system accounts.
The pain comes from PAM flexibility. If your needs are simple it can be daunting to figure out what to do, especially since different distributions configure PAM so differently. But once you understand how to configure PAM, you'll be able to easily understand it on any Linux distribution.
First take a look in your /etc/pam.d/directory. It contains a number of plain-text configuration files named for the services they control:
$ ls /etc/pam.d
chfn common-session gdm2 passwd su
chsh cron kcheckpass passwd.dpkg-old
common-account cupsys kscreensaver ppp
xscreensaver common-auth login samba
common-password gdm-autologin other ssh
PAM comes with a number of modules, which you'll find in /lib/security/. These are described, along with options and examples, in Chapter 6 of the The Linux-PAM System Administrators Guide. This may also be installed locally at /usr/share/doc/libpam-doc/. Much of the mystery of PAM is removed by this document.
There is also a body of third-party modules scattered here and there all over the Internet. A good starting point is here.
Most applications install their own PAM modules already configured, so chances are you won't have to spend much time messing with PAM. But if an application does not do this, or you want to customize the settings, or you are having trouble with remote or cross-platform logins, understanding PAM will save your blood pressure.
Remember that changes to anything in /etc/pam.d/are instant, so you can test them right away. Be careful! Make backups of everything and keep a bootable rescue disk handy. If you mess up you may not be able to log in.
Removing Login Delay
Our first job is getting rid of the silly delay that occurs after a failed login. I mean come on already -- what's the point? Ignore the FAIL_DELAY directive in /etc/login.defs because it has no effect. pam_unix.so is hard-coded to a 3-second delay. The docs say 1 second, but I timed it. Get rid of it with the nodelay option in the /etc/pam.d/common-authfile on Debian:
auth required pam_unix.so nodelay
On Fedora and Red Hat add nodelay to the corresponding line in /etc/pam.d/system-auth:
auth required /lib/security/$ISA/pam_unix.so nodelay
The $ISA token is a PAM builtin that automatically looks for modules of the correct architecture, such as 32-bit or 64-bit.
Locking Out Users Who Fail Too Much
It is generally a good and sensible practice to lock out a user after a number of failed login attempts, with some exceptions. You don't want to give an intruder unlimited attempts, but you don't want clumsy users pestering you all the time for login resets. And users who wish to be ornery can lock out other users by trying to suto a different user's account, and failing on purpose.
On Debian, add this line to /etc/pam.d/common-auth:
auth required pam_tally.so onerr=fail no_magic_root
And this line to /etc/pam.d/common-account:
account required pam_tally.so onerr=fail deny=3 reset no_magic_root
On Red Hat, add the above two lines to /etc/pam.d/system-auth. This gives users three chances to log in, then locks them out if they fail. The no_magic_rootoption is very important -- this prevents the root user from being locked out. In this era of great bootable rescue disks like Knoppix, that's nowhere near the catastrophe it used to be.
When the offending user has sufficiently soothed your upset sensibilities, restore access this way:
# pam_tally --user doofusfred --reset=0
user doofusfred (1006) had 29
It even tattles on how many times the user tried to login.