Update 10/2/2012: The Update Notifier plugin hasn't been updated since 9/20/2010. I've switched to WP Updates Notifier, a fork of Update Notifier that's being actively maintained.
The Update Notifier plugin sends email notifications for updates to WordPress, and, optionally, updates to plugins and themes (the components that would normally notify you on the Updates page of the admin panel).
For the sake of security and functionality, I try to install updates as soon as possible. This plugin helps me stay on top of them.
Emails are sent to the admin's primary address, with the subject "Updates are available for your WordPress site". The email body varies depending on the updates available:
It's time to update the version of WordPress running at http://yoursite.com from version 3.0.4 to 3.0.5.
The following plugin(s) have new versions available for download:
To perform the necessary updates visit your admin panel.