Update 10/2/2012: The Update Notifier plugin hasn't been updated since 9/20/2010. I've switched to WP Updates Notifier, a fork of Update Notifier that's being actively maintained.
The Update Notifier plugin sends email notifications for updates to WordPress, and, optionally, updates to plugins and themes (the components that would normally notify you on the Updates page of the admin panel).
For the sake of security and functionality, I try to install updates as soon as possible. This plugin helps me stay on top of them.
Emails are sent to the admin's primary address, with the subject "Updates are available for your WordPress site". The email body varies depending on the updates available:
It's time to update the version of WordPress running at http://yoursite.com from version 3.0.4 to 3.0.5.
The following plugin(s) have new versions available for download:
akismet (http://downloads.wordpress.org/plugin/akismet.2.5.2.zip)To perform the necessary updates visit your admin panel.
Since WP Updates Notifier has now also fallen behind in terms of maintenance and updates, you could also try out WP Lookout, https://wplookout.com/. It's a new, free service (with paid options for power users) that consolidates notifications about new versions of themes, plugins and WordPress core, across all of the sites you manage. We'd be glad for any feedback you have if you try it out!
Thanks for letting us know, Chris! Happy New Year!