Vanderbilt University Government Statistics Website 
(NOTE:  This is not a government website)
Arranged by topic, this is a very easy to use, comprehensive site for finding government statistics. While not a government website, it includes links to many official government sources.

- - - - - - - - - - - - - - - - - - - - - - - - -

Other Popular Sites:

American Factfinder Quick statistical facts on the U.S.

Bureau of Labor Statistics Useful economic data CPI, PPI, unemployment rate, etc. see "Economy at a Glance" on this site

FEDSTATS  A gateway to statistics from over 100 federal agencies

Statistical Almanac of the United States  Summary statistics on just about any topic (often for a range of years) - lacks detail to lower levels of geography (ie. counties, etc.) but is a good pointer to other sources.

STAT-USA STAT-USA/Internet, a service of the U.S. Department of Commerce, is a single point of access to authoritative business, trade, and economic information from across the Federal Government.

 U.S. Census Bureau Population Statistics and demographic information, etc. from the U.S. Census Bureau (includes "American Factfinder")