Ask HN: do you track your time?

Every job I've had has been working for clients, which shapes the fact that I used to have to do time sheets as part of billing and I so probably don't hate them as much as I "should". During the time running my own company, however, I have never had to do time sheets. When we were tiny, it wasn't how we billed clients and these days, I don't do a large proportion of billable work.

Nonetheless, I track my time. All of it. I use toggl (http://www.toggl.com) and I find that any time I slip away from tracking all my time, my productivity slips. I spend too much time in email and not enough getting things done.

I'm therefore intrigued that others hate them so much. Is it just the fact that bad management uses them to beat people down, or is it the actual tracking itself?