Maybe I wasn't so wrong after all. This teardown at iFixIt shows (at step 8) solder pads that are a perfect fit for a dock connector. It's obvious that Apple has thought about a dock connector for Apple TV. Perhaps they were primarily thinking about the dock connector as a conduit for playing your music in your living room using your iPod, but when you have AirPlay you don't really need that. I hope Apple comes around to the idea of using the dock connector more as a way to extend an ecosystem for software and hardware developers and places a dock connector on the next iteration of Apple TV.
~sThe Phantom Dock Connector