This isn't a silly question at all, the reason is that a download doesn't come to your computer in one huge chunk, it's divided up into lots of vert small chunks called "packets" each of which has to be marked in some way so it gets sent to your ip, the right port & the right program, this is done with a sort of address label called a "packet header", if an isp chooses to, they can install software that inspects these packet headers & either lets them by unhindered, throttles your bandwith, or drops them into complete oblivion.
Most isp's don't go to this extent as simply blocking a given port number is much easier, but unfortnately a few are doing this sort of thing.
