well there's a number of factors,
1, if your on DSL, your speed may rise and fall like a rollercoaster,
2, if the other party is on DSL, same effect,
3, either users can have problems, like some ISP's throttle at different times,
4, either users connecting to messenger or browsing the net at same time can effect flow,
5, if either user has more than 1 pc, connecting to internet/winmx/messenger on another one will affect flow,
6, they may have deliberately canceled your position in queue in favour of a friend.