Hi,
Remember you are using HTTP as transport protocol. Your behaviour is common when you don't have sufficient bandwith, or when you have several lost packets and retransmissions, or when the server is busy, or when a congestion appears in the network.
I suggest to change to another transport protocol, like RTSP/UDP (unreliable) or HLS (reliable).