Hello,
My proposal is to add a configurable setting with a double i/o streaming effect:
Increase input buffer size each time current is emptied because it was too small.
Smoothly slow served fps (same for audio) when input buffer is being consumed, to buffer exhaustion is not seen by spectator as a pause, and spectator feels only slight changes in real playing time.