IPTV - buffering, stalls, general drop outs. Settings that resolved things for me.

J Smith · 2017-11-01T12:28:29+00:00

Figured I'd share this, as I struggled for a long time with IPTV feeds - they would frequently stall, drop out, replay previous 10 seconds or so, etc. Whe...

Gerrit Gogel

I was fiddling around with my IPTV setup for some days now. I found a lot of helpful information in this thread. Especially the FFmpeg parameters provided by Troy Boy helped a lot.

Here are the full commands I'm using:

Adapt the paths according to your setup.

Download your m3u file:

wget -O /tvheadend/config/iptv.m3u http://path.to.your.m3u

If the file has Windows line breaks (CRLF), convert it to Linux line breaks (LF):

dos2unix /tvheadend/config/iptv.m3u

Insert the pipe command to every line starting with `http://`:

sed -i 's#^http://.*#pipe:///usr/bin/ffmpeg -reconnect 1 -reconnect_on_network_error 1 -reconnect_on_http_error 1 -reconnect_streamed 1 -reconnect_delay_max 4294 -fflags -nobuffer -err_detect ignore_err -i & -c copy -map 0 -f mpegts -tune zerolatency pipe:1#' /tvheadend/config/iptv.m3u

The FFmpeg command is based on the suggestion made by Troy Boy.

I added `-map 0`, such that every audio and subtitle stream is piped, if multiple are available.

I added `-reconnect_on_network_error 1 -reconnect_on_http_error 1 `, which force FFmpeg to reconnect when a TCP or HTTP error is received. For instance, my provider sometimes gives HTTP 404 errors.

Further, I increased `reconnect_delay_max` to the max value of 4.294 seconds. It determines how long FFmpeg tries to open the stream when an error occurs. Sadly, in some scenarios, even this can be too low. My IPTV provider seems to start streams on demand, meaning if nobody is watching them the stream is "sleeping". This often takes 10-20s. In this case, I have to start the stream a second time. But even with FFmpeg, this problem cannot be solved at the moment, due to the relatively low limit of `reconnect_delay_max`.

In any case, these parameters gave me the best experience. I use Kodi, Emby, and Jellyfin as clients.

EDIT:
I removed the following parameters because they made the audio become async after a while. I guess because the video FPS isn't determined correctly. I didn't notice any significant change in the stream start-up time.
`-probesize 1000k -analyzeduration 0 -fpsprobesize 0`

Bob Birch

Hey Gerrit,

Thanks for the ideas.

I created the new entry for one of my channels using your sed command. I edited the Mux and saved it.

When I try to play it from the TVHeadend web interface, it opes VLC, but fails.

Attached are the logs for the playing of the channel:


2022-09-04 12:09:17.628 [   INFO]:http: ::ffff:192.168.1.139: using ticket 5e14011a45fc78d9398dbe5cca9da1564d35a4dd for /stream/mux/f4c39330ad210fa9942e2592e6d23926
2022-09-04 12:09:17.631 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - add raw service
2022-09-04 12:09:17.634 [  DEBUG]:service: 3: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends si 0x562752560ef0 <unknown> weight 0 prio 11 error 0 (OK)
2022-09-04 12:09:17.634 [  DEBUG]:service: 2: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends si 0x562752517900 <unknown> weight 0 prio 11 error 0 (OK)
2022-09-04 12:09:17.634 [  DEBUG]:service: 1: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends si 0x5627528bb190 <unknown> weight 0 prio 11 error 0 (OK)
2022-09-04 12:09:17.635 [   INFO]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - tuning on IPTV #1
2022-09-04 12:09:17.863 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - open PID 0000 (0) [20/0x5627592d2b70]
2022-09-04 12:09:17.863 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - open PID 0001 (1) [16/0x562759d91230]
2022-09-04 12:09:17.863 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - open PID 0010 (16) [20/0x5627542f49a0]
2022-09-04 12:09:17.864 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - open PID 0011 (17) [20/0x5627581ace40]
2022-09-04 12:09:17.864 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - open PID 0011 (17) [16/0x562756a4e5a0]
2022-09-04 12:09:17.864 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - started
2022-09-04 12:09:17.864 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - open PID fullmux subscription [0003/0x5627536b8c90]
2022-09-04 12:09:17.864 [   INFO]:subscription: 0204: "HTTP" subscribing to mux "jq2zy9wz89c556rdyxr5.m3u - UK-Alibi", weight: 10, adapter: "IPTV #1", network: "TVLegends", service: "Raw PID Subscription", hostname="::ffff:192.168.1.139", username="admin", client="VLC/3.0.17.3 LibVLC/3.0.17.3"
2022-09-04 12:09:17.865 [   INFO]:spawn: Executing "/usr/bin/ffmpeg"
2022-09-04 12:09:18.936 [  ERROR]:spawn: ffmpeg version 4.4.2 Copyright (c) 2000-2021 the FFmpeg developers
2022-09-04 12:09:18.936 [  ERROR]:spawn:   built with gcc 11 (GCC)
2022-09-04 12:09:18.936 [  ERROR]:spawn:   configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom -
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libavutil      56. 70.100 / 56. 70.100
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libavcodec     58.134.100 / 58.134.100
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libavformat    58. 76.100 / 58. 76.100
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libavdevice    58. 13.100 / 58. 13.100
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libavfilter     7.110.100 /  7.110.100
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libavresample   4.  0.  0 /  4.  0.  0
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libswscale      5.  9.100 /  5.  9.100
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libswresample   3.  9.100 /  3.  9.100
2022-09-04 12:09:18.948 [  ERROR]:spawn:   libpostproc    55.  9.100 / 55.  9.100
2022-09-04 12:09:19.863 [  DEBUG]:service: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends: Status changed to [CA check] 
2022-09-04 12:09:24.170 [  ERROR]:spawn: Input #0, mpegts, from 'http://XXXXXX.XXX:8080/XXXXX/XXXXXX/59030':
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Duration: N/A, start: 1350.000000, bitrate: N/A
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Program 1 
2022-09-04 12:09:24.170 [  ERROR]:spawn:     Metadata:
2022-09-04 12:09:24.170 [  ERROR]:spawn:       service_name    : Service01
2022-09-04 12:09:24.170 [  ERROR]:spawn:       service_provider: FFmpeg
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 130 kb/s
2022-09-04 12:09:24.170 [  ERROR]:spawn: Output #0, mpegts, to 'pipe:1':
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Metadata:
2022-09-04 12:09:24.170 [  ERROR]:spawn:     encoder         : Lavf58.76.100
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 25 tbr, 90k tbn, 90k tbc
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 130 kb/s
2022-09-04 12:09:24.170 [  ERROR]:spawn: Stream mapping:
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Stream #0:0 -> #0:0 (copy)
2022-09-04 12:09:24.170 [  ERROR]:spawn:   Stream #0:1 -> #0:1 (copy)
2022-09-04 12:09:24.170 [  ERROR]:spawn: Press [q] to stop, [?] for help
2022-09-04 12:09:24.170 [  ERROR]:spawn: frame=    1 fps=0.0 q=-1.0 size=       0kB time=00:00:00.00 bitrate=   0.0kbits/s speed=  11x    
2022-09-04 12:09:24.170 [  DEBUG]:service: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends: Status changed to [Demuxed packets] [CA check] 
2022-09-04 12:09:24.170 [  DEBUG]:service: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends: Status changed to [Demuxed packets] [Reassembled packets] [CA check] 
2022-09-04 12:09:24.170 [  DEBUG]:webui: Start streaming /stream/mux/f4c39330ad210fa9942e2592e6d23926?ticket=5e14011a45fc78d9398dbe5cca9da1564d35a4dd
2022-09-04 12:09:24.170 [WARNING]:webui: Stop streaming /stream/mux/f4c39330ad210fa9942e2592e6d23926?ticket=5e14011a45fc78d9398dbe5cca9da1564d35a4dd, timeout waiting for packets
2022-09-04 12:09:24.171 [   INFO]:subscription: 0204: "HTTP" unsubscribing, hostname="::ffff:192.168.1.139", username="admin", client="VLC/3.0.17.3 LibVLC/3.0.17.3"
2022-09-04 12:09:24.171 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - close PID fullmux subscription [0003/0x5627536b8c90]
2022-09-04 12:09:24.171 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - stopping mux
2022-09-04 12:09:24.171 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - close PID 0000 (0) [20/0x5627592d2b70]
2022-09-04 12:09:24.171 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - close PID 0001 (1) [16/0x562759d91230]
2022-09-04 12:09:24.171 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - close PID 0010 (16) [20/0x5627542f49a0]
2022-09-04 12:09:24.171 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - close PID 0011 (17) [16/0x562756a4e5a0]
2022-09-04 12:09:24.171 [  DEBUG]:mpegts: jq2zy9wz89c556rdyxr5.m3u - UK-Alibi in TVLegends - close PID 0011 (17) [20/0x5627581ace40]

Not ure why it would't play. I made sure the new url in TVHeadend is all on one line when I copied it in. Is there a way to see the actual URL TVHeadend is using to open the mux?

Thanks.

Gerrit Gogel

Hey Bob,

Maybe try to open the channel a second time. I think many IPTV providers put channels to "sleep" if they aren't watched. The time to start them up exceeds the maximum timeout in FFmpeg. As you can see in your log the time from starting FFmpeg to closing is almost equal to `-reconnect_delay_max 4294`. Sadly, at the time of writing this parameter cannot be set higher. Also when I stream channels directly with VLC, I experience the same. The first attempt of opening some random channel almost every time fails.

By the way, the value 4294 comes from dividing the maximum value of an unsigned 32bit integer by 1000000. I don't know why FFmpeg devs decided to set this as a max value. Usually, the standard for timeouts is 30 s. 4294 ms just isn't enough.

Gerrit Gogel

I just tested it again and the value is actually seconds, as documented, and not milliseconds. But still for instance in this thread people also report that milliseconds are used instead of seconds: https://superuser.com/questions/1050481/ffmpeg-keep-connection-if-network-source-down

Maybe there is an issue related to specific builds of FFmpeg, where ms are used instead of seconds. No idea.

Gerrit Gogel

Did you maybe set the Data timeout parameter in Stream Profiles to 5s? I wouldn't recommend setting this to a low value. I set mine at 30 s. It means that the stream is closed if the data rate stays at 0 Bytes/s for x seconds. I saw values of 5-10s suggested here, which are too low. I noticed that some streams send data only about every 5-10s, so they would be terminated with this setting.

Bob Birch

I hadn't changed the Data Timeout. It is still at 0. But if I started the strem a second time....it worked!!!

How will that work when TVHeadend goes to record? Is it a VLC issue?

Thanks for the help. I'll see how it goes with doing some recordings.

Bob

Gerrit Gogel

I checked it again and indeed it seems to be a client issue. For instance, I start a channel with VLC, then VLC stops playing the channel if it doesn't receive data for a certain amount of time, which causes tvheadend to close the channel as well. I haven't checked recordings yet.

Greg H

Gerrit Gogel wrote:
> I was fiddling around with my IPTV setup for some days now. I found a lot of helpful information in this thread. Especially the FFmpeg parameters provided by Troy Boy helped a lot.
>
> Here are the full commands I'm using:
>
> Adapt the paths according to your setup.
>
> Download your m3u file:
>
> [...]
>
> If the file has Windows line breaks (CRLF), convert it to Linux line breaks (LF):
> [...]
>
>
>
> Insert the pipe command to every line starting with `http://`: > [...]
>
>
> The FFmpeg command is based on the suggestion made by Troy Boy.
>
> I added `-map 0`, such that every audio and subtitle stream is piped, if multiple are available.
>
> I added `-reconnect_on_network_error 1 -reconnect_on_http_error 1 `, which force FFmpeg to reconnect when a TCP or HTTP error is received. For instance, my provider sometimes gives HTTP 404 errors.
>
> Further, I increased `reconnect_delay_max` to the max value of 4.294 seconds. It determines how long FFmpeg tries to open the stream when an error occurs. Sadly, in some scenarios, even this can be too low. My IPTV provider seems to start streams on demand, meaning if nobody is watching them the stream is "sleeping". This often takes 10-20s. In this case, I have to start the stream a second time. But even with FFmpeg, this problem cannot be solved at the moment, due to the relatively low limit of `reconnect_delay_max`.
>
> In any case, these parameters gave me the best experience. I use Kodi, Emby, and Jellyfin as clients.
>
> EDIT:
> I removed the following parameters because they made the audio become async after a while. I guess because the video FPS isn't determined correctly. I didn't notice any significant change in the stream start-up time.

> `-probesize 1000k -analyzeduration 0 -fpsprobesize 0`

Just tried this with my HDHR and it is working, however the zapping time is really slow. Around 10-15 seconds. Any ideas on the cause?

tvheadend enthusiast

So I limited the output of ffmpeg to loglevel fatal (it was just too much log output as default) but I am also experiencing the slow channel switch time as mentioned above by Greg. Anyone have any ideas on how to fix this? I have .ts piped into the script from m3u4u and then run that through the script.

I want to add that before I piped the stream through ffmpeg I was unable to get recordings to actually keep recording when buffering or drop out occurred during playback. After using the ffmpeg pipe though I now have recordings that actually record correctly. Before using ffmpeg pipe channel switching was pretty quick but after I am experiencing that slow switching time as mentioned by Greg.

For context, my whole setup overall is running on a raspberry pi 4 with 4GB ram, inside a docker container with tvheadend (ran with podman as the other containers are) and then that goes through my vpn container using container network mode for internet access. I then forward the ports for tvheadend through the vpn container (because when using container network mode you can't specify port mappings as well). I am not sure if maybe the pi is just not powerful enough to handle transcoding with ffmpeg quickly or maybe it's the options I have set idk, but would be nice to speed up channel loading/switching times.

Additionally I saw people mentioning something with syslog (I think?) here or so https://tvheadend.org/d/5282-iptv-buffering-stalls-general-drop-outs-settings-that-resolved-things-for-me?r=35521#message-35521 and I am wondering if maybe implementing that would resolve my issues with the slow channel switching?

Sorry for the essay but any feedback is appreciated

Auser Resua

Hi all, this thread has been a great source of things to try and encouragement that possibly I can make IPTV stable enough to use with TVHeadend. The issue I have with many providers is when the stream goes bad, my recordings become unusable. The last few days I have made a bit of a breakthrough and it seems pretty stable. I thought I would share my work with you all to pay it forward and see if it can be improved. One thing I'm not happy with is it doesn't necessarily clean up the processes particularly when a mux scan happens. Anyways here's what I did:

/usr/local/bin/ffmpegin.sh

#!/bin/bash while [ 1 ]; do /opt/ffmpeg-5.1.1-amd64-static/ffmpeg -nostats -reconnect 1 -reconnect_on_network_error 1 -reconnect_on_http_error 1 -reconnect_streamed 1 -reconnect_delay_max 30 -fflags nobuffer -err_detect ignore_err -i $1 -c copy -map 0 -f mpegts pipe:1 1> /home/hts/dump.$2.stream done

/usr/local/bin/ffmpegout.sh

#!/bin/bash /opt/ffmpeg-5.1.1-amd64-static/ffmpeg -nostats -follow 1 -err_detect ignore_err -i /home/hts/dump.$1.stream -c:a aac -c:v mpeg2video -b:v 12M -map 0 -f mpegts pipe:1

/usr/local/bin/linker.sh

#!/bin/bash pid=$$ echo "main pid is $pid" | logger -t ffmpeg-linker trap 'echo "Caught trap, ffmpegin.sh pid is $PID" | logger -t ffmpeg-linker && rm -f /home/hts/dump.$pid.stream && trap - SIGTERM && kill -- -$$ | logger -t ffmpeg-linker' EXIT SIGINT SIGTERM rm -f /home/hts/dump.$pid.stream mkfifo /home/hts/dump.$pid.stream /usr/local/bin/ffmpegin.sh $1 $pid & PID=$! echo "ffmpegin.sh pid is $PID" | logger -t ffmpeg-linker /usr/local/bin/ffmpegout.sh $pid rm -f /home/hts/dump.$pid.stream

And of course, you need some kind of filter for your m3u file to add the pipe command to the front of each line: pipe:///usr/local/bin/linker.sh

This approach needs a bit of grunt in your machine too, I've got an old 2.5Ghz Xeon and each recording uses about 2 cores.

jon will

Auser Resua wrote:
> Hi all, this thread has been a great source of things to try and encouragement that possibly I can make IPTV stable enough to use with TVHeadend. The issue I have with many providers is when the stream goes bad, my recordings become unusable. The last few days I have made a bit of a breakthrough and it seems pretty stable. I thought I would share my work with you all to pay it forward and see if it can be improved. One thing I'm not happy with is it doesn't necessarily clean up the processes particularly when a mux scan happens. Anyways here's what I did:
>
> /usr/local/bin/ffmpegin.sh
> [...]
>
> /usr/local/bin/ffmpegout.sh
> [...]
>
> /usr/local/bin/linker.sh
> [...]
>
>
> And of course, you need some kind of filter for your m3u file to add the pipe command to the front of each line: pipe:///usr/local/bin/linker.sh
>

> This approach needs a bit of grunt in your machine too, I've got an old 2.5Ghz Xeon and each recording uses about 2 cores.

Can someone give me a step by step guide on how to implement this? I'm OK with linux but new to TVheadend.

julien julien

> This approach needs a bit of grunt in your machine too, I've got an old 2.5Ghz Xeon and each recording uses about 2 cores.

Thanks a lot i try for the moment all is ok

Nothing to do just create file in good folder and add

pipe:///usr/local/bin/linker.sh http://linkofvideo in your m3u

for me just change
/opt/ffmpeg-5.1.1-amd64-static/ffmpeg
to

ffmpeg

and I add
-user_agent Kodi/18

in /usr/local/bin/ffmpegin.sh after ffmpeg because my iptv provider block ffmpeg user agent

and change /usr/local/bin/ffmpegout.sh for have good quality

ffmpeg -nostats -follow 1 -err_detect ignore_err -i /home/hts/dump.$1.stream -c copy -map 0 -f mpegts pipe:1

thanks

« Previous Page