Project

General

Profile

[Solved - Tutorial linked] 30 minutes offset on German live TV via internet/IPTV

Added by H. Fux almost 4 years ago

Dear all,

I set up TVheadend (HTS Tvheadend 4.1-2427~g566c12f-dirty to be precise) on a Raspberry. I installed a number of IPTV automatic networks using m3u8 playlists, like this one (ARD das Erste):

http://daserste_live-lh.akamaihd.net/i/[email protected]/master.m3u8

Which basically works really fine - I get stable downstream, good quality, happy with that. One thing however I cannot get rid of and would be really happy to get help with: When I start watching such a stream, I travel back in time by 30 minutes, i.e. I see the then not-so-live TV from 30 minutes ago.

When I download the above m3u8, the content is a list of links to other m3u8's, which tvheadend then offers me as MUXes. When I download such a "MUX" m3u8, e.g. http://live-lh.daserste.de/i/[email protected]/index_2692_av-b.m3u8?sd=10&rebase=on, it contains 180 transport streams like http://live-lh.daserste.de/i/[email protected]/segment148711148_2692_av-b.ts?sd=10&rebase=on - each segment is 10 seconds long. This - surprise, surprise - adds up to 30 minutes.

My theory is, that when I start one of the mux/services, tvheadend works down the playlist top to bottom, always being 30 minutes behind the most recent segment. So, in principle, I need tvheadend to skip, say, 178 or so entries. Is there something to achieve this?

By the way, although the playlist only contains 30 minutes worth of playing material, the stream nicely continues beyond these 30 minutes - tvheadend is obviously well aware how to load the next playlist items after the old list is worked down.

Thanks for any hint what I miss!

Hauke


Replies (5)

RE: 30 minutes offset on German live TV via internet/IPTV - Added by H. Fux almost 4 years ago

Some additional info: I tried to figure out why there are 30 minutes in the m3u8 - actually that's because the TV stations offer 30 minutes time-shift in their Web-based viewer - so user may hit "Pause" and may do so for up to 30 minutes.

Reading further, it seems that the "rotating playlists" are exactly how HTS works...

My next step will be to bring a proxy inbetween and establish a rewrite rule that strips all but a few lines from m3u8-files - let's see if I am clever enough to accomplish this... If I succeed, will post the solution here.

How to configure tvheadend to use proxy? - Added by H. Fux almost 4 years ago

OK, I already hit a wall at the very first step: How do I configure tvheadend to actually use a proxy? Searched the internet for an hour and only found tutorials to proxy tvheadend, but not vice versa..

Anyone can give me a hint? Thanks!

*Solved* - Added by H. Fux almost 4 years ago

OK, solved the problem. In short, I created two PHP pages that modify the involved playlists as desired and use the PHP as tvheadend URLs for the network.

A little bit more detailed:

I have:
  • A m3u8 playlist that I use as network, which contains the services the TV station offers. I'll call it the network playlist below.
  • A m3u8 playlist that contains a sequence of transport streams, in my case 180 of them, which I will call the MUX playlist below.

The network playlist URL is like this:

http://daserste_live-lh.akamaihd.net/i/[email protected]/master.m3u8

and its content is (shortened):

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=184000,RESOLUTION=320x180,CODECS="avc1.66.30, mp4a.40.2"
http://daserste_live-lh.akamaihd.net/i/[email protected]/index_184_av-p.m3u8?sd=10&rebase=on
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=184000,RESOLUTION=320x180,CODECS="avc1.66.30, mp4a.40.2"
http://daserste_live-lh.akamaihd.net/i/[email protected]/index_184_av-b.m3u8?sd=10&rebase=on
[...]

My first PHP script, m3u8redirect.php, now changes each line of this playlist like this:

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=184000,RESOLUTION=320x180,CODECS="avc1.66.30, mp4a.40.2"
http://127.0.0.1:8000/m3u8shorten.php?Keep=5&TargetURL=http://daserste_live-lh.akamaihd.net/i/[email protected]/index_184_av-p.m3u8?sd=10&rebase=on
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=184000,RESOLUTION=320x180,CODECS="avc1.66.30, mp4a.40.2"
http://127.0.0.1:8000/m3u8shorten.php?Keep=5&TargetURL=http://daserste_live-lh.akamaihd.net/i/[email protected]/index_184_av-b.m3u8?sd=10&rebase=on
[...]

So, basically the part http://127.0.0.1:8000/m3u8shorten.php?Keep=5&TargetURL= was added in front of each URL, which causes TVheadend, when it collects the MUXes to enter these new URLs into the MUX URL, and when it checks the services, the MUX playlists will each be run through my second php script, m3u8shorten.php.

This second script then strips the unwanted sequences from the MUX playlist, et voilá, Live-TV is suddenly live. The original MUX playlist looks like this (again - shortened):

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-ALLOW-CACHE:YES
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:148753965
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148753965_184_av-b.ts?sd=10&rebase=on
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148753966_184_av-b.ts?sd=10&rebase=on
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148753967_184_av-b.ts?sd=10&rebase=on
[...]
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148754141_184_av-b.ts?sd=10&rebase=on
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148754142_184_av-b.ts?sd=10&rebase=on

And after m3u8shorten.php (this time complete file):

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXT-X-ALLOW-CACHE:YES
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:148754138
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148754138_184_av-b.ts?sd=10&rebase=on
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148754139_184_av-b.ts?sd=10&rebase=on
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148754140_184_av-b.ts?sd=10&rebase=on
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148754141_184_av-b.ts?sd=10&rebase=on
#EXTINF:10.000,
http://daserste_live-lh.akamaihd.net/i/[email protected]/segment148754142_184_av-b.ts?sd=10&rebase=on

Notice also the updated line #EXT-X-MEDIA-SEQUENCE:148754138.

And here in detail everything to get it up and running

My description for XBian 20170203-0. All steps done as root/sudo:

First, install webserver and php:

apt-get install lighttpd php5-cgi

Next: Configure lighttp - edit /etc/lighttpd/lighttpd.conf:

server.port = 8000

This is to keep port 80 open for web interfaces. Now add the following lines to /etc/lighttpd/lighttpd.conf:

fastcgi.server += ( ".php" => ((
"bin-path" => "/usr/bin/php-cgi",
"socket" => "/var/run/php.sock"
)))

Activates PHP. Now put the following two PHP scripts into /var/www/html:

m3u8redirect.php:

<?php

        /*

        This script adds to each URL in a m3u8-playlist a redirect-URL to the PHP-script that
        shortens the transport stream playlist to a given number of sequences.
        Hand over two GET variables:

        Keep            Number of sequences to keep in the rolling playlists
        TargetURL       URL of the MUX playlist that contains the rolling playlists for the services

        Written by Hauke Feb 2017
        */

        // Set correct content type
        header('Content-Type: application/x-mpegURL');

        // Get GET-Variables
        $SequencesToKeep = $_GET['Keep'];
        $M3u8URL = $_GET['TargetURL'];

        // Download original M3U8
        $M3u8 = file_get_contents($M3u8URL);

        // Add redirect-URL to each URL in playlist and send it out
        echo str_replace("http", "http://127.0.0.1:8000/m3u8shorten.php?Keep=" . $SequencesToKeep . "&TargetURL=http", $M3u8);

?>

And m3u8shorten.php:

<?php

        /*

        This script takes a m3u8 playlist that contains a sequence of transport streams. It
        drops all sequence URLs but the last ones - how many can be configured via GET variable.
        It adjusts the starting sequence number accordingly.

        Input GET variables:
        Keep            Number of sequences to keep
        TargetURL       URL of the original playlist

        Written by Hauke Feb 2017

        */

        // Set correct content type
        header('Content-Type: application/x-mpegURL');

        // Read GET variables
        $M3u8URL = $_GET['TargetURL'];
        $SequencesToKeep = $_GET['Keep'];

        // Download original playlist
        $M3u8 = file_get_contents($M3u8URL);

        // Split playlist into individual lines
        $M3u8Lines = explode("\n", $M3u8);

        // Arrays that will contain the individual transport stream URLs and the m3u8 EXTINF information
        $EXTINFs = [];
        $URLs = [];

        // Count sequences processed
        $Sequences = 0;

        // Loop through all lines
        foreach($M3u8Lines as $line) {
                if (strtoupper(substr($line, 0, 8)) == "#EXTINF:") {
                        // EXTINF found --> Add to array
                        $EXTINFs[] = $line;
                } else if (strtoupper(substr($line, 0, 4)) == "HTTP") {
                        // URL found --> Add to array and count sequences +1
                        $URLs[] = $line;
                        $Sequences++;
                } else if (strtoupper(substr($line, 0, 22)) == "#EXT-X-MEDIA-SEQUENCE:") {
                        // Filter out current sequence start number - needs to be updated later
                        $MediaSequenceStart = substr($line, 22);
                } else if (strlen(trim($line)) > 0) {
                        // Each other non-empty line is just echoed (assume it's a header line)
                        echo $line . "\n";
                }
        }

        // Adjust the starting sequence number and send it out
        echo "#EXT-X-MEDIA-SEQUENCE:" . ($MediaSequenceStart + $Sequences - $SequencesToKeep) . "\n";

        // now send out the desired number of sequence URLs from the end of the list
        for ($SequenceCounter = $Sequences - $SequencesToKeep; $SequenceCounter < $Sequences; $SequenceCounter++) {
                echo $EXTINFs[$SequenceCounter] . "\n";
                echo $URLs[$SequenceCounter] . "\n";
        }

?>

Thats all to install.

In order to configure tvheadend, if you create the network, the URL needs to be changed. Assume the original URL is:

http://daserste_live-lh.akamaihd.net/i/[email protected]/master.m3u8

You have to change this to:

http://127.0.0.1:8000/m3u8redirect.php?Keep=5&TargetURL=http://daserste_live-lh.akamaihd.net/i/[email protected]/master.m3u8

So you just add http://127.0.0.1:8000/m3u8redirect.php?Keep=5&TargetURL= to the original URL.

Everything else goes like usual, and you're done!

Adjust the value Keep=5 to your liking - the number says, how many sequences should in the end be in the MUX playlist. 5 I found working really well with the German TV stations.

/me is happy now :-)

Edit: Additional information regarding the HTS-protocol i found here: https://developer.apple.com/library/content/technotes/tn2288/_index.html

Tutorial available - Added by H. Fux over 3 years ago

Dear all,

since I struggled with several problems, I decided to publish a tutorial in my blog. Find it here:

http://projects.webvoss.de/2017/04/02/building-a-media-center-for-german-iptv/

Hope it's helpful.

0 minutes offset on German live TV via internet/IPTV - Added by Argus Nymus almost 3 years ago

Hi Hauke,
I just had the same problem with the NDR livestream, which is also provided by Akamai.

Found their HLS URL specs here:
https://learn.akamai.com/en-us/webhelp/media-services-live/media-services-live-stream-packaging-rtmp-ingest-hds-and-hls-outputs-user-guide/GUID-EB91F1A0-75EF-4604-9F57-61B66BB7B6C0.html

So the solution is simply to append a "dw" parameter with the desired playback in seconds to the m3u8 query string, e.g.:
http://ndrfs-lh.akamaihd.net/i/[email protected]/master.m3u8?dw=60

By the way, I am getting the HLS playlists from MediathekView(Web): https://mediathekviewweb.de/#query=live%20ndr

Cheers!

    (1-5/5)