XMLTV limited output/conversion

Saen Acro

At home I use SSIPTV on my SmartTV but there is a problem with EPG

EPG is huge and with unused contents

ex.


<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE tv SYSTEM "xmltv.dtd">
<tv generator-info-name="TVHeadend-4.3-1783~gec90d31" source-info-name="tvh-TVServer">
<channel id="7bcc00006a1daf203ac172a27b33014d">
  <display-name>RTL</display-name>
  <display-name>22</display-name>
  <icon src="http://tv.local:8000/l/RTL.png"/>
</channel>
<programme start="20190408170000 +0300" stop="20190408180000 +0300" channel="7bcc00006a1daf203ac172a27b33014d">
  <title lang="bul">Meine Geschichte-Mein Leben</title>
  <desc lang="bul">Dokusoap, D 2018.</desc>
</programme>
<programme start="20190408180000 +0300" stop="20190408183000 +0300" channel="7bcc00006a1daf203ac172a27b33014d">
  <title lang="bul">Freundinnen-Jetzt erst recht</title>
  <desc lang="bul">Dailysoap, D 2019 (Staffel: 1, Folge: 155).</desc>
</programme>
<programme start="20190408183000 +0300" stop="20190408190000 +0300" channel="7bcc00006a1daf203ac172a27b33014d">
  <title lang="bul">Unter uns</title>
  <desc lang="bul">Dailysoap, D 2018.</desc>
</programme>
</tv>

idea is to be converted to shorter format,

just to includeTITLE only,without +desc, sub-title, category+ etc.


<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE tv SYSTEM "xmltv.dtd">
<programme start="20190408170000 +0300" stop="20190408180000 +0300" channel="RTL">
  <title lang="bul">Meine Geschichte-Mein Leben</title>
</programme>
<programme start="20190408180000 +0300" stop="20190408183000 +0300" channel="RTL">
  <title lang="bul">Freundinnen-Jetzt erst recht</title>
</programme>
<programme start="20190408183000 +0300" stop="20190408190000 +0300" channel="RTL">
  <title lang="bul">Unter uns</title>
</programme>
</tv>

Is anyone have idea how can be done that?

Em Smith

It seems your EPG needs to be <5MB and can't be compressed.

It looks easy enough to update the code to add a "&format=short" to the request processor to skip desc, etc., but would it be useful without episode numbering and description?

I'd probably want to keep channel tags since presumably your tv uses that to match with the playlist and display channel logos.

However, would that make your epg fit the tv limits? Trying it with my OTA, I'd still be around 6MB. (By doing grep -e programme -e '<title' -e display -e icon on my existing xml file as a rough test).

You can add a limit in Tvheadend for how many days epg it keeps in channel->channels, select all, edit, limit epg (days). Perhaps that might help? I don't know if it immediately reduces the EPG days or if it just applies to new updates.

The SSIPTV suggests it can get its own EPG and avoid using the Tvheadend EPG, http://ss-iptv.com/en/operators/epg ?

It might be easiest to set up some web server, run a crontab to grab the xml from Tvheadend, run the grep above and send the output to a file on the web server. Then you set your tv to get the xml from the web server instead of from Tvheadend.

So something like:


#! /bin/sh
OUTPUT=/var/www/.../channels.xml # or wherever webserver stores files
curl http://tv.local:9981/xmltv/channels' | egrep '<?xml|<!DOC|</?tv|</?channel|</?display|</?programme|</?title|</?desc' > $OUTPUT.tmp && mv $OUTPUT.tmp $OUTPUT > /dev/null 2>&1

Saen Acro

SSIPTV show only TITLE rest of attributes is not used

http://i.imgur.com/wqSEmns.jpg

also how can be removed this translation of name insidechannel and this section also
in example


7bcc00006a1daf203ac172a27b33014d >>> RTL

even and on other software's importing TVH epg not show "human readable" name just HASH value


-rw-r--r-- 1 root   root   5803447 Apr 15 09:15 channels.xml <<< with removed lines
-rw-r--r-- 1 root   root   5771768 Apr 15 09:22 channels1.xml <<< as previous without channels section
-rwxr-xr-x 1 root   root       277 Apr 15 09:15 cronepg.sh*
-rw-r--r-- 1 root   root   8044208 Apr 15 09:17 xmltv.xml  <<< original size 
-rw-r--r-- 1 root   root   8000065 Apr 15 09:26 xmltvshort.xml  <<< when used &format=short /wged used/

when add&format=short to bash script result is empty file ??


#! /bin/sh
OUTPUT=/abyssws/htdocs/channels.xml # where web server stores files
curl 'http://localhost:9981/xmltv/channels&format=short' | egrep '<?xml|<!DOC|</?tv|</?channel|</?display|</?programme|</?title|</?desc' > $OUTPUT.tmp && mv $OUTPUT.tmp $OUTPUT > /dev/null 2>&1

p.s.

SSIPTV offered EPG service do not support my preferred language

p.p.s
What should be 24 or 72h call?

Em Smith

The "&format=short" doesn't exist yet in the code. I meant we can add it to the server if it is of benefit. (In xmltv.c line 159 we'd effectively add "goto done;" before the "if" statement, and add "done:" before line 212 which writes the "</programme>", you can add those two lines yourself if you want to force xmltv to not have those fields since I won't have time to do the proper fix this week).

The SSIPTV is a bit confusing, because this topic says it does not use "tvg-id" to link a playlist to the xmltv.

http://forum.ss-iptv.com/viewtopic.php?f=3&t=1951&p=17257&hilit=xmltv#p17257

So I think it is saying it ignores tvg-id from the .m3u file, reads the channel name "Boomerang", and uses that for the "programme" tags in the xmltv file. But, the programme tags in the xmltv fileare tvg-id tags not channel name tags.


.m3u:
#EXTINF:-1 tvg-id="a14232...",CNN
.xmltv:
<programme start="20190414144500 +0100" stop="20190414165500 +0100" channel="a14232...">

So, I don't understand how it links the two together.

SSIPTVshould use the display-name from the channels section to convert the hash to the human readable format, since that is how most of the xmltv files are that I have seen.

Examples I've seen are:


  <programme start="20120901081500 -0600" stop="20120901093000 -0600" channel="I10162.labs.zap2it.com">

 <programme start="20181212022500 +0100" stop="20181212024500 +0100" channel="rtlplus.de">

where
  <channel id="rtlplus.de">
    <display-name>RTL Plus</display-name>
    <display-name>PLUS</display-name>
  </channel>

The reason it is a hash is because it is unique, whereas the forum suggests you can have two channels from different sources with the same name but different contents like "Discovery" being "Discovery (Europe)" or "Discovery (America)" received in the same house. (And I have about ten channels all called "ITV", but are all have different region shows).

Have you tried manually changing the file to see if it would process the name correctly? So if "abcdef1234567890" is your Boomerang channel, add a "| sed 's/abcdef1234567890/Boomerang/g'" to your processing line after the grep.


curl 'http://localhost:9981/xmltv/channels' | egrep '<?xml|<!DOC|</?tv|</?channel|</?display|</?programme|</?title|</?desc' | sed 's/abcdef1234567890/Boomerang/g' > $OUTPUT.tmp && mv $OUTPUT.tmp $OUTPUT > /dev/null 2>&1

Saen Acro

Most of other software's use human readable names aka not use hashed name translation.
Xtreamcodes for example import epg correctly but then matching hashed name manually needed
same with Stalker Portal/Ministra
most of them accept Epg generated by https://github.com/hiroshiyui/epgrab without problem /with correct chanidents or by channel SiD/

Saen Acro

By the way, can anyone modify php script with was used before :9981/xmltv was invented,
https://tvheadend.org/issues/1908 to get time limited output not event.

Em Smith

You can try the attached patch.

This adds an option to the advanced section of the "user" dialog. You can then choose a format for xmltv and for htsp to eliminate the description and other long text fields.

I am not convinced about the "no hash" for the reasons I have previously mentioned. However, I have included it as an option.

I am not 100% sure that user is the best place to add these options, but it avoids having to do complicated setup on each client.

The xmltv still includes channels since that has logo, etc., which I think is useful for the majority of other clients.

Saen Acro

Will test tomorrow and report.

Hash is longer then channel name, SID is better solution

at point of view of client is very useless.
Name > hash ... generate index of hash to name ... in event hash > name
What a waste if CPU cycles usage

Saen Acro

WORK OK
LCN/ Service set number, is not needed also in short EPG.
Continue testing

Saen Acro

user#14856 have you seen this type of EPG

Em Smith

No, I haven't seen that format. They look like something an egg snoop/dump would produce.

Saen Acro

both files are formatted for human visual.

EIT.json is from MuMuDVB /project older then TVH/ near any IPTV player support it.

there is all information send by operator EPG generator.

Second one is most compact EPG I ever seen, same in XML is x7 bigger.
Еvents is attached to playlist by service numbers, most compatible for SmartTV's and STB's,
used in channel list, extra info content is called via info button on remote via second call.
<<

Em Smith

If you're happy with the changes then I can raise a bug and submit a pull request.

The reason I kept LCN is because it may be useful to other users. For example, in some countries people expect channels in a certain order, so that would allow a device to reproduce that order (even though your TV doesn't use that feature). Hopefully it doesn't use much file space.

I agree xmltv is very verbose. It came from a time when people were forcing everything to be xml or Java. I could tell some stories about such projects, but no-one would believe them!

Saen Acro

(/) Directly go to github for PR ;)

Json output is also good direction

Other, in feature options
TIME and/orFORMAT

:9981/xmltv/24/xmltv/channels :9981/xmltv/72/json/channels :9981/xmltv/120/jtv/channels # why not ;)

----

This is forum post about integration in SSIPTV app for smart tv's
http://forum.ss-iptv.com/viewtopic.php?f=3&t=6929

Em Smith

The xmltv supports xmltv/channelname/RTL

Or xmltv/channelid/abcdef

It might be possible to add the


#EXTM3U x-tvg-url="EPG_url"

But, it seems some others having url-tvg instead. I've not researched which is supported most.

Is there any advantage to having the tag in the channel list? Does it make it easier to setup?

Saen Acro

With tag sorting on some players is very cool and easy to sort

grouping as sport, movies, news, music, boring etc.

#EXTINF:-1 tvg-id="BTSport1.uk" tvg-name="UK: BT Sport 1 HD" tvg-logo="http://tv.logo/logos/btsport1hd.png" group-title="UK",UK: BT Sport 1 HD http://some.online.source/DUpdCXUJ14/1294.ts

group-title supported by KODI and it's equal tochannel tags in TVH

-----

Response from SSIPTV team


Questions 
1. how program name is compared from m3u and xml?
2. Is streaming format supported aka do I can take it not from stored file but from epg generator?

Answers 
1. Use "tvg-id" or "id" attribute in M3U and attribute "id" of tag "channel" on the xmltv side. IDs must be unique.
2. If you mean that you want to generate it "on the fly" - ok, it doesn't matter for the app, but you should keep in mind the required headers for http.

Saen Acro

EPG working but timing is some problematic time offset

https://i.postimg.cc/MKh1MwTj/image.png
On image real time 19:55 /same on TV OSD clock/
in red 23:00 need to be 20:00
this is on all channels.

Not sure from where problem come

p.s.
there is some strings with is not cutted


  <episode-num system="xmltv-ns"> . 1 .   </episode-num></programme>

Em Smith

I've double-checked the change and can't see any reason for Tvheadend to send bad timestamps, especially with one being incorrect for ten minutes rather than XX hours. So, I'd suspect it would be your guide data. Perhaps check the Tvheadend EPG UI.

I left the episode-num in on purpose since other clients might be able to use it, for example to lookup extra details/descriptions for a programme you are playing, even though no current TVs use it.

Saen Acro

I also not see problem in TVH side
I updated post with lot of pictures, in SS-IPTV forum and wait for answer.
My guessing is: client side not respecting timezone.

Em Smith

As you say in the other forum, the "same playlist and epg on ProgDVB and IP-TV Player all times is correct." So, it seems to be something with the SSIPTV since the xmltv you pasted in the other forum has the same date/time+TZ format as your first post.

However, it seems strange that the SSIPTV author (in Russia) doesn't have a similar problem with tv time being wrong or epg-time being wrong. But, perhaps he is using the json format with time_t (seconds) instead of a time string+TZ.