Project

General

Profile

Bug #4193

Descrambled services/muxes are regocnized as scrambled and can not be played.

Added by Paul Brunniem over 4 years ago. Updated over 4 years ago.

Status:
Fixed
Priority:
Normal
Assignee:
-
Category:
Descrambling
Target version:
-
Start date:
2017-01-25
Due date:
% Done:

100%

Estimated time:
Found in version:
4.1-2405~geb495a0~xenial
Affected Versions:

Description

Hi,
I have a setup with an IPTV .m3u List and an automatic Network. The List is provided by an enigma2 receiver with CI+ module. The Receiver gives mpeg-ts streams with descrambled video and audio. But, TVH recognizes the services extracted from the muxes as encrypted (services tab -> column "Encrypted"). So playing a mux with VLC is possible but playing the service is not possible (no output). Mapping the service to a channel is possible, but I'm not able to play it ether. I guess TVH fails here because the stream is encrypted (but is not) and I have no descrambling things configured.

I whould expect that TVH gives the output to the client even if it can not descramble or that there is a possibility to say TVH that a service is not encrypted.

I wrote about the issue in the forum. https://tvheadend.org/boards/5/topics/24497

Here are the results of my investigation so far:

The tv receiver delivers mpeg-ts streams. Here is an example from tvheadend.

Channel Junior:
Is encrypted, can be watched in the receiver and in TVH over the muxes tab.
VLC shows following codec info on playback:

Stream 0
    Type: Video
    Original ID: 1791
    Codec: H264 - MPEG-4 AVC (part 10) (h264)
    Resolution: 720x578
    Display resolution: 720x576
    Frame rate: 25
    Decoded format: Planar 4:2:0 YUV
Stream 1
    Type: Audio
    Original ID: 1792
    Codec: MPEG Audio layer 1/2 (mpga)
    Language: German
    Channels: Stereo
    Sample rate: 48000 Hz
    Bitrate: 192 kb/s
Program 19
    Scrambled: Yes

On the services tab TVH has created the service entry (encrypted: yes) with following info:

Index    PID    Type    Language    Details
     0x06ff /  1791    PCR          
     0x0066 /   102    PMT          
4    0x06ff /  1791    H264          
5    0x0700 /  1792    MPEG2AUDIO    ger     
1    0x1b70 /  7024    CA         CAIDS: 09c4:000000
2    0x1a70 /  6768    CA         CAIDS: 098c:000000
3    0x1f70 /  8048    CA         CAIDS: 09af:000000

After filtering and reordering (without PCR and PMT)
Index    PID    Type    Language    Details
4    0x06ff /  1791    H264          
5    0x0700 /  1792    MPEG2AUDIO    ger     
1    0x1b70 /  7024    CA         CAIDS: 09c4:000000
2    0x1a70 /  6768    CA         CAIDS: 098c:000000
3    0x1f70 /  8048    CA         CAIDS: 09af:000000

When I play the service, VLC can not play it (no error or something, just no output) and shows following codec info:

Stream 0
    Type: Video
    Original ID: 1791
    Codec: H264 - MPEG-4 AVC (part 10) (h264)
Stream 1
    Type: Audio
    Original ID: 1792
    Codec: MPEG Audio layer 1/2 (mpga)
    Language: German
    Description: clean effects

There are no errors in TVH web interface log all the time.

My solution in the meantime is to push the IPTV mpeg-ts through ffmpeg pipe to filter all unknown streams. So the CA streams are filtered.

The pipe looks like this:

pipe:///usr/bin/ffmpeg -loglevel fatal -i http://INPUT -map 0 -c copy -ignore_unknown -metadata service_provider=PROVIDER -metadata service_name=CHANNEL -f mpegts -tune zerolatency pipe:1

After that TVH generates the services without encrypted flag and I'm able to play the services, map them to channels, watch in Kodi and do records over TVH.

But:
It is slower and sometimes not really stable.

P.S.: I don't know if the category is right.


Files

debuglog_20170202_214200.txt (6.72 KB) debuglog_20170202_214200.txt Paul Brunniem, 2017-02-02 21:49
debuglog_vlc_20170202_221500.log (4.98 KB) debuglog_vlc_20170202_221500.log Paul Brunniem, 2017-02-02 22:19
debuglog_tvh_20170202_221500.log (18.2 KB) debuglog_tvh_20170202_221500.log Paul Brunniem, 2017-02-02 22:19
debuglog_vlc_20170202_223400.log (8.25 KB) debuglog_vlc_20170202_223400.log Paul Brunniem, 2017-02-02 22:36
debuglog_tvh_20170202_223400.log (6.97 KB) debuglog_tvh_20170202_223400.log Paul Brunniem, 2017-02-02 22:36
tracelog_tvh_20170206_210300.txt (2.31 MB) tracelog_tvh_20170206_210300.txt Paul Brunniem, 2017-02-06 21:06
tracelog_tvh_oscam_default_20170207_222500.txt (632 KB) tracelog_tvh_oscam_default_20170207_222500.txt oscam_default Paul Brunniem, 2017-02-07 22:26
tracelog_tvh_oscam_newest_20170207_223000.txt (572 KB) tracelog_tvh_oscam_newest_20170207_223000.txt oscam_newest Paul Brunniem, 2017-02-07 22:31
debuglog_vlc_directplay_20170207_194200.log (23.4 KB) debuglog_vlc_directplay_20170207_194200.log Paul Brunniem, 2017-02-07 23:05
tvheadend_gdb_valgrind_force_ca_ffff.txt (15.1 KB) tvheadend_gdb_valgrind_force_ca_ffff.txt Joe User, 2017-02-10 17:47
tracelog_tvh_20170219_215100.txt (813 KB) tracelog_tvh_20170219_215100.txt Paul Brunniem, 2017-02-19 21:58
debuglog_vlc_20170219_220000.log (16.2 KB) debuglog_vlc_20170219_220000.log Paul Brunniem, 2017-02-19 22:03
debuglog_vlc_20170220_203200.log (16.6 KB) debuglog_vlc_20170220_203200.log Paul Brunniem, 2017-02-20 22:58
tracelog_tvh_htsp_20170220_225400.txt (1.8 MB) tracelog_tvh_htsp_20170220_225400.txt Paul Brunniem, 2017-02-20 22:59
tracelog_tvh_20170220_203200.txt (3.67 MB) tracelog_tvh_20170220_203200.txt Paul Brunniem, 2017-02-20 22:59
tracelog_tvh_htsp_20170221_213500.txt (1.23 MB) tracelog_tvh_htsp_20170221_213500.txt Paul Brunniem, 2017-02-21 21:40

Associated revisions

Revision f156eb9b (diff)
Added by Jaroslav Kysela over 4 years ago

descrambler: when CAID is forced to 0xffff, expect external descrambler, fixes #4193

Revision ef89ef8b (diff)
Added by Jaroslav Kysela over 4 years ago

descrambler: fix forced CAID 0xffff (external descrambling), fixes #4193

Revision 3525a9ca (diff)
Added by Jaroslav Kysela over 4 years ago

mpegts input, iptv network: add 'Remove scrambled bits' option, fixes #4193

History

#1

Updated by Anonymous over 4 years ago

Try this instead:

pipe:///usr/bin/ffmpeg -loglevel fatal -i http://INPUT -map 0:0 -map 0:1 -acodec copy -movflags +faststart -vcodec copy -ignore_unknown -metadata service_provider=PROVIDER -metadata service_name=CHANNEL -tune zerolatency -f mpegts pipe:1

Let us know how steady it is.

#2

Updated by Paul Brunniem over 4 years ago

Mi Lano wrote:

Try this instead:

pipe:///usr/bin/ffmpeg -loglevel fatal -i http://INPUT -map 0:0 -map 0:1 -acodec copy -movflags +faststart -vcodec copy -ignore_unknown -metadata service_provider=PROVIDER -metadata service_name=CHANNEL -tune zerolatency -f mpegts pipe:1

Let us know how steady it is.

Hi,
I tried this line. Its not faster or anything. I don't think the changes in your line would have an effect anyway. The changed map specifies the first and second stream to map, what "-ignore_unknown" does already (-ignore_unknown leaves also a possible second audio inside). The copy flags have also the same effect as before. "-movflags +faststart" should only affect outputs of mp4 containers.

Thanks for your Answer, but my goal was to avoid the use of the ffmpeg pipe.

#3

Updated by Joe User over 4 years ago

have you tried fixing the problem at the source? Enigma2 should not be sending descrambled streams with the scrambled flag set. Do you have:

Include ECM in http streams      unchecked
Descramble sending http streams    checked

Have you tried using a CA filter to block all CAIDs??? (under Configuration->Stream->CA stream filters)

#4

Updated by Paul Brunniem over 4 years ago

Hi Joe,
the settings at the receiver are like you describe. So it's wired that the receiver sets the flag. I will ask about that problem at the openatv forum.

I tried also the CA filters and can see the filtering in the logs when I start a subscription.

2017-01-31 20:08:29.533 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 001 001 06659 ffff ffffffff IGNORE

2017-01-31 20:08:29.533 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 002 001 06915 ffff ffffffff IGNORE

2017-01-31 20:08:29.533 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 003 001 07939 ffff ffffffff IGNORE

But that changes nothing. I guess these filters are for the final streaming and not for the TVH mux to service translation. I tried to stream and also read the mux with the enabled filter. The filters appear in the logs, only when streaming.

Thanks for your ideas.

#5

Updated by Joe User over 4 years ago

So you can play it with VLC as a mux but not a service/channel?

When you play in VLC, under statistics, does it show and data being received?

Are you able to record?

Did you check "Continue even if descrambling fails:" in your stream profile?

If you play the stream with VLC directly from your enigma2 box, does it show the stream as scrambled?

Have you tried enabling the SAT>IP server (tab under general) and set the descrambling limit to 0?

#6

Updated by Paul Brunniem over 4 years ago

So you can play it with VLC as a mux but not a service/channel?

Yes.

When you play in VLC, under statistics, does it show and data being received?

As mux yes, normal data flow.
As service no, only some kb meta data.

Are you able to record?

No.

Did you check "Continue even if descrambling fails:" in your stream profile?

Yes, on every profile. There is also no TVH log entry with any descrambling fails.

If you play the stream with VLC directly from your enigma2 box, does it show the stream as scrambled?

Yes, the output is the same as from TVH (see my init post, first box) and it does also play the stream like playing the mux over TVH.

Have you tried enabling the SAT>IP server (tab under general) and set the descrambling limit to 0?

I have tried it now and it changed nothing.

#7

Updated by Joe User over 4 years ago

To test SAT>IP server you need to use a SAT>IP client (RTSP) There are some free android clients (I personally never used) or DVBViewer for PC (free lite version will work.)

A debug log might be helpful...

#8

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

To test SAT>IP server you need to use a SAT>IP client (RTSP) There are some free android clients (I personally never used) or DVBViewer for PC (free lite version will work.)

A debug log might be helpful...

Well, I don't want to use RTSP. My goal is to get picture and audio on kodi (HTSP) and VLC (pass). (I use VLC only for testing if TVH is able to do anything with the encrypt flagged channels.)

Problem seams to be that my source (enigma2) sends the encrypted flag. Thats no TVH problem, but I can not override the flag in TVH services, its readonly.

Here is a debug enabled log from reading the mux and playing the service after.

Loglevel debug: enabled

2017-02-02 20:26:37.855 mpegts: disneycine in IPTV  PIPE - tuning on IPTV

2017-02-02 20:26:37.857 subscription: 001B: "scan" subscribing to mux "disneycine", weight: 5, adapter: "IPTV", network: "IPTV  PIPE", service: "Raw PID Subscription" 

2017-02-02 20:26:52.855 mpegts: disneycine in IPTV  PIPE scan complete

2017-02-02 20:26:52.855 subscription: 001B: "scan" unsubscribing

2017-02-02 20:26:52.856 mpegts: disabling service IPTV  PIPE/disneycine/{PMT:98} [sid 007F/127] (missing in PAT/SDT)

2017-02-02 20:26:52.856 mpegts: disabling service IPTV  PIPE/disneycine/{PMT:99} [sid 0072/114] (missing in PAT/SDT)

2017-02-02 20:26:52.856 mpegts: disabling service IPTV  PIPE/disneycine/{PMT:97} [sid 0070/112] (missing in PAT/SDT)

2017-02-02 20:26:52.856 mpegts: disabling service IPTV  PIPE/disneycine/{PMT:100} [sid 006E/110] (missing in PAT/SDT)

2017-02-02 20:27:17.262 http: HOST: using ticket E79230C97BFB9C8B8C93704E19A81C7E4C0D73F9 for /stream/service/6d6e61f0d59d77fa500560043f560f25

2017-02-02 20:27:17.264 subscription: 001C: "HTTP" subscribing to service "IPTV  PIPE/disneycine/{PMT:96}", weight: 100, adapter: "IPTV", network: "IPTV  PIPE", mux: "disneycine", profile="pass", hostname="HOST", client="VLC/2.2.4 LibVLC/2.2.4" 

2017-02-02 20:27:18.975 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 001 001 06659 ffff ffffffff IGNORE

2017-02-02 20:27:18.975 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 002 001 06915 ffff ffffffff IGNORE

2017-02-02 20:27:18.975 service: esfilter: "IPTV  PIPE/disneycine/{PMT:96}" CA 003 001 07939 ffff ffffffff IGNORE

2017-02-02 20:27:46.656 subscription: 001C: "HTTP" unsubscribing, hostname="HOST", client="VLC/2.2.4 LibVLC/2.2.4" 

The debug log shows nothing more then the normal log.

If someone can give me advice how to change the encrypted flag for a single service in the config over the file system, that would be nice. So I could test if that is the only problem.

Thanks for your time.

#9

Updated by Joe User over 4 years ago

need the FULL log (read wiki about debugging) and without using the pipe and not streaming full mux.

#10

Updated by Joe User over 4 years ago

Have you tried creating just a single IPTV network (not automatic) then create a mux for that network with a single channel (url for one channel from enigma2)?

#11

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

need the FULL log (read wiki about debugging) and without using the pipe and not streaming full mux.
Have you tried creating just a single IPTV network (not automatic) then create a mux for that network with a single channel (url for one channel from enigma2)?

Ok, here is the full log (subsystems "all") of the new iptv (not auto) network "test" with the manual added mux (direct link without pipe, as before) played over the service tab: attachment

#12

Updated by Joe User over 4 years ago

In VLC , can you go to tools-> messages (CTRL-M) and set verbosity to 2 (debug) and then try to play the stream and upload the log?

#13

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

In VLC , can you go to tools-> messages (CTRL-M) and set verbosity to 2 (debug) and then try to play the stream and upload the log?

I have the logs of VLC and TVH attached. I started the subscription and waited until VLC says it cant play (little error dialog).

#14

Updated by Paul Brunniem over 4 years ago

Sorry, forget the last post. I made a mistake. The Receiver was not ready.

Here are the new logs.

#15

Updated by Joe User over 4 years ago

How are you adding the url?

On mine I see:

[   INFO] subscription: 0019: "HTTP" subscribing to service "enigma2/http://192.168.1.12:8001/1:0:19:4333:300C:13E:820000:0:0:0:/{PMT:531}", weight: 100, adapter: "IPTV", network: "enigma2", mux: "testingenigma", profile="all", hostname="192.168.1.16", client="VLC/2.2.4 LibVLC/2.2.4" 

on yours:

[   INFO]:subscription: 002C: "HTTP" subscribing to service "test/disneyCinemagic/{PMT:96}", weight: 100, adapter: "IPTV", network: "test", mux: "disneyCinemagic", profile="pass", hostname="HOST", username="admin", client="VLC/2.2.4 LibVLC/2.2.4" 

Try adding by:
1. configuration-> DVB Inputs-> Networks-> add-> IPTV Network
2. add just name to configuration page (i.e. enigma2)
3. switch to configuration-> DVB Inputs-> Muxes - add -> name from step 2. (i.e enigma2)
4. add just the url (something like: http://192.168.1.12:8001/1:0:19:4333:300C:13E:820000:0:0:0 )

It should then shortly automatically scan the new mux, if not go to configuration-> DVB Inputs-> Networks and select the network (i.e. enigma2) and press "Force Scan".

Then back in Muxes, the "Scan status" of the mux should be idle and the "Scan result" should be ok.

Then in Services you should see at least one service for the network that should be enabled and "Automatic checking" should be "auto check enabled". There may be some other services which are not enabled with "Automatic checking" set to "Missing In PAT/SDT". (referring to other channels on the transponder which are not part of the stream...)

Then try to play that enabled service.

#16

Updated by Paul Brunniem over 4 years ago

You are right Jon, there is a difference in the log entries. The difference is that I'm adding a service name "disneyCinemagic" to the mux, beside the url, and you not. All other steps you described are the same.

Do you use scrambled (descrambel from the enigma2 box) channels as mux? Can you play these channels? Is VLC Showing the "scrambled" flag "true" when you play the mux or the direct stream from the box?

If yes, yes and yes:
Whats the TVH Version you are using?

#17

Updated by Joe User over 4 years ago

I tried adding a name and it did not show up, I guess it did not take affect immediately because it shows it now..

Yes I tested with a scrambled channel which was descrambled by enigma2 with oscam. Scrambled flags are removed from the stream. I guess it is some issue with your CI+ module????

Maybe you can try a tracelog of httpc,tbl-base,service,subscription,descrambler,dvbcam,mpegts,iptv-pcr

#18

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

Maybe you can try a tracelog of httpc,tbl-base,service,subscription,descrambler,dvbcam,mpegts,iptv-pcr

Here you go (attachment).

#19

Updated by Joe User over 4 years ago

Mine:

2017-02-06 21:47:54.247 [  DEBUG] service: enigma2/testmux/testname: Status changed to [Hardware input]
2017-02-06 21:47:54.247 [  DEBUG] service: enigma2/testmux/testname: Status changed to [Hardware input] [Input on service]
2017-02-06 21:47:54.247 [  DEBUG] service: enigma2/testmux/testname: Status changed to [Hardware input] [Input on service] [Demuxed packets]
2017-02-06 21:47:54.247 [  DEBUG] service: enigma2/testmux/testname: Status changed to [Hardware input] [Input on service] [Demuxed packets] [Reassembled packets]

Yours:
2017-02-06 20:59:02.898 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] 
2017-02-06 20:59:02.898 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] 
2017-02-06 20:59:02.898 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [No access] 
2017-02-06 20:59:02.904 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [Demuxed packets] [No access] 
2017-02-06 20:59:02.904 [  DEBUG]:service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [Demuxed packets] [Reassembled packets] [No access] 

Not sure what "No access" means, I would have to look at the source code and I do not have time tonight. Maybe someone else knows...

Other difference is that I do have a CA setup and since there are CAIDs present, it does try to find keys even though the stream is already decrypted. (btw - it does not find keys since my CA is not setup for the channel being streamed from enigma2.)

#20

Updated by Joe User over 4 years ago

Maybe you can try setting up a fake CA?

Configuration-> CAs-> add-> CAPMT Give some client name and click on enabled. Leave other default settings and press create.

Then try the stream again (with the trace) and see if it works, or at least gives more info.

BTW - You will see continuous errors in log about not being able to connect.

Also, when you play the stream directly in VLC (without tvh) does it start immediately or is there a short delay. (Check messages and verbosity set to 2 - debug to see if some scrambled packets are sent at the beginning.)

And have you tried streaming a channel that was currently being played on the stb (already being decrypted...)?

#21

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

Maybe you can try setting up a fake CA?

Configuration-> CAs-> add-> CAPMT Give some client name and click on enabled. Leave other default settings and press create.

Then try the stream again (with the trace) and see if it works, or at least gives more info.

BTW - You will see continuous errors in log about not being able to connect.

I have done that and got the connection errors and all other logs where the same.
I thought if there were a oscam runnning and TVH would get errors from oscam then it should continue. So I installed oscam and configured a CAPMT with default setting (OSCAM rev >= 9095).

I got the following: attachment "oscam_default"

There is "[No available descrambler]" inside. So I changed the CA setting to the newest oscam version: OSCam new pc-nodmx (rev >= 10389).
Then I got the following: attachment "oscam_newest"

I played around some more and it seams to me that TVH is sending nothing to oscam when playing a service. So I removed the CA stream filter and enabled the "Intigrate ECM in HTTP-Streams" option in the receiver. After that tvh sends the ECM to oscam, but nothing changes. I played around with the kind of connection to oscam (dvbapi and newcamd). Also nothing changes. TVH logs that the ecm can not be decrypted or something. The option "continue when discrambing fails" seams not to work like I taught.

Without fake CA and without CA stream filter the log locks like this:

2017-02-07 23:16:45.234 service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] 

2017-02-07 23:16:45.234 service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] 

2017-02-07 23:16:45.234 service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [Reassembled packets] 

2017-02-07 23:16:45.234 service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [Reassembled packets] [No access] 

2017-02-07 23:16:45.234 service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [Reassembled packets] [No available descrambler] [No access] 

2017-02-07 23:16:45.234 webui: Start streaming /stream/service/cf0bc4c7b83f3cac204374348942f974?ticket=02DBAB257F33E85764346121EA8F02CD82EF16FE

2017-02-07 23:16:45.241 service: test/disneyCinemagic/disneyCinemagic: Status changed to [Hardware input] [Input on service] [Demuxed packets] [Reassembled packets] [No available descrambler] [No access] 

2017-02-07 23:16:45.241 tbl-base: pat: 0x55562e8d8770: tsid 000D (13)

So I think the solution could be to override the encrypted flag in services tab + filter the CA streams.
But I don't know how to edit the TVH config in the filesystem. The configs are all kind of gzipped.

Also, when you play the stream directly in VLC (without tvh) does it start immediately or is there a short delay. (Check messages and verbosity set to 2 - debug to see if some scrambled packets are sent at the beginning.)

It starts directly and I see no scrambled packets in the log (see attachment)

And have you tried streaming a channel that was currently being played on the stb (already being decrypted...)?

The channel that I'm testing is always being played on the receiver. That's the only way the CI+ module works at all in HTTP streams.

#22

Updated by Jaroslav Kysela over 4 years ago

  • Status changed from New to Fixed
  • % Done changed from 0 to 100
#23

Updated by Jaroslav Kysela over 4 years ago

If you force CAID for the service to 0xffff, TVH won't do any descrambling. v4.1-2419-gf156eb9

#24

Updated by Paul Brunniem over 4 years ago

Jaroslav Kysela wrote:

If you force CAID for the service to 0xffff, TVH won't do any descrambling. v4.1-2419-gf156eb9

Is the fix contained in the nightlies for xenial or do I have to build it my self?
apt-get update/upgrade won't load a new version.

#25

Updated by Mark Clarkstone over 4 years ago

Paul Brunniem wrote:

Jaroslav Kysela wrote:

If you force CAID for the service to 0xffff, TVH won't do any descrambling. v4.1-2419-gf156eb9

Is the fix contained in the nightlies for xenial or do I have to build it my self?
apt-get update/upgrade won't load a new version.

This is because the build scripts are broken right now. So, unfortunately yes, you'll have to build it.

#26

Updated by Joe User over 4 years ago

I just tried setting force caid to 0xffff and when attempting to play an IPTV service, tvheadend crashed at descrambler.c:431

  lock_assert(&t->s_stream_mutex);

Attached is gdb and valgrind output...

Sorry, don't have time to debug further now, but can look at it later if Jaroslav doesn't recognize and solve the problem easily...

#27

Updated by Jaroslav Kysela over 4 years ago

Fixed in v4.1-2426-gef89ef8

#28

Updated by Paul Brunniem over 4 years ago

Hi,
unfortunately it does not work for me.
I made the build with the source with this steps:
build: https://tvheadend.org/projects/tvheadend/wiki/Building
start: https://tvheadend.org/projects/tvheadend/wiki/Development

I re-scanned the mux and set the service setting "Force CA ID (e.g. 0x2600):" to "0xffff".

When I play with VLC is does not work.
- no video/audio
- codec info as before
- statistics info as before (no data)

But, the TVH log does not show "no access" anymore -> little success.

BTW, it does also not work if I activate the CA filters.

I attached the TVH log and VLC log.

#29

Updated by Joe User over 4 years ago

ts debug: new PMT program number=111 version=27 pid_pcr=255
ts debug: * es pid=255 type=27 dr->i_tag=0x52
ts debug: * es pid=255 type=27 fcc=h264
core debug: selecting program id=111
ts debug: * es pid=259 type=6 dr->i_tag=0xa
ts debug: * es pid=259 type=6 dr->i_tag=0x52
ts debug: * es pid=259 type=6 dr->i_tag=0x6a
ts debug: * Stream Component Identifier: 7
ts debug: found language: deu
ts debug: * es pid=259 type=6 fcc=a52 
ts debug: * es pid=260 type=6 dr->i_tag=0xa
ts debug: * es pid=260 type=6 dr->i_tag=0x52
ts debug: * es pid=260 type=6 dr->i_tag=0x6a
ts debug: * Stream Component Identifier: 8
ts debug: found language: eng
ts debug: * es pid=260 type=6 fcc=a52 
core debug: using demux module "ts" 

The pids listed in vlc are different than in tvh - do you have some of the rewrite options selected in you default profile? (or specificed profile for the user) If so, can you try without any rewrite options selected?

Also it seems vlc gives up too quickly, or are you manually stopping the playback?

#30

Updated by Paul Brunniem over 4 years ago

The pids listed in vlc are different than in tvh - do you have some of the rewrite options selected in you default profile? (or specificed profile for the user) If so, can you try without any rewrite options selected?

The rewrite options were all selected. I unselected all and tried again. No change. I attached new logs.
Only the "pass" profile has these options, so I mapped the service to a channel to test the htsp profile with kodi, also no video/audio. TVH says "no input" after few seconds and restarts the channel (see attachment).

Also it seems vlc gives up too quickly, or are you manually stopping the playback?

I stopped VLC after a view seconds, but this time I waited more then a minute.

#31

Updated by Joe User over 4 years ago

In your VLC log I still see the wrong pids, did you do it with all the rewrites unchecked?

Joe User wrote:

Then in Services you should see at least one service for the network that should be enabled and "Automatic checking" should be "auto check enabled". There may be some other services which are not enabled with "Automatic checking" set to "Missing In PAT/SDT". (referring to other channels on the transponder which are not part of the stream...)

Then try to play that enabled service.

I tried playing one of the other services for the mux ( one with "Missing In PAT/SDT") and it shows the same behavior you are seeing. Are you making sure to select the channel with "Automatic checking" should be "auto check enabled"??

In the status->stream tab, what does it show for the bandwidth, and what does it show in status->subscriptions when you are streaming?

BTW - my tvheadend info for the channel also shows it is encrypted (probably because ECM info there) but VLC does not show scrambled.

#32

Updated by Joe User over 4 years ago

Never mind about wrong pids, I see now you are using a different channel than the original post...

Can you save a short stream and upload it somewhere?

something like this:

 curl http://192.168.1.11:8001/1:0:19:4333:300C:13E:820000:0:0:0: -o test.ts -m 15

replacing the url with yours of course.

#33

Updated by Joe User over 4 years ago

Another thing to try: try adding "?descramble=0" to the end of the tvheadend url and see if that helps...

http://USER:[email protected]:9981/stream/service/4645484bb69606acc77e1e133f6b4f65?descramble=0
#34

Updated by Fabio Santos over 4 years ago

I can confirm that "descramble=0" works.

One more thing, after mapping the service to a channel, it plays fine without "descramble=0" and even with the CA client stopped.

#35

Updated by Jaroslav Kysela over 4 years ago

Paul Brunniem wrote:

Hi,
unfortunately it does not work for me.
I made the build with the source with this steps:
build: https://tvheadend.org/projects/tvheadend/wiki/Building
start: https://tvheadend.org/projects/tvheadend/wiki/Development

I re-scanned the mux and set the service setting "Force CA ID (e.g. 0x2600):" to "0xffff".

When I play with VLC is does not work.
- no video/audio
- codec info as before
- statistics info as before (no data)

But, the TVH log does not show "no access" anymore -> little success.

It looks like that the MPEG-TS packets are marked as scrambled. Otherwise, I cannot explain this behaviour.

See: https://github.com/tvheadend/tvheadend/blob/master/src/descrambler/descrambler.c#L694

The dr->dr_external should be set when CAID is 0xffff, thus the packets are passed to the next layer, but only when the scrambled bit in the MPEG-TS packet is zero.

#36

Updated by Joe User over 4 years ago

Jaroslav Kysela wrote:

It looks like that the MPEG-TS packets are marked as scrambled. Otherwise, I cannot explain this behaviour.

Yes, I am pretty sure this is the case (reason VLC reports stream as scrambled.) That is why I asked for sample to verify this is the case...

Jaroslav Kysela wrote:

The dr->dr_external should be set when CAID is 0xffff, thus the packets are passed to the next layer, but only when the scrambled bit in the MPEG-TS packet is zero.
See: https://github.com/tvheadend/tvheadend/blob/master/src/descrambler/descrambler.c#L694

Yes, that is the code I didn't have time to track down. ;) - thanks!

Where does it get overridden to send the scrambled packets? (descramble=0)

I am also not sure of the internal differences when using an IPTV stream compared to a tuner?

#37

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

Never mind about wrong pids, I see now you are using a different channel than the original post...

Sorry for that. I changed the channel because the old one does not send after 8 p.m.

Can you save a short stream and upload it somewhere?

Sure, I used the following:
curl http://receiver:8001/1:0:19:6F:D:85:C00000:0:0:0: -o test.ts -m 15

uploaded here:
http://upfile.mobi/Dasz6jvp9b0

Another thing to try: try adding "?descramble=0" to the end of the tvheadend url and see if that helps...

That doesn't change anything for me.

Jaroslav Kysela wrote:

It looks like that the MPEG-TS packets are marked as scrambled. Otherwise, I cannot explain this behaviour.

Yes, I am pretty sure this is the case (reason VLC reports stream as scrambled.) That is why I asked for sample to verify this is the case...

Jaroslav Kysela wrote:

The dr->dr_external should be set when CAID is 0xffff, thus the packets are passed to the next layer, but only when the scrambled bit in the MPEG-TS packet is zero.
See: https://github.com/tvheadend/tvheadend/blob/master/src/descrambler/descrambler.c#L694

Yes, that is the code I didn't have time to track down. ;) - thanks!

Where does it get overridden to send the scrambled packets? (descramble=0)

I am also not sure of the internal differences when using an IPTV stream compared to a tuner?

Is there a chance to get this to work by building TVH without the descrambling module? ("--disable-cwc" or something?)

#38

Updated by Paul Brunniem over 4 years ago

Compiling with cwc disabled does not work, TVH does not start after that.

Jaroslav Kysela wrote:

See: https://github.com/tvheadend/tvheadend/blob/master/src/descrambler/descrambler.c#L694

The dr->dr_external should be set when CAID is 0xffff, thus the packets are passed to the next layer, but only when the scrambled bit in the MPEG-TS packet is zero.

I changed the line 695 from:

if ((tsb[3] & 0x80) == 0) {

to:

if ((tsb[3] & 0x80) == 0 || 1 == 1) {

compiled again, startet TVH, played the service, and...
works! Got the video/audio in less than a second.

#39

Updated by Paul Brunniem over 4 years ago

Ok so VLC works with the little hack, but htsp (Kodi) does not.
TVH runs in timeout with "no input detected". (see attachment)

#40

Updated by Joe User over 4 years ago

Joe User wrote:

Another thing to try: try adding "?descramble=0" to the end of the tvheadend url and see if that helps...

[...]

Looking at the code, it should bypass the descrambler and just send the packets.

[[https://github.com/tvheadend/tvheadend/blob/master/src/input/mpegts/tsdemux.c#L223]]

"s_scrambled_pass" is set when you add the option "?descrambler=0" to the url request to tvheadend (not url to enigma2...)

I used this to force the sending of scrambled streams to oscam streamrelay and it does work.

For kodi, not sure what you can do, would have to ask on kodi forums.

Yes, cwc is for newcamd connection.

#41

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

Looking at the code, it should bypass the descrambler and just send the packets.

OK you are right, I made a mistake.

I first tried with the URL param and the link I got from the service tab (also ticket param inside).
But with clean URL it works with the service (I of course rejected the hack I made to the source).

http://receiver:9981/stream/service/4645484bb69606acc77e1e133f6b4f65?descramble=0

But the channel URL does not work.

http://receiver:9981/stream/channel/2c8e588a4a1dc6c34c443dfaa914911a?descramble=0

VLC shows no video/audio and TVH does not show any error.

Other question: If the param descramble=0 works, how can this help me to play the channels in Kodi?

#42

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

For kodi, not sure what you can do, would have to ask on kodi forums.

Why do you think that Kodi is the problem? TVH seams to have the problem with the input data.

the log says:

2017-02-21 21:33:50.044 [WARNING]:subscription: 000D: service instance is bad, reason: No input detected

#43

Updated by Joe User over 4 years ago

Yes, I looked in "src/webui/webui.c" and it does not parse the descramble option for channels, but it could be added.

But.... it will still be sending unscrambled packets falsely flagged as scrambled and I do not know how kodi will handle that.

Aside from the ffmpeg/pipe, you would have to make a hack in tvheadend to clear the scrambled flags for every packet. Probably not simple, but should be doable. Other path is to try to fix the problem at the source. I am not sure which stb you have and what image you are running, although it may not matter if the problem is with the CI+ module (which I expect is the case - the descrambler should clear the flag...)

#44

Updated by Joe User over 4 years ago

BTW - why do you not just use the VU+/Enigma2 add-on for kodi to access your stb directly?

#45

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

Yes, I looked in "src/webui/webui.c" and it does not parse the descramble option for channels, but it could be added.

Would this affect the output of the htsp connections? As I understand the URL param dose only change the behavior of the http streams.

But.... it will still be sending unscrambled packets falsely flagged as scrambled and I do not know how kodi will handle that.

I will test this tonight with the enigma2 plugin in Kodi. So Kodi should get the same as TVH when playing direct from the box.

I am not sure which stb you have and what image you are running...

Its a Megasat force 2 with openatv 5.3 running. I asked in the openatv forum but no answers until now.

BTW - why do you not just use the VU+/Enigma2 add-on for kodi to access your stb directly?

I have more than 1 Client and want to use the great TVH flexibility in input sources/recording/EPG...
The receiver will only be a cheep source giving descrambled DVB-S.

#46

Updated by Jaroslav Kysela over 4 years ago

Paul Brunniem wrote:

Ok so VLC works with the little hack, but htsp (Kodi) does not.
TVH runs in timeout with "no input detected". (see attachment)

Remove

  if (tsb2[3] & 0xc0) /* scrambled */
    continue;

condition in src/input/mpegts/tsdemux.c in ts_recv_packet0().

But anyway, your stream don't follow any specification / rules - it is just broken, so these changes cannot be in the tvh upstream code.

#47

Updated by Joe User over 4 years ago

Or replace it with:

tsb2[3] &= 0x3F;

which will clear the scrambled flag.

#48

Updated by Paul Brunniem over 4 years ago

Joe User wrote:

Or replace it with:
[...]

which will clear the scrambled flag.

That does not work because its only readable.

Jaroslav Kysela wrote:

Remove

[...]

condition in src/input/mpegts/tsdemux.c in ts_recv_packet0().

I removed it and it works now with HTTP stream (without the URL param) and htsp in Kodi.

But anyway, your stream don't follow any specification / rules - it is just broken, so these changes cannot be in the tvh upstream code.

What prevents you from using the same flag ("The dr->dr_external should be set when CAID is 0xffff") from the service in the channels?

Anyway, it works now. If that means that I have to build TVH my self and modify a few lines when I update, than it's ok for me.

#49

Updated by Jaroslav Kysela over 4 years ago

Paul Brunniem wrote:

But anyway, your stream don't follow any specification / rules - it is just broken, so these changes cannot be in the tvh upstream code.

What prevents you from using the same flag ("The dr->dr_external should be set when CAID is 0xffff") from the service in the channels?

There are basically two expectations for the stream:

1) don't descramble stream at all - for SAT>IP server or special external descrambler (HTTP parameter), the data are passed AS-IS
2) external descrambling (hardware CAM modul) - usually, the hardware CAM modules don't descramble first few MPEG-TS packets, so they should be skipped (they have the scrambling bits set)

It's the difference between s->s_scrambled_pass and dr->dr_external. TVH also should not parse the A/V packets when the scrambling bits are set (garbage).

Thinking more about your problem, the scrambled flag in the MPEG-TS packets should be cleared in the input code before any processing.

#50

Updated by Jaroslav Kysela over 4 years ago

Added 'Remove scrambled bits' to the IPTV network settings - v4.1-2445-g3525a9c .

#51

Updated by Paul Brunniem over 4 years ago

Works like a charm. Thank you all very much for your help (it was greatly appreciated). Also I learned many new things.

#52

Updated by Joe User over 4 years ago

Paul Brunniem wrote:

That does not work because its only readable.

Sorry, I was in a hurry and did not notice the ts buffer is passed around as a const.

I was thinking you could make a constant code word entry and then in tvhcsa.c check for caid 0xffff and if so just clear the scrambled bits... but Jaroslav already made a fix. :)

@Jaroslav, do you think this could be part of the problems in this issue: [[https://tvheadend.org/issues/2794]]???

Also available in: Atom PDF