OK guys (I'm answering myself again), I found a solution for my use case but its a bit slow.
Background:
Like I said in my fist post, the mpeg-ts source from the receiver is decrypted already. What we also see in the first post is that the source contains the CA streams (encryption something) besides the video and audio streams. So I think Tvheadend recognizes the services generated from the mux as Encrypted. It shows the services in services-tab as encrypted by checking the appropriate column. I guess Tvheadend tries to decrypt the streams when I want to play the service, but cant.
So I thought about how could I get Tvheadend to not think the channels are encrypted.
Solution:
I'm now using a pipe string to push the input mpeg-ts trough ffmpeg, mapping all streams, discarding unknown streams (like CA) and returning mpeg-ts again.
The pipe looks like this:
pipe:///usr/bin/ffmpeg -loglevel fatal -i http://INPUT -map 0 -c copy -ignore_unknown -metadata service_provider=PROVIDER -metadata service_name=CHANNEL -f mpegts -tune zerolatency pipe:1
After that Tvheadend generates the services without encrypted flag and I'm able to play the services, map them to Channels, watch in Kodi and do records over tvheadend. -> Nice :)
But:
It is little slower then without pipe and has a little odd behavior:
I cant, or cant most the time, play the muxes from the muxes tab with VLC. The log looks like this:
2017-01-19 21:51:21.099 mpegts: services_pipe.m3u - Disney Cinemagic HD in IPTVPIPE - tuning on IPTV
2017-01-19 21:51:21.106 subscription: 00E6: "HTTP" subscribing to mux "services_pipe.m3u - Disney Cinemagic HD", weight: 10, adapter: "IPTV", network: "IPTVPIPE", service: "Raw PID Subscription", hostname="****", client="VLC/2.2.4 LibVLC/2.2.4"
2017-01-19 21:51:21.108 spawn: Executing "/usr/bin/ffmpeg"
2017-01-19 21:51:26.842 webui: Stop streaming /stream/mux/396cb73f01bf2661a6e9e00ed3a1fbd3?ticket=E1BF047F223D70DED00354FAC6F5C63B774B4373, timeout waiting for packets
2017-01-19 21:51:26.842 subscription: 00E6: "HTTP" unsubscribing, hostname="****", client="VLC/2.2.4 LibVLC/2.2.4"
Ok that "Problem" is no gamebreaker for me. So is also not the next one.
When I play the service or the mapped channel with VLC, VLC shows fragments in the video and it does stutter. I would blame VLC for this. Maybe something with caching what can do Kodi better or something.
The tuning of the piped channels is slower then direct play. This gets relevant when zapping with Kodi. Zapping from one channel to another does now take about 10 Seconds. Unencrypted channels without pipe take 5 seconds.
So you see, its littleof slower then direct but OK for me at the moment.
Final questions are:
1. Is it a bug that Tvheadend recognizes not encrypted streams/services as encrypted when there are encryption info (CA) in the container?
2. Is there a chance to tell Tvheadend that a service is not encrypted? (Maybe make the flag editable or add a flag to the network to tell that there are no encrypted muxes in the network)
3. when I use pipes, what is the right syntax to write spaces in the metadata without getting errors when tuning (like -metadata service_name=Disney Junior HD)?
regards ;)