Proper CableCARD/HDHomeRun Prime support

Robert Cameron

CableCARD Network support

This branch/patchset adds a CableCARD network type to Tvheadend. When adding muxes, the "Channel" field is used for the channel number as assigned by your cable provider and present in the CableCARD. After a mux has been tuned at least once (including for the discovery of the service carried on that channel), two read-only options are present as well: Callsign, which is the channel name as provided by your cable provider; and Frequency, which is the actual frequency/mux that the channel is carried on.

Because cable providers do not often populate the transport stream metadata fields, information such as a service's channel number, name and provider name are filled at creation time by information present in the mux (channel number and name) or in your network settings (in the case of the provider name).

HDHomeRun Prime

Presently, support for CableCARD networks is only exposed to the HDHomeRun driver. The new functionality in this patchset allows HDHomeRun Prime devices to be recognized as CableCARD tuners, to allow them to be set as the adapter/tuner for CableCARD networks.

Because of the way that these devices tune virtual channels as mapped by the CableCARD, each mux will only have a single service discovered on it. The reason for this is that the hardware device itself performs the PID filtering and program selection when it decrypts a channel's stream. To ensure that the PID filter is not set when using a CableCARD, all calls to PID filtering/resets have been skipped for that particular network type.

Todo

I still want to add the ability to use a device's

lineup.json

file to create an initial set of muxes much in the same way that pre-defined muxes are used for other network types.

Also, because there is no guide data carried on the network itself, perhaps the ability to use SiliconDust's limited 24 hour guide data provided with their device's could be implemented as a type of internal/OTA EPG grabber type.

Sources

I have created forks on GitHub for both the master development (4.3) branch, as well as a branch for the current stable/release (4.2) branch. Also, attached are the patchsets for either master or release/4.2 branches.

"cablecard-4.3" "cablecard-4.2"

Ted Romer

Thanks, I'm looking forward to trying this out!

A thought: short of adding support for automatically creating muxes from lineup.json, it ought to be possible to cobble together a script using the tvheadend web api, along the lines of this example https://github.com/ties/tvheadend_m3importer/blob/master/channels.py (can't vouch for it, I just Googled "/api/mpegts/network/mux_create").

Robert Cameron

Yes, something like that might work as a stop-gap. I'll look into and possibly see about something with plain POSIX

sh@, @curl

and @jq@.

I've also pushed some updates and refactors to my GitHub repo for both 4.2 and 4.3, and if requested, I'll also make the diffs available, too.

Ted Romer

I created a CableCard network and associated the three HDHomeRun tuners with it, and manually added muxes (through the UI) for two channels using my provider's channel numbers. The muxes go to scan status "PEND" and stay there indefinitely (10+ minutes) ... am I missing a step?

Ted Romer

I also tried with "Accept zero value for TSID" checked, same result.

This is running with your cablecard-4.2 branch @7ffca2488.

I enabled debug and trace output for mpegts,service,subscription,tbl-base and see the following output when adding a mux for channel 105 on network "131eb245":

mpegts: adding mux Multiplex [onid:FFFF tsid:FFFF] in 131eb245 to scan queue weight 5 flags 1000
mpegts: Multiplex [onid:FFFF tsid:FFFF] in 131eb245 - created
mpegts: 105 in 131eb245 - add raw service
subscription: 0003: creating subscription for service 105 in 131eb245 weight 5 using profile <none>
subscription: 0003: find instance for 105 in 131eb245 weight 5
subscription: 0003: "scan" unsubscribing

Robert Cameron

Either manually choosing a mux to play will force the mux to scan for services first, and jump start the scanning of pending muxes; alternatively, restarting Tvheadend will allow the scan to proceed as normal. I believe this is related to the network not having an adapter associated with it when the mixes enter a pending state.

Also, on the adapter page for the adapter itself, not each frontend, ensure that "Remove scrambled bits" is checked. This ensures that the encrypted flag set on the stream does not cause Tvheadend to attempt to use an external descrambler.

Ted Romer

I checked remove scrambled bits for each of the 3 adapters and restarted tvheadend. The muxes still stay in pending.

When I try to play the mux I get an empty response and I see this in the logs:


http: 127.0.0.1: using ticket NNN for /stream/mux/MMM
mpegts: 107 in 131eb245 - add raw service
subscription: 003F: creating subscription for service 107 in 131eb245 weight 10 using profile <none>
subscription: 003F: find instance for 107 in 131eb245 weight 10
subscription: 003F: instance not available, retrying
subscription: 003F: find instance for 107 in 131eb245 weight 10
subscription: 003F: No input source available for subscription "HTTP" to mux "107 in 131eb245"
webui: Couldn't start streaming /stream/mux/MMM?ticket=NNN, No assigned adapters
subscription: 003F: "HTTP" unsubscribing, hostname="127.0.0.1", client="Mozilla/5.0 (X11; CrOS x86_64 10452.85.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.158 Safari/537.36"

Ted Romer

I restarted from scratch and the muxes scan just fine. Not sure what was different this time around -- perhaps I checked "Remove scrambled bits" on the tuners before adding any muxes.

Also worth noting that it is necessary to check the "Accept zero value for TSID" box for the muxes.

Robert Cameron

Ted Romer wrote:
> I restarted from scratch and the muxes scan just fine. Not sure what was different this time around -- perhaps I checked "Remove scrambled bits" on the tuners before adding any muxes.
>

> Also worth noting that it is necessary to check the "Accept zero value for TSID" box for the muxes.

Actually, the "accept zero value" is only necessary if your cable provider does not set one. I'm in Southern California on Spectrum, and I've never needed to set that. Each of my muxes has a different TSID (and sometimes they change as Charter tends to like reshuffling the channels sometimes).

However, glad to hear that it's working out for you.

Robert Cameron

Ted, I merged in your PR, and then augmented it a little. I turn off OTA EPG and idle scanning by default as well, since neither feature really works with or makes sense for a CableCARD network. I also applied the same changes to the 4.3/master branch.

Ted Romer

Thanks!

Here's a quick and dirty incantation to add channels from lineup.json. jq was new to me, nifty tool.


HDHR=hdhomerun_hostname_goes_here
NETWORK="CableCARD Network"
NETWORK_UUID=$(
  curl -s http://localhost:9981/api/mpegts/network/grid | 
    jq "$(printf '.entries | map(select(.networkname == "%s"))[0].uuid' \
          "$NETWORK")" | 
    sed -e 's@"@@g')
curl -s "http://${HDHR}/lineup.json" | 
  jq '.[].GuideNumber|tonumber' |
  while read vchan; do 
    echo $vchan
    curl --data "uuid=${NETWORK_UUID}" \
         --data "$(printf \
                  'conf={
                     "enabled":1, "epg":1,
                     "vchan":%s,
                     "scan_state":2,
                     "charset":"",
                     "tsid_zero":true,
                     "pmt_06_ac3":0,
                     "eit_tsid_nocheck":false,
                     "sid_filter":0}' \
                  $vchan)" \
         http://localhost:9981/api/mpegts/network/mux_create
    echo
  done

Robert Cameron

That looks good.

A couple of notes to help generalize it, perhaps for a more interactive script:
* The endpoint



/api/mepgts/network/grid

returns an object with an

entries

array enumerating all of the configured networks.
* A call to



http://api.hdhomerun.com/discover

returns an array of HDHomeRun devices on your local network, each one with an

DiscoverURL

key pointing to the the IP address of each individual HDHR device. (Actually, the way it works is that it returns all HDHR devices that have phoned-home to SD's servers that have the same public-facing IP address as the device that made the call to

api.hdhomerun.com

.)
* A call to



http://${HDHR_IP}/discover.json

returns an object with a

FirmwareName

key that can be filtered to see if it contains "_cablecard" to ensure that only CableCARD devices are prompted for their lineups. (Also, by filtering on this field instead of checking for "PRIME" in the

FriendlyName

key, it ought to still work for the TECH models, as well as the new 6-tuner Prime when it is eventually released.)

Hopefully by this weekend I may have some more time to fiddle around with it and see what I can come up with, too.

Robert Cameron

For those tracking this fork, I'd like to mention that it has been merged into master/4.3. For release/4.2 you can still pull from my cablecard-4.2 branch which will maintain feature parity with any changes to this feature-set made to master.

Robert Cameron

Ted, I liked your script, and I pulled in a couple points from yours when crafting mine. They're essentially the same. Ultimately the goal is to allow for either interactive use, or to have it default to the first CableCARD network and first

_cablecard

HDHomeRun device unless a configuration file is found.


#!/bin/sh
_TVHEADEND="http://localhost:9981"  # URL used to access Tvheadend WebUI
_NETWORK=Cable                      # Name of CableCARD network to add muxes to
_DEVICE=FFFFFFFF                    # DeviceID of HDHomeRun device

_UUID=$(
  curl -sS "${_TVHEADEND}/api/idnode/load"                 \
    -d "class=dvb_network_cablecard" -d "enum=1"           |
  jq ".entries[] | select(.val == \"${_NETWORK}\") | .key" |
  tr -d \")

_LINEUP=$(
  curl -sS "http://api.hdhomerun.com/discover"                |
  jq ".[] | select(.DeviceID == \"${_DEVICE}\") | .LineupURL" |
  tr -d \")

for _CHANNEL in $(
  curl -sS "${_LINEUP}" |
  jq '.[].GuideNumber | tonumber')
do
  curl -sS "${_TVHEADEND}/api/mpegts/network/mux_create" \
    -d "uuid=${_UUID}"                                   \
    --data-urlencode "conf={\"epg\":0, \"vchan\":${_CHANNEL}}" > /dev/null
done

Reggie Burnett

Well done Robert! Going to see if the linuxserver tvheadend docker image has been updated to include this.

Reggie Burnett

Also thanks to Ted and Robert for the script. Saved me some time creating all my muxes. Whew!

Robert Cameron

When using a script that pulls the

lineup.json

from the tuner, it might be best disable the channels in the tuner's webui for those you don't care about. By default when requesting

lineup.json

channels that are disabled (have the red X) are not listed in the lineup.

Yusef Neblett

Robert Cameron wrote:
> Ted, I liked your script, and I pulled in a couple points from yours when crafting mine. They're essentially the same. Ultimately the goal is to allow for either interactive use, or to have it default to the first CableCARD network and first

_cablecard

HDHomeRun device unless a configuration file is found.
>

> [...]

Hi Robert,

Can you please share how to run your script?

I have installed the tvhd 4.3 with CableCard drivers and trying to make it work.

Thanks

Robert Cameron

Yusef Neblett wrote:
> Robert Cameron wrote:
> > Ted, I liked your script, and I pulled in a couple points from yours when crafting mine. They're essentially the same. Ultimately the goal is to allow for either interactive use, or to have it default to the first CableCARD network and first

_cablecard

HDHomeRun device unless a configuration file is found.
> >
> > [...]
>
> Hi Robert,
>
> Can you please share how to run your script?
> I have installed the tvhd 4.3 with CableCard drivers and trying to make it work.
>

> Thanks

Create a CableCARD network in Tvheadend. Then, in the script, set the variables for your particular needs:
*



_TVHEADEND

– This is the URL you use to access your Tvheadend server. If you need a user/pass, include it in the URL, such as

http://user:pass@127.0.0.1:9981



_NETWORK

– This is he name of the CableCARD network you created in Tvheadend
*

_DEVICE

– This is the DeviceID of the HDHomeRun Prime that has your channel lineup. You need to have run a channel scan through the device's web UI first; and channels that you don't want to have added to Tvheadend can be disabled in the Prime's web UI, and they will be ignored when creating muxes

After you've made those adjustments, just run the script. When the script completes its run, you will have a mux for each channel of your Prime that hasn't been disabled.

Yusef Neblett

Thanks Robert for your time,

Just to be clear in what commands to use to run the srcipt. Should I create the script file xxxx.sh or xxx.py? and I guess the file path used to run the script doesnt matter correct?

I have basic linux knowledge on scripts.

Appreciatte your time