(Auto)Scan badly misses time-out

Digard

I have been fighting with an otherwise almost impeccable tvheadend for the last weeks, together with the very helpful support of Sundtek.
I couldn't get a useful, complete, scan out of it; tried on different computers, different plugs and cables. The autoscan would always get stuck at a certain percentage, or bring up wrong results (https://support.sundtek.com/index.php?topic=2612.0), while other hardware (TV, cable tuner) and other software (w_scan) would perform okay on the same plug and cable.

Yesterday we found out, that for certain problems (local interference?) the 370 MHz signal is not clean.
In these cases, tvheadend gets stuck, keeps trying for literally hours, until I abort the scan (https://support.sundtek.com/index.php?topic=2603.0).
The log does

SIGNAL: [.............................    ] ( 88%) BER:      0 CNR: 33.70 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256
SIGNAL: [.............................    ] ( 88%) BER:      8 CNR: 33.70 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256
SIGNAL: [.............................    ] ( 88%) BER:      8 CNR: 33.50 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256
SIGNAL: [.............................    ] ( 88%) BER:      8 CNR: 33.70 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256
SIGNAL: [.............................    ] ( 88%) BER:      6 CNR: 33.50 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256
SIGNAL: [.............................    ] ( 88%) BER:      6 CNR: 33.50 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256
SIGNAL: [.............................    ] ( 88%) BER:      0 CNR: 33.50 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256
SIGNAL: [.............................    ] ( 88%) BER:      0 CNR: 33.70 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256
SIGNAL: [.............................    ] ( 88%) BER:      0 CNR: 33.70 FREQ: 370000000  Hz LOCKED: YES SYM: 6900000 MOD: QAM256

for eternity, or until I cancel. Logically, the next muxes will never be touched, and read out. Therefore I end up with a kind of variant number of services, or - worse - even with wrong parameters in the list (see first forum post, where a service in the 370 MHz-band was registered under 170 MHz).

For the time being I ASSUME that the other hard-/software simply skips muxes/services when problems are encountered, and continue with the next one, so that me, the user, gets an eventually not quite complete list, but a completed scan.

I wished that tvheadend could do likewise, and instead of running circles about a mux with eventually distorted response, after some time continue with the next mux/service.

Any further enlightenment, ideas, help respectively a code modification including a timeout will be welcome!

davep

There is a timeout, see Configuration -> Channel/EPG -> EPG Grabber -> OTA Grabber Settings. However this may only affect the grabber itself, and if the signal is corrupt the grabber may never get started.

Digard

davep
The EPG Grabber doesn't start, can't start, because the service that I need, TV5 Monde, got recorded on 170 MHz instead of 370 MHz. The cable tuner, everything the same except plugging the cable from the Sundtek stick to the cable tuner, finds, plays and records this channel flawlessly.
And, even if repeating myself, this channel is found 100 percent reliably on the identical hardware with w_scan:

$ cat scan_w.log | grep Monde
        service = TV 5 Monde (fre) (Digital Free)
TV 5 Monde (fre);Digital Free:370000:M256:C:6900:451=2:452=fra@3:454:0:53007:61441:10002:0

I can also "Watch TV" this channel. Then it tunes to 170 MHz and some unknown error occurs. On 170 MHz this provider offers only encrypted channels; so the error is most obvious.

Just for completeness I attach the channel configuration of the provider, and the scan result for this channel. (I don't know if and how other channels are effected.) Though this is secondary.
Primarily I expect the software to time-out, continue despite of a "corrupt" mux/service, move to the subsequent, and finish cleanly.

Jonas Lang

Might be worth scanning each mux individually once you’ve identified which ones are active.

Digard

Jonas Lang
Yep. Wished I new how to do that!
Would be a possible workaround, since I have the complete list. Would take lots of time, 32 muxes, but I have the complete list of the provider available.

Jonas Lang

Digard Enter one known working mux into the Network section and do a full Network scan. That should throw up the remaining working muxes and scan them too.

Digard

Of course, "knew" (already using a brown bag ...)

or, the other way round, since w_scan seems to do a perfect job, can't we just import the scan results into tvheadend, and thereby circumvent the not-so-optimal scan of tvheadend?

Digard

Jonas Lang Still thinking, and working on this ... .
Sorry, I don't really understand. Of course, I have started with a predefined list, got contact with the provider, received some more muxes than had been noted in the pre-defined list, had the scan starting, finding muxes, and so forth.
Unfortunately, all scans, from the first one after my initial install (just a few weeks ago) until the last one, got stuck at some moment. (All the details can be found in the two forum links in the first post.)

Maybe for demonstration purposes, I attach the w_scan log. This always goes through flawlessly. The scan with tvheadend always gets stuck. The service with which I noticed the consequences is "TV 5 Monde" on 370 MHz.

Further up, thanks to the hint given in the Sundtek forum, I could give the command to actually see what is going on. There it became obvious (I only pasted a few lines of thousands of identical lines) that the scan process got stuck.

Now I could imagine as workarounds, to either scan EVERYTHING, excluding this mux, hoping that the scan goes through without this one; or import the values in the attachment that look good, as list of services to be used in tvheadend.
Or, preferably, an addition to the code that times out on muxes/services on which the scan goes around in circles.

My excuses if I have missed the point of your answer. But so far I have found two ways of initiating a scan: installation->wizard; or Force Scan. I have no idea on how to do better; on initiating a scan solely for one mux, or so.

scan-w.txt

83kB

Flole

But that's w_scan and not Tvheadend? So you're kinda at the wrong place with that..... You'd have to report it to the w_scan maintainers.

I'm still failing to see the issue here, you're doing thousands of things at the same time. Focus on the Tvheadend issue only, it doesn't matter if other software does it differently or better, that's entirely irrelevant here and only causes confusion.

Digard

Flole
Sorry. No need to report to w_scan, because the scan goes through. I only attached it to show that it does go through, identical hardware. (And so does the external hardware cable tuner, attached to same plug with same cable.)
Yep, let's focus on tvheadend! The scan has so far never gone through, because the 'temperature' or channel percentage display stopped at a certain percentage. Further up you can see, where it starts to run at circles for eternity: the 370 MHz mux.

My question is, and remains, how can I actuate a scan that does go through? Even in case if we had some local interference here at 370 MHz (I have no equipment to find out), then how can I exclude this mux? I have the list of all muxes and services from the provider. Then I could manually add one after the other, and scan the muxes individually? Could this be done? How?

I think it is fair, to ask, if tvheadend has a way to import existing scan results. If it doesn't, so be it.

Flole

Well you still haven't posted a debug log from Tvheadend.....

Digard

Flole
I'd love to! I thought the one further up was sufficient. Please, what exactly do you want me to do? How do I produce the log that is needed?
You want to contents of the box in the web interface accessible with the double arrow?

Jonas Lang

I’ve noticed over the last couple of days there has been a rush of users, possibly new users all simultaneously posting their issues. Reminds me of our public transport system. Nothing for ages then everything arrives together. Good to see some of the more seasoned users back contributing to the forum too. I digress.

You now have a Network defined. Forget about using predefined mux tables. They are woefully out of date and basically useless.

Ensure your defined network is empty and free of all muxes. From there you can add a known working mux and enable a full network scan. That should provide you with a full list of your local muxes fully scanned and ready for services to be mapped.

I’ve posted this procedure a number of times on the forum so rather than repeat myself again, search the forum for further details.

Digard

Jonas Lang
Tried your suggestion, I think, correctly. That is, starting from scratch, adding TVadapter, enabling it, add DVB-C network, enabling it, add a mux as given on the current website of the provider, 370000000, symbol rate 6900000, QAM256, saved it. Since then it is resting there, saying PEND. Tried to change it to IDLE and ACTIVE, Saved and Applied, to no avail. It stays on PEND. I have Force Scan-ed the network. So far nothing.
Am I missing anything?

DeltaMikeCharlie

You may also need to change the following:

DeltaMikeCharlie

@Digard - Can you please go into Configuration -> DVB Inputs -> Muxes -> Select 370Mhz -> Edit, and take a screen print of the settings for that mux?

It would be interesting to compare your TVH values to the scan-w values for the same mux.

Digard

DeltaMikeCharlie
I think this is the one? I had submitted it already a few days ago to the Sundtek forum. There I was told that the 64QAM was wrong. It was assumed that I had set it; but I never had (being a beginner, didn't even know it existed, having just started with wizard). Then I changed to 256QAM, Force Scan, and the stuck state, observed with mediaclient, was posted in my first post as above.

(I'll look into the other post(s) tomorrow, since it is nightfall here, sorry.)

Jonas Lang

The majority of the issues as described above relate to using the dreaded predefined mux tables during the Configuration Wizard process. Maybe it’s time we either ditch the predefined muxes option in the Configuration Wizard or maintain our own predefined mux tables. Personally I’d opt for ditching the predefined mux tables and point the new user towards a full network scan using a known mux.

Maybe the developers might want to have a closer look at the Configuration Wizard with a view to overcoming this recurring issue. Maybe invite the user to enter a known working mux and enable a full network scan during the Configuration Wizard process.

Digard

Jonas Lang
As you know, I'm a beginner. A few weeks ago I didn't know what a mux was (and still today, I don't, really).
But w_scan doesn't know either, and makes no mistake; and the 'stupid' hardware cable tuner also doesn't know anything about my provider. At least, so I think.

What hinders tvheadend to do likewise; respectively import a proper parameter list? That should be serious fun, if I could share that list with friends and neighbours after some allocation change, to just import that list? I guess everyone here, sitting on the same network, gets the same services?

So, why would a user have to google for a working mux?

DeltaMikeCharlie

Digard As you know, I'm a beginner. A few weeks ago I didn't know what a mux was (and still today, I don't, really).

'mux' is short for 'multiplexer'. Multiplexing is a technique for combining multiple data streams into one data stream that can be transmitted and then later be de-multiplexed into the original multiple data streams once it has been received. In the context of digital TV, it is the way that broadcasters send multiple TV channels over the same frequency.

TVH, or any digital TV system, needs to know the details about the mux so that it can decode the data that is being broadcast. This includes the frequency, but also some data encoding and error correction parameters.

Digard But w_scan doesn't know either, and makes no mistake;

Because 'w_scan' is able to determine the list of channels on your 370Mhz mux, we just need to tell TVH the correct settings to use.

Here is an extract from your scan-w file.

-_-_-_-_ Getting frontend capabilities-_-_-_-_ 
Using DVB API 5.10
frontend 'Sundtek DVB-C (III)' supports
INVERSION_AUTO
QAM_AUTO not supported, trying QAM_64 QAM_256.
FEC_AUTO
FREQ (42.00MHz ... 1002.00MHz)
SRATE (0.520MSym/s ... 7.233MSym/s)
-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_

searching QAM256...

tune to: QAM_256  f = 370000 kHz S6900C0  (61441:9999:10002) (time: 12:53.717) 
        already known: (QAM_256  f = 346000 kHz S6900C0  (61441:9999:10008)), but not found by pids

        service = TV 5 Monde (fre) (Digital Free)

You mention that you have already changed your settings to QAM_256.

I have also noticed that your FEC is set to 'NONE', whereas your w_scan has the value set to "AUTO". Have you tried changing this parameter?

Digard

DeltaMikeCharlie
A simple 'NO'.
That's why I pointed out the change of QAM, the only one I ever effected.

The fresh (second) install and using the wizard, selecting the closest provider (actually mine), TV 5 Monde was 'found' on the 170 MHz mux. And during recording, it recorded something on 170 MHz; though there are only encrypted signals.
(The whole set of info can be seen in the post in the Sundtek forum; see above, in case.)