Project

General

Profile

Bug #1593

TS recordings eats all memory and then tvheadend is killed by oom killer

Added by Eric Valette over 7 years ago. Updated over 7 years ago.

Status:
Fixed
Priority:
Normal
Category:
Muxers
Target version:
Start date:
2013-02-03
Due date:
% Done:

0%

Estimated time:
Found in version:
3.3.338~gbb51160
Affected Versions:

Description

While recording two HD channels on the same Mux to hunt the MKV format generation error, tvheadend was killed due to memory exhaution. Nothing else was running on the machine.

[ 9186.114399] knotify4 invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
[ 9186.114412] knotify4 cpuset=/ mems_allowed=0
[ 9186.114420] Pid: 6415, comm: knotify4 Tainted: P O 3.7.5 #85
[ 9186.114424] Call Trace:
[ 9186.114444] [<ffffffff8144e62e>] ? dump_header.isra.9+0x6a/0x1d6
[ 9186.114455] [<ffffffff8106cf7a>] ? rcu_do_batch.isra.24+0xef/0x24e
[ 9186.114465] [<ffffffff811cc8f5>] ? cap_netlink_send+0x3/0x3
[ 9186.114474] [<ffffffff8107aa7f>] ? oom_kill_process+0x69/0x30c
[ 9186.114482] [<ffffffff8107b06c>] ? out_of_memory+0x1e8/0x247
[ 9186.114490] [<ffffffff8107dc35>] ? __alloc_pages_nodemask+0x5bb/0x6d0
[ 9186.114499] [<ffffffff8107a0fd>] ? filemap_fault+0x24e/0x36d
[ 9186.114508] [<ffffffff8108cf2f>] ? __do_fault+0xa1/0x3e9
[ 9186.114516] [<ffffffff8108f43b>] ? handle_pte_fault+0x28d/0x610
[ 9186.114524] [<ffffffff810906b7>] ? handle_mm_fault+0x4e/0x191
[ 9186.114534] [<ffffffff8101f67f>] ? __do_page_fault+0x373/0x3cb
[ 9186.114543] [<ffffffff810aa744>] ? sys_newstat+0x21/0x29
[ 9186.114552] [<ffffffff814549a2>] ? page_fault+0x22/0x30
[ 9186.114557] Mem-Info:
[ 9186.114561] DMA per-cpu:
[ 9186.114567] CPU 0: hi: 0, btch: 1 usd: 0
[ 9186.114572] CPU 1: hi: 0, btch: 1 usd: 0
[ 9186.114577] CPU 2: hi: 0, btch: 1 usd: 0
[ 9186.114582] CPU 3: hi: 0, btch: 1 usd: 0
[ 9186.114586] DMA32 per-cpu:
[ 9186.114591] CPU 0: hi: 186, btch: 31 usd: 46
[ 9186.114596] CPU 1: hi: 186, btch: 31 usd: 30
[ 9186.114601] CPU 2: hi: 186, btch: 31 usd: 107
[ 9186.114605] CPU 3: hi: 186, btch: 31 usd: 4
[ 9186.114617] active_anon:267773 inactive_anon:90095 isolated_anon:0
[ 9186.114617] active_file:109 inactive_file:112 isolated_file:0
[ 9186.114617] unevictable:0 dirty:0 writeback:0 unstable:0
[ 9186.114617] free:2733 slab_reclaimable:3795 slab_unreclaimable:4426
[ 9186.114617] mapped:123 shmem:42 pagetables:6020 bounce:0
[ 9186.114617] free_cma:0
[ 9186.114643] DMA free:6024kB min:48kB low:60kB high:72kB active_anon:4808kB inactive_anon:4868kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15648kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:68kB slab_unreclaimable:20kB kernel_stack:0kB pagetables:60kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[ 9186.114648] lowmem_reserve[]: 0 1495 1495 1495
[ 9186.114670] DMA32 free:4908kB min:4920kB low:6148kB high:7380kB active_anon:1066284kB inactive_anon:355512kB active_file:440kB inactive_file:448kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:1531780kB mlocked:0kB dirty:0kB writeback:0kB mapped:492kB shmem:168kB slab_reclaimable:15112kB slab_unreclaimable:17684kB kernel_stack:2184kB pagetables:24020kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:2002 all_unreclaimable? yes
[ 9186.114675] lowmem_reserve[]: 0 0 0 0
[ 9186.114682] DMA: 8*4kB 9*8kB 4*16kB 5*32kB 3*64kB 1*128kB 3*256kB 1*512kB 2*1024kB 1*2048kB 0*4096kB = 6024kB
[ 9186.114700] DMA32: 212*4kB 0*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 4912kB
[ 9186.114720] 7617 total pagecache pages
[ 9186.114724] 7278 pages in swap cache
[ 9186.114730] Swap cache stats: add 539945, delete 532667, find 8731/14189
[ 9186.114734] Free swap = 0kB
[ 9186.114738] Total swap = 1952756kB
[ 9186.127377] 393104 pages RAM
[ 9186.127385] 9195 pages reserved
[ 9186.127389] 262274 pages shared
[ 9186.127392] 379990 pages non-shared
[ 9186.127396] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
[ 9186.127434] [ 1071] 0 1071 6031 1 17 212 -1000 udevd
[ 9186.127443] [ 1147] 0 1147 6030 1 16 197 -1000 udevd
[ 9186.127451] [ 1148] 0 1148 6030 1 16 196 -1000 udevd
[ 9186.127463] [ 2449] 0 2449 4780 11 15 53 0 rpcbind
[ 9186.127471] [ 2484] 104 2484 5873 1 17 112 0 rpc.statd
[ 9186.127480] [ 2500] 0 2500 6361 0 17 55 0 rpc.idmapd
[ 9186.127488] [ 2866] 118 2866 489871 1 51 781 0 privoxy
[ 9186.127497] [ 2909] 0 2909 4585 7 12 79 0 oscam
[ 9186.127506] [ 2910] 0 2910 105646 52 25 153 0 oscam
[ 9186.127515] [ 2928] 0 2928 65749 4 30 118 0 rsyslogd
[ 9186.127523] [ 2985] 123 2985 4351 21 13 32 0 dirmngr
[ 9186.127532] [ 3003] 114 3003 31190 0 61 1202 0 timidity
[ 9186.127540] [ 3135] 0 3135 4206 0 12 40 0 atd
[ 9186.127548] [ 3146] 0 3146 1098 0 8 81 0 acpid
[ 9186.127557] [ 3195] 0 3195 17400 0 37 197 0 cupsd
[ 9186.127565] [ 3213] 0 3213 5733 23 16 40 0 cron
[ 9186.127574] [ 3250] 105 3250 8229 115 20 112 0 dbus-daemon
[ 9186.127582] [ 3696] 0 3696 21652 0 13 94 0 pcscd
[ 9186.127590] [ 3982] 120 3982 12530 1396 29 1942 0 tor
[ 9186.127599] [ 4084] 0 4084 41105 401 47 1714 0 wicd
[ 9186.127607] [ 4143] 0 4143 19558 267 42 1312 0 wicd-monitor
[ 9186.127616] [ 4160] 0 4160 4120 1 13 40 0 getty
[ 9186.127624] [ 4161] 0 4161 18396 1 42 124 0 login
[ 9186.127632] [ 4162] 0 4162 4120 1 12 41 0 getty
[ 9186.127640] [ 4163] 0 4163 4120 1 13 39 0 getty
[ 9186.127649] [ 4164] 0 4164 4120 1 13 41 0 getty
[ 9186.127657] [ 4165] 0 4165 4120 1 13 41 0 getty
[ 9186.127666] [ 4186] 0 4186 2529 55 5 518 -1000 dhclient
[ 9186.127674] [ 4251] 0 4251 13077 0 30 153 -1000 sshd
[ 9186.127682] [ 4309] 0 4309 523980 0 61 372 0 console-kit-dae
[ 9186.127691] [ 4379] 124 4379 94533 0 50 879 0 polkitd
[ 9186.127699] [ 4395] 0 4395 5297 1 14 481 0 bash
[ 9186.127708] [ 6045] 0 6045 6705 0 18 54 0 kdm
[ 9186.127716] [ 6051] 0 6051 44087 316 85 6704 0 Xorg
[ 9186.127724] [ 6076] 0 6076 17778 0 41 138 0 kdm
[ 9186.127733] [ 6106] 1000 6106 1081 1 8 30 0 x-session-manag
[ 9186.127741] [ 6155] 1000 6155 3133 8 9 75 0 ssh-agent
[ 9186.127750] [ 6158] 1000 6158 6086 0 17 78 0 dbus-launch
[ 9186.127758] [ 6159] 1000 6159 8252 1 21 180 0 dbus-daemon
[ 9186.127767] [ 6196] 1000 6196 1022 0 7 22 0 start_kdeinit
[ 9186.127775] [ 6197] 1000 6197 94364 63 160 3718 0 kdeinit4
[ 9186.127784] [ 6199] 1000 6199 96056 62 143 4181 0 klauncher
[ 9186.127792] [ 6201] 1000 6201 307081 549 229 5369 0 kded4
[ 9186.127800] [ 6207] 1000 6207 72340 0 132 1455 0 kglobalaccel
[ 9186.127806] [ 6209] 1000 6209 95052 0 134 1212 0 kactivitymanage
[ 9186.127813] [ 6211] 0 6211 56066 0 45 231 0 upowerd
[ 9186.127820] [ 6284] 0 6284 48893 135 31 98 0 udisks-daemon
[ 9186.127827] [ 6293] 0 6293 11904 14 27 73 0 udisks-daemon
[ 9186.127834] [ 6406] 1000 6406 1056 0 8 25 0 kwrapper4
[ 9186.127841] [ 6407] 1000 6407 140774 56 193 4889 0 ksmserver
[ 9186.127848] [ 6410] 1000 6410 143242 1 220 4430 0 kwin
[ 9186.127855] [ 6415] 1000 6415 313023 494 245 2434 0 knotify4
[ 9186.127862] [ 6418] 1000 6418 388938 1422 297 9839 0 plasma-desktop
[ 9186.127868] [ 6437] 1000 6437 70934 0 127 1107 0 kuiserver
[ 9186.127875] [ 6481] 1000 6481 74148 0 132 1081 0 krandrtray
[ 9186.127882] [ 6483] 1000 6483 110123 162 162 1821 0 konsole
[ 9186.127889] [ 6485] 1000 6485 398546 734 313 7285 0 krunner
[ 9186.127896] [ 6486] 1000 6486 110200 175 163 1741 0 konsole
[ 9186.127903] [ 6487] 1000 6487 54217 0 106 4307 0 smart-notifier
[ 9186.127910] [ 6490] 1000 6490 5300 2 15 482 0 bash
[ 9186.127917] [ 6503] 1000 6503 94893 1 94 610 0 pulseaudio
[ 9186.127924] [ 6505] 119 6505 42714 2 20 56 0 rtkit-daemon
[ 9186.127930] [ 6509] 1000 6509 96605 0 138 1173 0 polkit-kde-auth
[ 9186.127937] [ 6549] 1000 6549 5300 2 14 475 0 bash
[ 9186.127944] [ 6603] 1000 6603 87403 0 38 175 0 at-spi-bus-laun
[ 9186.127951] [ 6607] 1000 6607 8063 0 21 70 0 dbus-daemon
[ 9186.127958] [ 6610] 1000 6610 32234 0 32 156 0 at-spi2-registr
[ 9186.127965] [ 6726] 1000 6726 14672 1 34 100 0 su
[ 9186.127972] [ 6738] 0 6738 4926 1 14 142 0 bash
[ 9186.127979] [ 6839] 1000 6839 52026 0 40 157 0 gvfsd
[ 9186.127985] [ 6862] 1000 6862 13435 26 31 94 0 gconfd-2
[ 9186.127993] [ 7092] 1000 7092 14672 1 33 99 0 su
[ 9186.127999] [ 7103] 0 7103 4926 2 15 141 0 bash
[ 9186.128038] [ 7184] 116 7184 1329540 344124 1615 415776 0 tvheadend
[ 9186.128047] Out of memory: Kill process 7184 (tvheadend) score 873 or sacrifice child
[ 9186.128061] Killed process 7184 (tvheadend) total-vm:5318160kB, anon-rss:1376496kB, file-rss:0kB
[ 9186.474260] dib0700: could not acquire lock
[ 9186.474269] dvb-usb: error while stopping stream.

History

#1

Updated by Adam Sutton over 7 years ago

  • Status changed from New to Need feedback

Are these encrypted or FTA streams?

Adam

#2

Updated by Eric Valette over 7 years ago

Adam Sutton wrote:

Are these encrypted or FTA streams?

No its FTA HD channels. Recording even 3 such streams as mkv works well. I usually do not use TS but now I want material that the TV can play directly (see bug I just openned) ;-) Just compiling git today to see if TS memory leak bug is still there. It took one about one hour with two HD recordings to eat the available memory.

#3

Updated by Adam Sutton over 7 years ago

OK, I've got my own system recording 2 simultaneous HD streams (same mux), something I do regularly. Also got a few others doing the same, they're not seeing anything.

There must be something specific to the streams and/or setup. Maybe you could grab a raw mux dump, that might help with debugging.

Adam

#4

Updated by Eric Valette over 7 years ago

Adam Sutton wrote:

OK, I've got my own system recording 2 simultaneous HD streams (same mux), something I do regularly. Also got a few others doing the same, they're not seeing anything.

There must be something specific to the streams and/or setup. Maybe you could grab a raw mux dump, that might help with debugging.

Adam

How much RAM do you have on your machine? I have only 2 GB and 2 GB swap... Why would MKV work and not TS on same stream? You have the stream info in the bug report on MKV and hardware decoder. Started a config from scratch when moving to 3.3.xx. Can give you a raw mux dump if needed.

BTW I have a question how can I produce the mkv via tvheadend when I recorded the TS? (to hunt the MKV bug).

#5

Updated by Adam Sutton over 7 years ago

That machine has 2G (with ZERO swap, and a reasonable chunk dedicated to RAM storage), but that's kinda missing my point. What I was implying is that the memory usage was static, about 3% constantly, a memory leak of the proportions you're talking about would be bringing my server down all the time.

So there must be something specific about the streams or something strange in the way your system is setup. There has been a long standing report about possible memory leaks when streaming multiple encrypted streams (#1383), however no one has managed to replicate and the issue was confused by several other problems.

I don't doubt there could be issues in there, just that there not clear cut and almost certainly specific to a given environment/input/etc. A muxdump would definitely help, especially if you can first reproduce the problem using that stream as input (-r) to tvheadend.

Adam

#6

Updated by Eric Valette over 7 years ago

Adam Sutton wrote:

That machine has 2G (with ZERO swap, and a reasonable chunk dedicated to RAM storage), but that's kinda missing my point. What I was implying is that the memory usage was static, about 3% constantly, a memory leak of the proportions you're talking about would be bringing my server down all the time.

Well that's something new for me also. I used to record in TS for various testing purpose fine even with 3.3 until last night test. I will rebuild the last git and see if valgrind point to something (if I manage to run it correctly).

How to produce the full mux dump (I know there is a checkbox in the GUI but do not remember how to trigger it).

tvheadend -r populates the input but how do you program the output (trigger the recording on this input). I searched last time and missed the info.

#7

Updated by Eric Valette over 7 years ago

Got a new one when trying to record as TS yesterday. I'm currently in coverity to see if static analysis finds something. So far only minor leak and non related to TS only...

[ 9866.155871] tvheadend invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
[ 9866.155881] tvheadend cpuset=/ mems_allowed=0
[ 9866.155887] Pid: 11129, comm: tvheadend Tainted: P O 3.7.6 #87
[ 9866.155891] Call Trace:
[ 9866.155905] [<ffffffff8144e657>] ? dump_header.isra.9+0x6a/0x1d6
[ 9866.155914] [<ffffffff8108265c>] ? shrink_slab+0x18b/0x19d
[ 9866.155921] [<ffffffff8107aadb>] ? oom_kill_process+0x69/0x30c
[ 9866.155927] [<ffffffff8108410b>] ? do_try_to_free_pages+0x28e/0x30c
[ 9866.155934] [<ffffffff8107b0c8>] ? out_of_memory+0x1e8/0x247
[ 9866.155940] [<ffffffff8107dc91>] ? __alloc_pages_nodemask+0x5bb/0x6d0
[ 9866.155948] [<ffffffff8107a159>] ? filemap_fault+0x24e/0x36d
[ 9866.155955] [<ffffffff8108cf8b>] ? __do_fault+0xa1/0x3e9
[ 9866.155962] [<ffffffff8108f497>] ? handle_pte_fault+0x28d/0x610
[ 9866.155969] [<ffffffff81090713>] ? handle_mm_fault+0x4e/0x191
[ 9866.155977] [<ffffffff8101f69b>] ? __do_page_fault+0x373/0x3cb
[ 9866.155986] [<ffffffff814549a2>] ? page_fault+0x22/0x30
[ 9866.155989] Mem-Info:
[ 9866.155993] DMA per-cpu:
[ 9866.155997] CPU 0: hi: 0, btch: 1 usd: 0
[ 9866.156001] CPU 1: hi: 0, btch: 1 usd: 0
[ 9866.156044] CPU 2: hi: 0, btch: 1 usd: 0
[ 9866.156049] CPU 3: hi: 0, btch: 1 usd: 0
[ 9866.156052] DMA32 per-cpu:
[ 9866.156057] CPU 0: hi: 186, btch: 31 usd: 30
[ 9866.156060] CPU 1: hi: 186, btch: 31 usd: 5
[ 9866.156064] CPU 2: hi: 186, btch: 31 usd: 90
[ 9866.156068] CPU 3: hi: 186, btch: 31 usd: 0
[ 9866.156080] active_anon:272012 inactive_anon:91474 isolated_anon:0
[ 9866.156080] active_file:91 inactive_file:154 isolated_file:0
[ 9866.156080] unevictable:0 dirty:0 writeback:0 unstable:0
[ 9866.156080] free:2725 slab_reclaimable:3274 slab_unreclaimable:3169
[ 9866.156080] mapped:567 shmem:554 pagetables:2663 bounce:0
[ 9866.156080] free_cma:0
[ 9866.156106] DMA free:6024kB min:48kB low:60kB high:72kB active_anon:4760kB inactive_anon:4768kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15648kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:156kB slab_unreclaimable:52kB kernel_stack:0kB pagetables:80kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[ 9866.156111] lowmem_reserve[]: 0 1495 1495 1495
[ 9866.156130] DMA32 free:4876kB min:4920kB low:6148kB high:7380kB active_anon:1083288kB inactive_anon:361128kB active_file:364kB inactive_file:616kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:1531780kB mlocked:0kB dirty:0kB writeback:0kB mapped:2276kB shmem:2216kB slab_reclaimable:12940kB slab_unreclaimable:12624kB kernel_stack:1656kB pagetables:10572kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:598 all_unreclaimable? yes
[ 9866.156136] lowmem_reserve[]: 0 0 0 0
[ 9866.156143] DMA: 7*4kB 6*8kB 10*16kB 3*32kB 3*64kB 1*128kB 1*256kB 2*512kB 2*1024kB 1*2048kB 0*4096kB = 6028kB
[ 9866.156161] DMA32: 207*4kB 0*8kB 1*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 4876kB
[ 9866.156181] 37263 total pagecache pages
[ 9866.156185] 36432 pages in swap cache
[ 9866.156191] Swap cache stats: add 552909, delete 516477, find 19178/25376
[ 9866.156195] Free swap = 0kB
[ 9866.156198] Total swap = 1952756kB
[ 9866.167833] 393104 pages RAM
[ 9866.167840] 9195 pages reserved
[ 9866.167843] 263300 pages shared
[ 9866.167846] 379733 pages non-shared
[ 9866.167849] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
[ 9866.167849] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
[ 9866.167876] [ 1070] 0 1070 6029 1 17 206 -1000 udevd
[ 9866.167883] [ 1134] 0 1134 5995 1 16 194 -1000 udevd
[ 9866.167890] [ 1135] 0 1135 5995 1 16 194 -1000 udevd
[ 9866.167900] [ 2438] 0 2438 4779 10 15 51 0 rpcbind
[ 9866.167906] [ 2473] 104 2473 5872 1 17 113 0 rpc.statd
[ 9866.167913] [ 2489] 0 2489 6359 0 17 56 0 rpc.idmapd
[ 9866.167920] [ 2857] 118 2857 5484 0 16 150 0 privoxy
[ 9866.167927] [ 2878] 0 2878 4584 7 12 78 0 oscam
[ 9866.167934] [ 2880] 0 2880 105644 65 24 150 0 oscam
[ 9866.167940] [ 2899] 0 2899 65748 0 30 150 0 rsyslogd
[ 9866.167947] [ 2973] 123 2973 4350 21 11 33 0 dirmngr
[ 9866.167954] [ 2995] 114 2995 31188 0 62 1203 0 timidity
[ 9866.167961] [ 3055] 116 3055 1378052 323096 1693 472888 0 tvheadend
[ 9866.167968] [ 3089] 0 3089 4204 5 12 36 0 atd
[ 9866.167974] [ 3111] 0 3111 1097 0 8 80 0 acpid
[ 9866.167981] [ 3208] 0 3208 17399 0 37 197 0 cupsd
[ 9866.167988] [ 3214] 0 3214 5731 22 15 41 0 cron
[ 9866.167994] [ 3256] 105 3256 8135 55 20 68 0 dbus-daemon
[ 9866.168002] [ 3636] 0 3636 21651 0 13 93 0 pcscd
[ 9866.168031] [ 3664] 0 3664 6704 0 18 54 0 kdm
[ 9866.168040] [ 3697] 0 3697 35857 667 72 4640 0 Xorg
[ 9866.168049] [ 3974] 120 3974 12888 2135 29 1439 0 tor
[ 9866.168057] [ 4075] 0 4075 41104 405 49 1709 0 wicd
[ 9866.168066] [ 4121] 0 4121 19558 262 42 1318 0 wicd-monitor
[ 9866.168074] [ 4139] 0 4139 10391 1 26 107 0 kdm
[ 9866.168082] [ 4144] 117 4144 95383 732 145 1208 0 kdm_greet
[ 9866.168090] [ 4160] 0 4160 4118 1 13 40 0 getty
[ 9866.168098] [ 4161] 0 4161 4118 1 12 42 0 getty
[ 9866.168106] [ 4162] 0 4162 4118 1 13 41 0 getty
[ 9866.168114] [ 4163] 0 4163 4118 1 13 41 0 getty
[ 9866.168122] [ 4164] 0 4164 4118 1 13 41 0 getty
[ 9866.168130] [ 4165] 0 4165 4118 1 14 40 0 getty
[ 9866.168139] [ 4179] 0 4179 2528 52 7 522 -1000 dhclient
[ 9866.168147] [ 4246] 0 4246 13075 1 28 152 -1000 sshd
[ 9866.168156] [ 7555] 0 7555 523429 0 59 370 0 console-kit-dae
[ 9866.168165] [ 7625] 124 7625 94339 0 48 667 0 polkitd
[ 9866.168171] Out of memory: Kill process 3055 (tvheadend) score 914 or sacrifice child
[ 9866.168185] Killed process 3055 (tvheadend) total-vm:5512208kB, anon-rss:1292384kB, file-rss:0kB

#8

Updated by Eric Valette over 7 years ago

Valgrind reports:

8242 207,043,655 (204,901,890 direct, 2,141,765 indirect) bytes in 9,516 blocks are definitely lost in loss record
442 of 442
8242 at 0x4C2C47E: realloc (vg_replace_malloc.c:662)
8242 by 0x5D8C551: ??? (in /usr/lib/x86_64-linux-gnu/libavcodec.so.54.59.100)
8242 by 0x45EF8C: lav_muxer_write_pkt (muxer_libav.c:372)
8242 by 0x4386E7: dvr_thread (dvr_rec.c:456)
8242 by 0x75F5E0D: start_thread (pthread_create.c:311)
8242 by 0x7BF094C: clone (clone.S:113)
#9

Updated by Eric Valette over 7 years ago

Eric Valette - wrote:
av_bitstream_filter_filter(lm->lm_h264_filter,
st->codec,
NULL,
&packet.data,
&packet.size,
pktbuf_ptr(pkt->pkt_payload),
pktbuf_len(pkt->pkt_payload),
pkt->pkt_frametype < PKT_P_FRAME);

But
1) packet.data is never freed
2) if as code suggest its a real realloc then the pktbuf_ptr(pkt->pkt_payload) location is already freed!!!

#10

Updated by Eric Valette over 7 years ago

Sample code of how to use av_bitstream_filter_filter:

while (bsfc) {
AVPacket new_pkt = *pkt;
int a = av_bitstream_filter_filter(bsfc, avctx, NULL,
&new_pkt.data, &new_pkt.size,
pkt->data, pkt->size,
pkt->flags & AV_PKT_FLAG_KEY);
if(a == 0 && new_pkt.data != pkt->data && new_pkt.destruct) {
uint8_t *t = av_malloc(new_pkt.size + FF_INPUT_BUFFER_PADDING_SIZE); //the new should be a subset of the old so cannot overflow
if(t) {
memcpy(t, new_pkt.data, new_pkt.size);
memset(t + new_pkt.size, 0, FF_INPUT_BUFFER_PADDING_SIZE);
new_pkt.data = t;
a = 1;
} else
a = AVERROR;
}
if (a > 0) {
av_free_packet(pkt);
new_pkt.destruct = av_destruct_packet;
} else if (a < 0) {
av_log(NULL, AV_LOG_ERROR, "Failed to open bitstream filter %s for stream %d with codec %s",
bsfc->filter->name, pkt->stream_index,
avctx->codec ? avctx->codec->name : "copy");
print_error("", a);
if (exit_on_error)
exit_program(1);
}
*pkt = new_pkt;

bsfc = bsfc->next;
}
#11

Updated by Eric Valette over 7 years ago

Removing libav useage (defaulting to no in config) makes the problem vanish. So until it is fixed I suggest to revert the default to no instead of auto...

#12

Updated by Adam Sutton over 7 years ago

  • Status changed from Need feedback to Accepted
  • Affected Versions 3.4 added

Eric,

I'm about to start thinking about branching for 3.4, I agree that it would make sense to make libav usage default=off for 3.4, though I think I'll leave it on in master.

But I'll try and discuss with John.

Adam

#13

Updated by Eric Valette over 7 years ago

Adam Sutton wrote:

Eric,

I'm about to start thinking about branching for 3.4, I agree that it would make sense to make libav usage default=off for 3.4, though I think I'll leave it on in master.

But I'll try and discuss with John.

Adam,

You're the boss but the code has 0 chance to work as it is with recent libav (1.0.x, 1.1.x):
the return value >0 indicates if the memory has been reallocated but then you should not suppose original passed pointer shall be freed again. 0 indicates the memory is still valid but the new buffer value pointer may be different than the original passed one. Hunting internet I found a lot of messages about memory leaks using this function. If you want the memory to be freed by call to av_interleaved_write_frame, you should set new_pkt.destruct = av_destruct_packet; as the ffmpeg.c code does.

So I'm not sure you want people using TS crash their recording as default behavior ;-) As far as I'm concerned I fixed the two bugs that were really annoying me (see mkv bug). As time permits I will provide some fixes for code flagged by coverity but I'm rather busy at work and the tool is only available there.

#14

Updated by Adam Sutton over 7 years ago

Indeed I do not want that and that is why I have accepted your suggestion for the upcoming release. However master is by its definition development/experimental code (i.e. use at your own risk) and so I'd prefer we leave this in (bug and all) and John can fix it as time permits.

If we were not about to make a new release I might think differently. But in the coming weeks other experimental stuff will be added to master that will make it pretty unstable again for a while anyway, so I'm not too concerned at this stage.

But hopefully your investigations will help John to fix it more quickly, so thank you.

Adam

#15

Updated by John Törnblom over 7 years ago

  • Status changed from Accepted to Need feedback

fix commited to master, pls test

#16

Updated by Eric Valette over 7 years ago

John Törnblom wrote:

fix commited to master, pls test

Since you do no test the result of av_bitstream_filter_filter, the code cannot be right (ptr may change without being reallocated). Again read ffmpeg.c code.

#17

Updated by John Törnblom over 7 years ago

well, the memory leak is fixed for me at least. the reallocation you suggest prevent overflow from happening but looking at avconv.c from ffmpeg, the reallocation is not done at all. But perhaps the buffer is big enough on so it might be okey. In any case, the overflow issues might affect other libav calls and is not specific to the use of filters.

#18

Updated by Eric Valette over 7 years ago

John Törnblom wrote:

well, the memory leak is fixed for me at least. the reallocation you suggest prevent overflow from happening but looking at avconv.c from ffmpeg, the reallocation is not done at all. But perhaps the buffer is big enough on so it might be okey. In any case, the overflow issues might affect other libav calls and is not specific to the use of filters.

1) the function may fail and can produce negative return. Unchecked.
2) > 0 or equal to 0 tells if reallocation happens not the pointer chnaging value.

What version of ffmpeg are you looking at?

#20

Updated by Eric Valette over 7 years ago

John Törnblom wrote:

http://www.ffmpeg.org/doxygen/trunk/avconv_8c-source.html

look at line 943 in this file.

#21

Updated by Eric Valette over 7 years ago

Eric Valette wrote:

John Törnblom wrote:

http://www.ffmpeg.org/doxygen/trunk/avconv_8c-source.html

look at line 943 in this file.

or line 579 of http://www.ffmpeg.org/doxygen/trunk/ffmpeg_8c_source.html

#22

Updated by Adam Sutton over 7 years ago

  • Status changed from Need feedback to Fixed
  • Target version set to 3.4

The reported problem is now fixed, I'll leave you 2 to argue about fixing the other problems with the code possibly being non-compliant etc...

Adam

Also available in: Atom PDF