I am seeing something similar on my 2 tvHeadends running T230 DVB-T2. I started monitoring swap with nodered, see the attached graph.
The series are:
blue: a pi4 running tvheadend (in docker)
red: a pi3 running tvheadend as a SAT>IP server to blue (in docker)
green: a pi3 running motioneye (in docker)
The EPG is being downloaded at 2:00 and 14:00 and you can see an increase in swap usage while its being processed at around 2:30. A similar reduction is seen while playing recorded programs with kodi on the pi4 at around 22:00 to 22:30.
Although not significant at this time swap usage accumulates over time and reboot needed about once a week.
The situation was significantly worse when running 2 tuners on one pi4.