I use ext4. But typical recording usage is quite light (max 2 or 3 simultaneous recordings), but I've tested recording all channels in a mux + then some, and I had no problems. Recently I've started using snapraid + mergerfs (I have other media files to store, too, and this simplifies storing things) with ext4 on the disks. Works for me, and adding storage (disks) should be quite easy.
I don't know what you mean with deleting taking ages - I haven't noticed such problems, deleting is (practically) instantenous even for large directory trees...
For a typical recording server / backend, I don't think you will need RAID (the writing speed should be enough on any 5400RPM HD even for simultaneous HD recordings). The main benefit of using RAID is not being bound by the space of a single disk (in this use case!). But there is a also a caveat - do plan ahead what to do if disk(s) in an array break down!
Ext4 is not the best FS for raid arrays, I must admit. Some FS do allow adding disk(s), other require re-building the whole array (and if more than redundancy number of disks break down, remember all is lost). "Fake raid", such as snapraid are more usable (IMO) for media servers!
Otherwise, for (a typical) recording server, the choice of FS does not matter that much. Things might be different, if you have >2 tuners - i.e. recording more than two muxes at a time - and several frontends perhaps looking at different recordings simultaneously... but you must have a very large household with a lot of TV addicts if that happens ;)