Ahh, lower is better, thanks that helped me sort it.
As it happens 5 and 10 didn't work for me so I started grepping around the code and it looks like it's the type of connection that plays a part.
tda_extrapriority + tda->tda_hostconnection * 10
where for USB tda_hostconnection is 1 or 2 and for PCI it's 3.
I just set 5 (PCI) and 50 (USB) instead of 5 and 10 and it worked.
I guess 0 and something should also work but that's untested :-)