Their hardware support is not as good as Linux, especially for WiFi and for embedded SoCs. Their network stacks are lacking in more advanced features like QoS that's not from the '90s (AQM, FQ, traffic shaping that accounts for the overhead your DSL or cable modem adds) and I'm not aware of any efforts to eliminate bufferbloat from NIC drivers the way BQL has for most Linux Ethernet drivers. I'm not sure how Linux compares against the BSDs for dumb packet forwarding and NAT performance, but for real-world performance the better QoS makes it no contest.
Linux is what the chipset manufacturers target, it's what the router manufacturers ship, it's what most of the academics seem to turn to when they're not using a network simulator, and Linux seems to have the most active networking developers. The only compelling argument for BSDs is that pf.conf is more approachable than the Linux tools, but BSD advocates usually don't mention that it's because pf does a lot less than tc and the other Linux tools.
FreeBSD has QoS available via PF and ALTQ. As for performance, Netflix chose FreeBSD for their CDN for the better network performance over linux.
I do take your point about hardware support on consumer routers though - most of these are based on linux so its relatively easy to get linux based *wrt installed on them
Stop thinking like QoS has a singular meaning. ALTQ provides the aforementioned '90s-era inferior QoS techniques, and it isn't even available on FreeBSD without recompiling the kernel. The dummynet module is a little more modern, and in February patches appeared implementing the CoDel and FQ-CoDel AQMs that Linux has had for four years.