Generally, using any ICMP opens users to the possibility of a sidechannel - CPU has a very visible effect on ICMP packet latency.
Second. If we have to make decisions based on security vs performance tradeoff we will choose security almost every time if there isn’t a crippling effect on usability.
We disable connection tracking by default so there would be added attack surface.