That’s because I use CPU pinning by default to isolate VM CPUs from eachother to prevent side-channel attacks on crypto if there is a vuln in OpenSSL. So even if you add more cores you are essentially overloading a single real one with this setting until it is removed.
To get rid of this behavior remove the
cpuset='1'
in the vcpu tag.