That depends what the definition of the word “should” is. 
Yes, should work.
Best chance is to start with a minimal installation without GUI or XFCE if it must be one and then installing a meta package (cli or xfce).
I wrote that because I think it’s a bad use of development time to support individual geeky exercises if something better (Whonix-Host) can be created instead.