So that means I only have to remove KDE specific packages like Dolphin and replace them with Xfce based ones?
That's actually not simple. See:
And I don't think packages like dolphin eat any RAM if they are not started.
And also what can explain this extra RAM that is being used (more than 100Mo) compared with other AppVMs?
I have no such issue. A debian-8 based AppVM as well as anon-whonix both
use approximately 500 MB RAM as per Qubes VM Manager (QVMM).
Maybe use ksysguard, sort by memory / or shared memory. Set the column