basic the question
It would seem that virt-manager should be run on the host since it manages the different VMs using libvirt.
virt-manager is only a a front-end which does not impact the performance of the VMs per se. It can even connect remotely.
Are you concerned about the performance of the graphical console?
i meant all the virtualization process, if the guest vms performance would be same if it was layered
Guest VMs can run anywhere, no matter where virt-manager runs. This depends on the “session” which you are connecting to, and that can be even remote on a different host.
Now, given the way toolbox works, I don’t expect any difference in performance between guests running within a box or outside on the same host, because the access to all resources is “direct” (if permitted), not through some additional layer.
Also, due to the way toolbox works, it might be difficult to even find out what runs where 
thanks