I don’t think there is a way to do that through some Gnome setting, however you can change the wakeup ACPI events the system responds to. You can find them in /proc/acpi/wakeup like so (example example from my system, the | grep enabled filters only those events that are currently enabled):
ce@caprica ~ % cat /proc/acpi/wakeup | grep enabled
PS2K S4 *enabled pnp:00:06
PXSX S4 *enabled pci:0000:04:00.0
PXSX S4 *enabled pci:0000:05:00.0
EHC1 S4 *enabled pci:0000:00:1d.0
EHC2 S4 *enabled pci:0000:00:1a.0
PWRB S4 *enabled platform:PNP0C0C:00
EHC* and XHC include your USB controllers, so disabling the one that has the mouse attached would make the computer events from it. There might also be USB* devices enabled, same thing.
You can toggle each of these between disabled and enabled using e.g.
ce@caprica ~ % echo EHC1 > /proc/acpi/wakeup
This won’t persist past the next reboot, but you can use that to figure out which of the events to disable. If it works as expected you can make the change permanent by e.g. creating an appropriate systemd tmpfile.
I think the ACPI event that caused the wakeup is also given in the system log, so you can probably find the correct event to disable that way as well.