If you take the gearbox/motor assembly out of the loader completely and put batteries in, does it still do the same thing?
If not, then it could be that your gear/motor assembly isn't slotted in properly, i've had this problem too.
Disassemble the loader all the way down.
Take the gearbox assembly out, and replace it carefully making sure the four ridges (two on the left, two on the right) match up with the ridges on the inside of the shell.
Now before you reassemble the loader, put the batteries in and test the button on the back to make sure it turns the loader on and off.
If it works, then problem solved. If not, I'd wait for another reply from Ash.