I took out a Pandora battery from my PSP after doing some stuff, while the USB cable was still plugged in.
After I pulled out the battery the PSP ran for like 5 seconds! It was in the Recovery menu on a Slim. The "Always USB Charge" setting was enabled.
I tried again, but It hasn't happened since.
EDIT: And I mean the standard data cable. Not any of the USB to power adapters.