Not sure if this is the right place to do it, but I was about to ask in here but I found out how.
1. Boot PSP into Recovery mode (cold boot while holding R)
2. Go to Configuration, press X
3. Go to Skip SCE logo, press X
4. Go back, Exit.
Done! While it doesn't explicitly disable autorun, I found that when I have it disabled, my UMDs won't boot automatically. It will go straight to the XMB instead!