I remember when I was a kid, about 12 years old. My brother got a commodore Amiga. When he was out, I discarded the 'system' icon into the trash can. It never worked after that, and after much head scratching, he took it back to the shop, where they had to reinstall the system back onto it. Ho de ho. He wasn't impressed.
Anyways, as for the recycle bin... I have never tried to delete it. I have successfully managed to delete the 'My Documents' icon off the desktop, on a PC at college. My tutor was not impressed with this either. Like the Recycle Bin, it is a system folder, so I'm not sure if it is possible to delete it or not. I wouldn't like to try it.
Why don't you give it a go and get back to us with the results?