Insert the following in /etc/fstab
none /proc/bus/usb usbfs devgid=123,devmode=666 0 0
UPDATE:
To get it working on Intrepid Ibex I added this:
#Make USB Work in Sun VirtualBox none /proc/bus/usb usbfs devgid=46,devmode=664 0 0
Execute:
sudo mount -a
Now you should be okay to run Vbox and use USB devices.