What is happening when you select it is that you are viewing the contents of the folder. As it is an empty folder all you see is the heading i.e. the oblong box, with the option for how to get back to the media list (‘upper folder’).
To delete you should NOT open the folder.
Instead you highlight the folder but stop short of actually opening it. While the folder is highlighted (in the media list) press the OPT+ button on the remote (NOT the OK button which you may have been doing) and then choose ‘delete’.
If it keeps on opening the folder then press ‘OK’ to go to the ‘upper folder’ and then you will be able to use the OPT+ button after highlighting the folder you want to delete.