Zoom out until you see your cluster of orange dots are closer together (the ones you want to delete!)
Then, click and delete quickly, like spam alternate click both. You will clear them quicker. You might miss a few, but no harm done. I find this quicker, but you can also do Ctrl + Left click and select them one by one, then delete as a 30 object piece.
And yes, an easier way to delete multiple objects would be nice. Maybe something like placing your cursor on an area and left click dragging to make a dotted box around, and everything inside that box will be deleted? Something like windows desktop and the left click drag box.