I'm trying to find a CF card (of any capacity more than 4MB, because that's all that will be used) that has either a built in USB cable, or a USB port in it.

It's for my datalogger, and rather than having to take parts of the car to pieces to get at it I'd much rather just plug the laptop in directly. This will also mean I can waterproof the datalogger as I'll never have to remove the card...

Does such a thing exist? I've tried and failed to find one.

i highly doubt it... cf is essentially an ide interface which is complex enough to fit on a card that size
have you tried looking for something that serves as a sort of cf riser+usb adapter?