Eat em raw. Maybe peel them if the fluff bothers you. Never had any other way, never felt inclined to try.

To avoid pests, you can find small fabric bags to dress over the fruits while they mature. That way you can eat tree ripened fruits, much more flavorful.