Definitely get rid of the vines. That will help the overall health of the tree and allow more sun to hit the trunk and help with the "mold"--which is probably just harrmless moss or lichen.
Check the limbs on which mold is growing to see how sound they are. If they are starting to rot, it's time to prune them off. Otherwise, follow the basic rules of pruning a fruit tree (which should be done in early spring just as it's starting to leaf out). Cut out any dead wood. Cut out any branches that are rubbing together or are crossed. And then make any cuts to make sure the tree branches grow outward and upward and can get some light into the center of the tree, if possible. If the tree is large, this may be too big a job for one person. Consider having a quality landscape care service come out for a consultation, which they often do for free. Or call your city to see if they have an arborist on staff.