Purchase loan activity has dropped in five of the last six weeks as some buyers remain wary, according to the Mortgage Bankers Association.