partial working and think logic may be slightly off and inefficient but returns on correct answer. also mutlti word produced a set intersection

building out more error handling for crawler, also can build for a certain number of docs mode, modified url crawling scoring