building out more error handling for crawler, also can build for a certain number of docs mode, modified url crawling scoring
modifiying all of the parsed urls to parsed url pointers for better error handling, also implementing an abstract urlfrontier class to handle alot of the more balanced crawler work
