building out more error handling for crawler, also can build for a certain number of docs mode, modified url crawling scoring
modifiying all of the parsed urls to parsed url pointers for better error handling, also implementing an abstract urlfrontier class to handle alot of the more balanced crawler work
added a kill all spiders function so that we can start to terminate a run and then restart it at the same point
