Commits · 79e9c9212c65906e24e61448877115f91f949328 · vcday / eecs398-search

Mar 29, 2018
- shutdown mechanism for the crawler + indexer with atomic bool working · 9a1995c4
  jsclose authored 6 years ago
  
  9a1995c4
- modifying queue rate · 24f56373
  jsclose authored 6 years ago
  
  24f56373
- housekeeping thread to write urls in queue to disk · 70e42436
  jsclose authored 6 years ago
  
  70e42436
- working tests, trying to add a shutdown method · 7b25b094
  jsclose authored 6 years ago
  
  7b25b094
Mar 28, 2018
- refactor · a63621a6
  jsclose authored 6 years ago
  
  a63621a6
Mar 27, 2018
- same as above · 43d67dc8
  jsclose authored 7 years ago
  
  43d67dc8
Mar 21, 2018
- hacked into making queries · c2667b49
  Nicholas Yang authored 7 years ago
  
  c2667b49
- removed the on disk doc map look upstuff, and created an isolated crawler test · 3cc58b8e
  jsclose authored 7 years ago
  
  3cc58b8e
Mar 20, 2018
- Style changes · 9ea59286
  vcday authored 7 years ago
  
  9ea59286
- indexer thread is now receiving from the parser · 36fc45a2
  jsclose authored 7 years ago
  
  36fc45a2
- intergrated indexer producer consumer queue · d8cc4e0c
  jsclose authored 7 years ago
  
  d8cc4e0c
- added a kill all spiders function so that we can start to terminate a run and... · 1e92c676
  jsclose authored 7 years ago
  
  added a kill all spiders function so that we can start to terminate a run and then restart it at the same point
  1e92c676
- Created a checkstatus function for the web readers so that we dont pull from a site that is bad · 4e2d4d5e
  jsclose authored 7 years ago
  
  4e2d4d5e
- modifiying duplicate url · 8364f3db
  jsclose authored 7 years ago
  
  8364f3db
- fixed bug (multiple initialization of ssl library) · 90ac15e7
  benbergk authored 7 years ago
  
  90ac15e7
Mar 19, 2018
- converted url frontier · b181bd3f
  jsclose authored 7 years ago
  
  b181bd3f
Mar 15, 2018
- fixing crawl/parse · f3a234af
  vcday authored 7 years ago
  
  f3a234af
Feb 18, 2018
- working OpenSSL · 81d48785
  jsclose authored 7 years ago
  
  81d48785
- working on ssl · 8f7de9b7
  jsclose authored 7 years ago
  
  8f7de9b7
Feb 15, 2018
- added command line args · e8774451
  benbergk authored 7 years ago
  
  e8774451
- update · 2f874e38
  jsclose authored 7 years ago
  
  2f874e38
Feb 14, 2018
- trying with 3 spiders · 1a4a3149
  jsclose authored 7 years ago
  
  1a4a3149
- modified seed url set up, working pull from the web · 0efd4c02
  jsclose authored 7 years ago
  
  0efd4c02
- fixed linker error, made formatting changes · 2693bb0f
  benbergk authored 7 years ago
  
  2693bb0f
Feb 13, 2018
- working on url testing + docMap work · e4c9b47b
  jsclose authored 7 years ago
  
  e4c9b47b
Feb 12, 2018
- working on shared docMap and docMapLookup · 32fcdac0
  jsclose authored 7 years ago
  
  32fcdac0
- working on doc map, url obj · 1cddffb7
  jsclose authored 7 years ago
  
  1cddffb7
Feb 06, 2018
- refactoring with utils · 0ae2c4d8
  jsclose authored 7 years ago
  
  0ae2c4d8
Feb 01, 2018
- Created Crawler Statistics · 3685c6b9
  benbergk authored 7 years ago
  
  3685c6b9
- Made thread class, fixed bugs · e6bbbee1
  benbergk authored 7 years ago
  
  e6bbbee1
Jan 31, 2018
- working to compile · 80b2b69f
  jsclose authored 7 years ago
  
  80b2b69f
Jan 28, 2018
- initial spider + crawler classes · 5d939592
  jsclose authored 7 years ago
  
  5d939592
Jan 24, 2018
- added main function · d9a730a4
  benbergk authored 7 years ago
  
  d9a730a4
- added main · 86457cfe
  benbergk authored 7 years ago
  
  86457cfe
Jan 23, 2018
- added readme · 451e682b
  benbergk authored 7 years ago
  
  451e682b