Implement crawl state listener
- monitor crawls and update their status on error/complete
- hold state info on queue/progressing internally
(from redmine: issue id 507, created on 2018-01-22tgradl, closed on 2018-01-24)
(from redmine: issue id 507, created on 2018-01-22tgradl, closed on 2018-01-24)