Implement autocrawl mechanisms
- Periodically crawl based on CR definition
- Plan crawls at night
- Allow exceptions based on API URL (repository)
(from redmine: issue id 896, created on 2018-01-22tgradl, closed on 2018-02-15)
- Relations:
- duplicates #895
- duplicates #612