Bug: Crawling threads not terminating

After completion or error the threads remain open and fill up the thread pool

Additional information:

Threads just remain open and are reused. No change required.

(from redmine: issue id 95, created on 2018-01-22tgradl, closed on 2018-01-22)