Reimplement automatic online crawl capabilities

Edited by Gradl, Tobias