Reimplement automatic offline crawl capabilities