This is more to like document my thoughts openly for this project. I feel like I should give the crawler a name, given that Google, Bing, and other search engine web crawlers have names. We’ll see I guess…