Hi all, I've built my first bot which is ready to crawl just less than 1000 pages. I've dropped a "wait 3 seconds" command in the loop so as to not hammer the server, but I've not used a proxy or anything. The scrape will take around 1 hour to complete which is good with me, but I'm wondering if my crawl rate is still too high? Should I be using a list of proxies to scrape 5-10 pages per second to get the job done quicker and avoid raising red flags with the server admins? Just looking for any tips from the community on what you might call best practices. Any tips or plugins which you fi