Twitter Dominator 125 Posted March 15, 2010 Report Share Posted March 15, 2010 Need Someone to create article scraper bot for me What it has to do is by keyword, go to each top article site and find these articles, then grab the article content and place it into separate {title of article here}{keyword here}.txt files . And be able to do all articles on that keyword it finds. Even if it could use top 10 article sites would be fine. So that one could scrape a niche set of articles from all article sites to re write them. Will Pay, for this to get done. ASAP Thanks Quote Link to post Share on other sites
alcr 135 Posted March 15, 2010 Report Share Posted March 15, 2010 That would pretty much mean writing a bot for each article site since they don't really follow any standard. Quote Link to post Share on other sites
clintk 2 Posted June 26, 2019 Report Share Posted June 26, 2019 Did anyone do this? I am looking for some source code for scraping an article site to learn from so I don't mind if it is just for one article site? Quote Link to post Share on other sites
clintk 2 Posted June 26, 2019 Report Share Posted June 26, 2019 I found this which looks to be exactly what I am looking for but sadly the link is dead :-(http://network.ubotstudio.com/forum/index.php/topic/4449-ezine-article-scraper-with-ubot-source-code/?hl=aaron Quote Link to post Share on other sites
MasteRofBotS 10 Posted July 22, 2019 Report Share Posted July 22, 2019 Did anyone do this? I am looking for some source code for scraping an article site to learn from so I don't mind if it is just for one article site? Yes. I did what you can call "article scraper" and what I would recommend to you is to start and choose any site and just go with a normal, most basic, navigate and scrape type of bot for 1 specific site to understand what steps you need to make in order to go to site, grab a content, save it & repeat. It might be confusing to learn from someone else code since as someone stated above, you will need to create kinda "custom" bot for each and every site you will be getting articles from anyways, some elements will be same but they are using different ways to search and show content so the same steps on 1 site might not work correctly on other site. If you need some help or have questions, let me know and I will help you as much as I can. Feel free to PM me about that. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.