JasonWest 0 Posted May 5, 2016 Report Share Posted May 5, 2016 I'm scrapping information off of a website and then adding these information into a table. While the bot is running, a message pop up once about every 15 minutes stating that CefSharp.BrowserSubprocess.exe has crashed. At the same time the scraped information stop being added into the table. Also, before the CefSharp.BrowserSubprocess.exe crashed, the bot is running really slow.I search everywhere, but can't seem to find the solution to this problem. Any suggestions would be much appreciated. Quote Link to post Share on other sites
pash 504 Posted May 6, 2016 Report Share Posted May 6, 2016 what is your code? Quote Link to post Share on other sites
JasonWest 0 Posted May 6, 2016 Author Report Share Posted May 6, 2016 Here it is, thank you in advance! add list to list(%scraped urls,$list from file("C:\\Users\\JW\\Documents\\Scraped\\URL 10-99.txt"),"Don\'t Delete","Global")clear list(%scraped authors name)clear list(%scraped authors link)clear table(&author info)set list position(%scraped urls,0)loop(9000) { navigate($next list item(%scraped urls),"Wait") add item to list(%scraped authors name,$scrape attribute(<itemprop="author">,"innertext"),"Delete","Global") add item to list(%scraped authors link,$scrape attribute(<itemprop="author">,"fullhref"),"Delete","Global") add list to table as column(&author info,0,0,%scraped authors name) add list to table as column(&author info,0,1,%scraped authors link) save to file("C:\\Users\\JW\\Documents\\Scraped\\Author Scraped.txt",&author info)} Quote Link to post Share on other sites
deliter 203 Posted May 6, 2016 Report Share Posted May 6, 2016 hard to say..code looks fine first check if the chrome 21 browser works better,in settings,add in a wait for browser event,everything loaded,after the navigate,see if that fixes it or try this,this just resets the browser every 20 pages,will only work on chrome 39,it will only hold the script up a few seconds every 20 pages add list to list(%scraped urls,$list from file("C:\\Users\\JW\\Documents\\Scraped\\URL 10-99.txt"),"Don\'t Delete","Global") clear list(%scraped authors name) clear list(%scraped authors link) clear table(&author info) set list position(%scraped urls,0) set(#position,0,"Global") loop(9000) { if($comparison($division remainder(#position,20),"= Equals",0)) { then { reset browser } } navigate($list item(%scraped urls,#position),"Wait") wait for browser event("Everything Loaded","") add item to list(%scraped authors name,$scrape attribute(<itemprop="author">,"innertext"),"Delete","Global") add item to list(%scraped authors link,$scrape attribute(<itemprop="author">,"fullhref"),"Delete","Global") add list to table as column(&author info,0,0,%scraped authors name) add list to table as column(&author info,0,1,%scraped authors link) save to file("C:\\Users\\JW\\Documents\\Scraped\\Author Scraped.txt",&author info) increment(#position) } Quote Link to post Share on other sites
JasonWest 0 Posted May 6, 2016 Author Report Share Posted May 6, 2016 hard to say..code looks fine first check if the chrome 21 browser works better,in settings,add in a wait for browser event,everything loaded,after the navigate,see if that fixes it or try this,this just resets the browser every 20 pages,will only work on chrome 39,it will only hold the script up a few seconds every 20 pages add list to list(%scraped urls,$list from file("C:\\Users\\JW\\Documents\\Scraped\\URL 10-99.txt"),"Don\'t Delete","Global") clear list(%scraped authors name) clear list(%scraped authors link) clear table(&author info) set list position(%scraped urls,0) set(#position,0,"Global") loop(9000) { if($comparison($division remainder(#position,20),"= Equals",0)) { then { reset browser } } navigate($list item(%scraped urls,#position),"Wait") wait for browser event("Everything Loaded","") add item to list(%scraped authors name,$scrape attribute(<itemprop="author">,"innertext"),"Delete","Global") add item to list(%scraped authors link,$scrape attribute(<itemprop="author">,"fullhref"),"Delete","Global") add list to table as column(&author info,0,0,%scraped authors name) add list to table as column(&author info,0,1,%scraped authors link) save to file("C:\\Users\\JW\\Documents\\Scraped\\Author Scraped.txt",&author info) increment(#position) } Thank you! I changed the setting to Chrome 21 and it is running smoothly so far! Quote Link to post Share on other sites
pash 504 Posted May 7, 2016 Report Share Posted May 7, 2016 Not tested because no URL. add list to list(%scraped urls,$list from file("C:\\Users\\JW\\Documents\\Scraped\\URL 10-99.txt"),"Don\'t Delete","Global") clear list(%scraped authors name) clear list(%scraped authors link) clear table(&author info) set list position(%scraped urls,0) set(#BackUpTitle,"","Global") loop(9000) { ScrapeProcess($next list item(%scraped urls)) SaveToFile() } define ScrapeProcess(#_url) { navigate(#_url,"Wait") wait for browser event("Everything Loaded","") if($comparison(#BackUpTitle,"= Equals","")) { then { set(#BackUpTitle,$title,"Global") } else if($comparison($title,"= Equals",#BackUpTitle)) { reset browser navigate(#_url,"Wait") wait for browser event("Everything Loaded","") } else { set(#BackUpTitle,$title,"Global") } } add item to list(%scraped authors name,$scrape attribute(<itemprop="author">,"innertext"),"Delete","Global") add item to list(%scraped authors link,$scrape attribute(<itemprop="author">,"fullhref"),"Delete","Global") } define SaveToFile { add list to table as column(&author info,0,0,%scraped authors name) add list to table as column(&author info,0,1,%scraped authors link) append to file("C:\\Users\\JW\\Documents\\Scraped\\Author Scraped.txt","{$new line}{&author info}","End") clear list(%scraped authors name) clear list(%scraped authors link) clear table(&author info) } more to try. allow flash("No") allow images("No") allow javascript("No") allow css("No") Quote Link to post Share on other sites
jason 101 Posted May 7, 2016 Report Share Posted May 7, 2016 Open a ticket and let us know the original code, please, that is causing the error. Thanks Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.