stanf 43 Posted January 6, 2015 Report Share Posted January 6, 2015 hope u feeling better Quote Link to post Share on other sites
HelloInsomnia 1103 Posted January 7, 2015 Author Report Share Posted January 7, 2015 hope u feeling better Thanks, getting better ever day. I'm going to be doing more emails today/PM's and hopefully get started on the next update. Quote Link to post Share on other sites
michaelkonig 0 Posted January 9, 2015 Report Share Posted January 9, 2015 HI CAN THIS PROGRAM SCRAPE PHP FILES TOO? Quote Link to post Share on other sites
michaelkonig 0 Posted January 9, 2015 Report Share Posted January 9, 2015 HI INSOMNIA CAN THIS PROGRAM SCRAPE PHP FILES TOO AND SAVE IT? Quote Link to post Share on other sites
HelloInsomnia 1103 Posted January 9, 2015 Author Report Share Posted January 9, 2015 It can scrape PHP files that are served up to the user. But it can't scrape the PHP source itself. What are you trying to do? Quote Link to post Share on other sites
michaelkonig 0 Posted January 10, 2015 Report Share Posted January 10, 2015 I am trying to scrape php files from any website because I am studying php file source codes as examples thats it no for other purposes. Quote Link to post Share on other sites
HelloInsomnia 1103 Posted January 10, 2015 Author Report Share Posted January 10, 2015 I am trying to scrape php files from any website because I am studying php file source codes as examples thats it no for other purposes. Well you can't see the source code of PHP files because they are served server side. Meaning, the end user can only see the output and not the source code. I suggest you take a look at places like Github where code is shared for free you can study PHP files there. Quote Link to post Share on other sites
michaelkonig 0 Posted January 11, 2015 Report Share Posted January 11, 2015 insomnia thank u very much Quote Link to post Share on other sites
Pete_UK 9 Posted January 12, 2015 Report Share Posted January 12, 2015 Nick, did you manage to do a new video covering the new version of the code? Quote Link to post Share on other sites
HelloInsomnia 1103 Posted January 19, 2015 Author Report Share Posted January 19, 2015 Nick, did you manage to do a new video covering the new version of the code? Hey Pete, after the next update the new source code video will come out since the code will change again. Quote Link to post Share on other sites
botsimmer 16 Posted January 20, 2015 Report Share Posted January 20, 2015 Hi HelloInsomnia, I sent you a PM. Thanks Quote Link to post Share on other sites
jayemes 1 Posted January 26, 2015 Report Share Posted January 26, 2015 I compiled the code as is and when I try to run it with the exact settings in OP video, nothing happens. Are there settings I need to initialize that I did not see? Thanks Quote Link to post Share on other sites
HelloInsomnia 1103 Posted January 29, 2015 Author Report Share Posted January 29, 2015 I compiled the code as is and when I try to run it with the exact settings in OP video, nothing happens. Are there settings I need to initialize that I did not see? Thanks I'll send you a PM and we can try to figure it out! Quote Link to post Share on other sites
HelloInsomnia 1103 Posted February 3, 2015 Author Report Share Posted February 3, 2015 Well it took forever and a day but I think the memory climbing issue is finally sorted: http://imgur.com/a/GCnkm It was going between 90 and 120 mb during that test run, it just happened to increment for each photo but after each one it was down at 90 again at one point or more. I am going to release 1.3 soon after one or two kinks are worked out. Because I know this update has taken forever but I've probably made 15 different versions of this and spent days of testing time. After that version 2 will come out and it will make the code a lot cleaner and hopefully serve as a very strong foundation that can be built upon. Hopefully it will be able to crawl basically indefinitely. 1 Quote Link to post Share on other sites
Pete_UK 9 Posted February 3, 2015 Report Share Posted February 3, 2015 Great news Nick. I do find it a little difficult at the moment to really pinpoint what code I should change for my own use.Can you record a video or simple instructions to show an example of how someone would take the code and change it for their own use case. Just something simple like:Here is the base code...Now let change it for this scenario ...Crawl site ABC looking for pages containing ZYX.Save the URLS and scrape ZYX from the page. Thanks,Pete 1 Quote Link to post Share on other sites
HelloInsomnia 1103 Posted February 3, 2015 Author Report Share Posted February 3, 2015 Update 1.3 To download the update login at: http://imautobots.com/wp-login.php Then go back to the homepage and click on "Purchase History" Also, there is a video showing how to make changes in 1.3 to customize what you scrape. V 2.0 coming soon 1 Quote Link to post Share on other sites
HelloInsomnia 1103 Posted February 5, 2015 Author Report Share Posted February 5, 2015 Update 1.4 - Lots of bug fixes- 6 hour stress test ran with no errors and decent memory management To download the update login at: http://imautobots.com/wp-login.php Then go back to the homepage and click on "Purchase History" 1 Quote Link to post Share on other sites
botsimmer 16 Posted February 5, 2015 Report Share Posted February 5, 2015 Thanks HelloInsomnia. Top man. 1 Quote Link to post Share on other sites
HelloInsomnia 1103 Posted February 5, 2015 Author Report Share Posted February 5, 2015 Fixed a bug when running multiple links, cleaned up a few unnecessary commands. I'm going to wait to make sure there are no more bug reports before updating though just in case. 1 Quote Link to post Share on other sites
Chris M 55 Posted February 6, 2015 Report Share Posted February 6, 2015 You found a bug in 1.4 Nick? Funny because it's running better than ever on my machine. 1 Quote Link to post Share on other sites
HelloInsomnia 1103 Posted February 6, 2015 Author Report Share Posted February 6, 2015 You found a bug in 1.4 Nick? Funny because it's running better than ever on my machine. Yes when using multiple urls, since I had changed so much and was only testing on the big sites I didn't notice when it switches urls it won't work! I'll get 1.5 out today unless there is another major bug found - I just don't want to update too fast and have everybody downloading updates twice a day. Quote Link to post Share on other sites
HelloInsomnia 1103 Posted February 7, 2015 Author Report Share Posted February 7, 2015 Updated to 1.5 - Fixed bug where you couldn't run mutiple urls- Cleaned up a few unnecessary nodes To download the update login at: http://imautobots.com/wp-login.php Then go back to the homepage and click on "Purchase History" Quote Link to post Share on other sites
botsimmer 16 Posted February 8, 2015 Report Share Posted February 8, 2015 Thanks HelloInsomnia, no errors identified.Thanks. Quote Link to post Share on other sites
HelloInsomnia 1103 Posted February 20, 2015 Author Report Share Posted February 20, 2015 ETA for V2 is looking more like mid next week now, trying to get it done ASAP but also get it done the right way. The price for this after V2 comes out is going to rise dramatically by the way. And of course the update to V2 for current users is free. Quote Link to post Share on other sites
Pete_UK 9 Posted February 21, 2015 Report Share Posted February 21, 2015 look forward to it. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.