98quinnk
-
Content Count
11 -
Joined
-
Last visited
Posts posted by 98quinnk
-
-
I noticed that this happens when browser.exe "in memory" size reaches a bit more than 1GB.
However, are "members" that you already scraped still displayed on the site/contained in its HTML?
If so you should remove them with "change attribute" command and that way reduce the HTML (size) that needs to be rendered by the UBot browser. This should reduce browser.exe RAM usage and prevent browser from crashing.
They are not contained with in the html, its a java pop windows. so removing the members after scrape wont help.
-
On the site we are scraping members to message.
When we scrape members its a javascript popup on there website shows 12 users, and a loadmore button at the bottom.
we scrape the 12 the program presses load more and it shows the next 12 to be scrapped.
Gets to about 800 scrapped and then freezes up as it sky rockets the memory usage of browser.exe.
But if u close that window and reopen it , it shows the first 12 so all is lost.
-
Yes its the browser.exe getting too large, but if we close the browser and open it and list starts back at 12 and then it has to load more to scrape more again.
-
Can you post some code? That always helps.
This is the code, im working along side of nick.
define Scrape Users {
click(<class="user-list-link subs ud-popup">, "Left Click", "No")
set(#start, $true, "Global")
loop while($exists($element offset(<class="btn">, 1))) {
then {
click($element offset(<class="btn">, 1), "Left Click", "No")
wait for browser event("Everything Loaded", "")
wait(5)
add list to list(%urls, $scrape attribute(<href=w"https://www.*.com/message/?action=send-new&userId=*">, "href"), "Delete", "Global")
wait(20)
else {
}
}
}
}
-
Its not the list,
Whats happening is the following.
I scrape users to message.
But the the window that comes up is javascript it displays 12 users so I scrape them, then you have to press load more to display another 12 so it scrapes them now 24 are displaying, as the program continues to do this when It gets up to 700 scrapped it lags really bad, browsers.exe it using up all memory on the computer. If I where to close the window it goes back to just showing 12 users. And there can be like over 5000 users in the list I need to scrape.
Any help please its the only thing standing in my way for completion of my program, everything else works fine.
-
It scraped 900 urls and then its gone all laggy i dont understand why this is happening
-
Need some help.
Program scrapes urls and then it lags down and crashes.
define Scrape Users {
click(<class="user-list-link subs ud-popup">, "Left Click", "No")
set(#start, $true, "Global")
loop while($exists($element offset(<class="btn">, 1))) {
then {
click($element offset(<class="btn">, 1), "Left Click", "No")
wait for browser event("Everything Loaded", "")
wait(5)
add list to list(%urls, $scrape attribute(<href=w"https://www.*.com/message/?action=send-new&userId=*">, "href"), "Delete", "Global")
wait(20)
else {
}
}
}
}Is there away i can stop this loop, ive tried putting a set at the begining, and a set on a button which makes it false but it doesnt stop the loop.
Kind Regards
Kevin
-
I sorted it a few days ago, thank you
-
Not sure what your code looks like but you could try a replace before adding it to the box. Just replace the space with a comma.
This is my code for the Ui List box.
ui list box("List Of Urls To Message", %urls, #savefile)
the %urls has all the stored scraped urls in it.
-
Hi
Im trying to display the url list i scrape into a ui list box.
But everytime i do it , It displays like example.com example.com example it goes across.
what i want is it to go down,
example.com
example.com
example.com
Kind regards
Kevin
Help with browser crashing I think
in Scripting
Posted
How could I configure it,
So that it opens a new browsers scrapes 12 loads more. closes
Then goes back in?
That might keep the browser.exe down in memory usage.
Thanks
Kevin