Jump to content
UBot Underground

Anyone Have Any Work Available?


Recommended Posts

  • 2 weeks later...

If you are bored, I have been trying to get this bot to work.

It is a Exercise bot for the forum

It scrapes tables from a dropdown which contains every town or city in texas

the tables can contain more than 1 page for any give town

it obviously requires a loop which i can not get my head around.

Please let me know if you can help

Regards

Graham

 

Link to post
Share on other sites

If you are bored, I have been trying to get this bot to work.

It is a Exercise bot for the forum

It scrapes tables from a dropdown which contains every town or city in texas

the tables can contain more than 1 page for any give town

it obviously requires a loop which i can not get my head around.

Please let me know if you can help

Regards

Graham

<EDIT> : I didn't get what you wanted at first,so I am building it right now,the bot will create a HTML page with dropdowns in alphabetical order,click on a town and it will bring you to the link</EDIT>

 

I said I would make that bot (10 minute job!) here I am 2 hours later,the problem is their  HTML isn't structured from a template,every page is practically random with regards to its layout,best off to find a better site,

 

to welcome you to the forum I made a crappy bot WITH some bugs in it(some left their for kicks ha ha) so you can use it and learn from it hopefully,the regex is VERY basic,I only watched a 40 min youtube video and still only use it with that level of complexity,so it would be useful for you learning regex,but as I said,if each page is totally random(the html document) ie no HTML 5 your better off to move along and scrape off a more modern site,which would have taken two minutes,credits too, to the site it is a lovely service they have created and a ridiculously difficult one at that,so the href's go to their site as they deserve a lot of credit for building this resource

 

Let me know what you guys think,their is still one or two little bugs in it but easy to fix :) ,point of this excercise using that site is just to show how difficult and messy a job can be by not using the most modern source

 

navigate("http://texasescapes.com/TOWNS/Texas_towns_A_to_Z.htm","Wait")

wait for browser event("Everything Loaded","")

add list to list(%AZ href,$find regular expression($document text,"href=\"Texas-Towns-\\w\\.htm"),"Delete","Global")

set(#position,0,"Global")

loop($list total(%AZ href)) {

    navigate("http://texasescapes.com/TOWNS/{$replace regular expression($list item(%AZ href,#position),"href=\"","")}","Wait")

    wait for browser event("Everything Loaded","")

    set(#page scrape,$find regular expression($document text,"<a href=\"Texas-Towns-Y.htm\">Y</a> <font color(?s).+?</table.+?/table|(?s)<a href=\"Texas-Towns-Z.htm\">Z</a>.+?Texas-Towns-\\w\\.htm"),"Global")

    set(#town href,$find regular expression(#page scrape,"href=.+?htm\".+?</a|(?s)href=.+?htm\".+?</a>"),"Global")

    set(#replace,$replace regular expression(#town href,"href=\"..","http://texasescapes.com"),"Global")

    set(#replace href,$replace regular expression(#replace,"\"",""),"Global")

    set(#option start,$replace regular expression(#replace href,"http","<option value=http"),"Global")

    set(#option end,$replace regular expression(#option start,"/a","/a</option>"),"Global")

    set(#each dropdown,"<form name=\"nav\">

    <select name=\"navigation\" onChange=\"window.document.location.href=this.options[this.selectedIndex].value;\" value=\"GO\" style=\"width:100%;\">

        <option selected=\"selected\">{$replace regular expression($find regular expression($url,"\\w\\.htm"),"\\.htm","")}</option>

 {#option end}

      

    </select>

</form>","Global")

    add item to list(%each dropdown,#each dropdown,"Don\'t Delete","Global")

    increment(#position)

}

load html(%each dropdown)

save to file("texas.html",%each dropdown)

Link to post
Share on other sites
  • 2 weeks later...

Hey guys I hired deliter a few weeks ago to make an alexa scraper for me...he did a solid job and I'm definitely happy with the outcome and the price...waaaaay cheaper than what other guys wanted to charge me..

Link to post
Share on other sites

Hey guys I hired deliter a few weeks ago to make an alexa scraper for me...he did a solid job and I'm definitely happy with the outcome and the price...waaaaay cheaper than what other guys wanted to charge me..

 

my pleasure,thanks

Link to post
Share on other sites
  • 2 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...