Jump to content
UBot Underground

Looping through a dynamic URL


Recommended Posts

I'm trying to figure out how to loop through a single URL with a dynamic ID, and then scrape the data on that page, and proceed to the next record (autoincrementing by 1) . I want to define which ID to start at, and how many loops to run. The process would be:

 

1. Define the starting id (ex. #15). I have a textbox element called #startnum

2. Nav to a domain with that starting id (ex.http://www.domain.com/index.php?id=15)

3. Scrape data (I can figure that part out :)

4. Then proceed to the next record (http://www.domain.com/index.php?id=16), and scrape data on that page. It will always increment by 1

5. Complete this loop X number of times (Defined by another textbox element #loopcount). So if I put in #loopcount of 2, it would do records 15 and 16

 

Probably easily solved by more experienced folks...any help would be appreciated!

Link to post
Share on other sites

Need to make a variable for a counter.

 

$variable = 1

 

Loop (X tumber of times)

navigate http://www.domain.com/index.php?id=$variable

scrape

inc $variable

 

That will keep looping and incrementing

 

Thanks Aaron, didn't realize there was a INC/DEC feature :) BTW, tried to PM you but it says you can't receive PM's, so I emailed your Yahoo account.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...