clicking a button that opens a new window

[graysqwrl 0]

Ok... I'm new and don't really know much about what I'm doing. So, please hang with me.

 

I'm trying to scrape real estate records within 1 mile of a certain house.

 

conveniently, there's a button in the search criteria that says "Radius Search", but when I click it within ubot nothing happens. In IE it opens a new popup window where I fill in the address select "1 mile radius" and hit the submit button.

 

I navigated specifically to the radius search window using a subwindow in ubot, but when I submit that way it doesn't manage to update the session in the main window with the new radius data.

 

I have a feeling it has something to do with javascript... because I don't know anything about javascript and so it figures that would be my problem.

 

Let me know what you need to help me out... or start tossing out ideas.

 

Thanks a bunch!

[pftg4 102]

Hi

 

why don't you leave the url so we can take a look

 

Pftg4

[meter 145]

Usually you want to scrape the URL of the window the button is popping up and then use the Nav command to nav to it.

[TommyTx 5]

Be very careful scraping real estate sites unless you are faking the true look of a browser... then scrape slowly to make it look like a visitor and not a nasty spider... like UBOT.. most real estate sites are monitored carefully for unusual activity... and you must admit scraping 100 pages around a specific address... looks shady to me... since it would be done very rapidly like in 30 seconds...

 

Not too much of a problem if you are scraping a single agents site (they just simply ban you and cannot cancel your MLS license), but if you are inside an mls with a license they will remove your membership in a blink of the eye... then you can no longer sell real estate as you no longer have access... worse yet your friend is an agent and let you access his password to help him and you just in a blink of the eye removed his livlihood... its easy to scrape and appear to be a normal user, raising no eyebrows.. if you use the right tools and that is not UBOT as it leaves a trail.

 

How do I know so damn much about this... I are a licensed agent, and have sold more than 10,000 copies of a software that monitors sites for unusual activity for banning... and I continue to add daily scrapers to the list as they come out and are easily identified.... especially if they scrape more than one page in 5 seconds...

 

Why am I telling you all this... well, I design software mostly for fun all day in every possible language..you can imagine.. and I don't mean language like Japanese, Russinan, Chinese and English... what I mean is visual basic, mysql, UBOT, C++, and whatever other language you can think of....

 

So what does this mean to you.... if you have an idea that I think is interesting... contact me on Skype at jackrabbitsec and we can discuss different methods you can use... and if its an interesting project, I might pop it out for you in a better language for free..

 

Don't get me wrong... i love UBOT, but its currently difficult to make it look like a real browser....

I use it for many things but never for attacking a site that would kill me if it got detected... and its easy to detect UBOT as not a real broweser...

 

Good luck with your project... and there are tons of nice folks that will help you, and to help you sometimes you need to post a url so the nice folks here can give it a try and solve it for you.. Problem is, that 100 of us willl be hitting that page and trying to solve and that alone will alert the owner about something funny... then when you get it working you are already banned.... we don't purposefully do that, but we are trying to solve...

 

A better way sometimes to get our help without getting you in trouble, is as follows:

1. Especially if a password is reqd to get the page..

2. Use the Page save in IE to save the entire page with the images...

3. Copy this dupe to one of your servers not connected with the original site..

4. Make it look as original as possible...

5. Then give us the new link to try to solve...

6. Once solved, you should be able to easily convert the program to work on the origainal site with a few url changes..

7. I do this myself a lot when its a hard one to solve... since I do not want to alert the owner while I am experimenting... with a ton of hits...

 

Hope this helps... use SKype if you are interested in learning more...its a fun hobby for me....

What you don't have Skype? YOu should get it, its free and beats the hell out of MSN and Yahoo..

 

Tom

[graysqwrl 0]

I tried to paste the code from the page here, but it turned out to be too long.

 

I copied the code to a couple of pages on my server.

 

Here's the search criteria page: bahzing.com/mainsearch.aspx

 

it's the Radius Search button I was trying to click which would open the window that looked like this:

 

bahzing.com/radius_button.aspx

 

When I fill the form on that page and hit enter, it should update the main search page. However, when I nav in ubot specifically to the radius button page which actually is this page: http://www.themls.com/maps/RadiusSearch.aspx

 

fill the form and hit enter... it doesn't update the session in the main search page.

 

If you can help.... please do.

 

Also, thanks for the concern about multiple records being pulled, but I have to do all this by hand pretty much daily anyways. I'm pulling and comparing comparable properties from my local MLS, so the lists end up being small.... not hundreds.

[TommyTx 5]

OK! We are gonna need a lot more info to help....

1. I assume you can't give us access to that second form cause its inside the password area..

2. I see there is a guest entry... does this allow us access to these forms or no..

3. The form you enter the original address in.. is that your form or one of the MLS forms?

4. When you manually fill in the radius form, it then auto goes and fills in the data on all the addresses?

5. Does the manual portion work ok..

6. Does UBOT fill in the radius form ok.. but then just fails to click the button.

7. or is ubot clicking the button, but then the filling of the data is just not occurring after the UBOT click.

8. Ok... did I get it all right, part of it, or none...heee...heee..

 

Have you tried having ubot fill in the form and then stop and you manually click the button just to make sure that all the other stuff is working fine...

 

What happens if we enter an address and click... will it go to the form and fill in data or is all that inside the password area... course we would need to know a valid in range address.. looks like the California area.

[Praney Behl 314]

Try to click it in a 'new sub-window'

[graysqwrl 0]

OK! We are gonna need a lot more info to help....

1. I assume you can't give us access to that second form cause its inside the password area..

2. I see there is a guest entry... does this allow us access to these forms or no..

3. The form you enter the original address in.. is that your form or one of the MLS forms?

4. When you manually fill in the radius form, it then auto goes and fills in the data on all the addresses?

5. Does the manual portion work ok..

6. Does UBOT fill in the radius form ok.. but then just fails to click the button.

7. or is ubot clicking the button, but then the filling of the data is just not occurring after the UBOT click.

8. Ok... did I get it all right, part of it, or none...heee...heee..

 

Have you tried having ubot fill in the form and then stop and you manually click the button just to make sure that all the other stuff is working fine...

 

What happens if we enter an address and click... will it go to the form and fill in data or is all that inside the password area... course we would need to know a valid in range address.. looks like the California area.

 

1. You are correct. I can't give anyone direct access... as it is members only

2. I screwed around in the guest entry and couldn't find a similar form

3. The way I have it set up is I enter an address in the UI boxes of ubot. Ubot runs the search for the public records on the property and scrapes a list of all pertinent data.. ie sqft, lot size, beds, baths, etc.

It uses those variables to populate the "mainsearch" page to find properties that are similar to the subject property and within a proximity of 1 mile. I hope that answers your question. Initially "I" enter the address into ubot, but the only form actually filled with the address is the "radius_button" page which is provided by the MLS.

4. Not quite sure what you meant, but after ubot enters the address and radius it should return to the mainsearch page to continue filling in the rest of the search variables.

5, 6, 7. If by manual you mean... can I click the radius search button manually from within ubot. Yes, I can click it but it doesn't open the new window with the radius search. So I was trying to nav there in a sub window. When I nav there, ubot can fill the form, but when it clicks enter... the sub window goes white, and nothing happens in the mainsearch page. Actually, it just seems to fail to return to the search session correctly. It seems to me whatever passes those variables/search criteria to the regular search session just fails.

 

Yes, I've tried manually clicking the radius search button. No new window opens.

Yes, I've tried manually clicking enter in the radius search page, but that sub window just turns white.

 

A valid address is:

1650 W 23rd st

Los angeles, ca 90007

 

that was the last property I inspected

[TommyTx 5]

This sounds a little more complicated that I have time to work on....

I would guess that no one will be able to help you here unless you allow access and walk them thru exactly what you are trying to do... after that explanation, I have no idea of exactly what you are trying to do or what you need help with....

My guess is the only way anyone can help you here is to allow them temp access and walk them thru the problem step by step then get suggestions or a solution and then immediately change the password once they are out.

Not much damage anyone can do by being in the site a few hours.... just make sure its not someone who has done 3 posts .. first they probably dont' know enough to help you and second.. the more posts the longer they have been around and visibiliy on the forum for long periods without someone bad mouthing them means they are relatively trustworthy... and again not much damage anyone can do in a few hours....

 

Some projects are so intricate there can be no solution unless the person has infinite details of the problem.. and can see them first hand.... all our MLS's do the market analysis for you by comparing all the nearby properties automatically so I have never had a need to invent such a tool...