mdc101
-
Content Count
119 -
Joined
-
Last visited
-
Days Won
2
Posts posted by mdc101
-
-
HI guys
I'm doing a simple insert into a database table from a ubot table using table cells.
In the ubot table I see 13.2 as an example.
Using set the insert statement is generated as seen two lines down.
When I run the insert into mySql table the decimal is dropped and the value is rounded down to 13.
#inset_record: INSERT INTO sometable
(
sometable.profileid,
sometable.theme_keywords,
sometable.ID,
sometable.Flag,
sometable.kwSource,
sometable.LARI,
sometable.PPC_CostclickMonthly,
sometable.PPC_clickMonthly,
sometable.TrafficForecast,
sometable.CompetingPages
)
VALUES
(
'5060',
'dysfunction',
'0000',
'xxxxxx',
'xxxxxxxx',
'100',
'13,2',
'12643,995',
'71769,96',
'13500000'
)All decimals are being dropped.
Why is this happening?
The table in mysql is set to double.
Any help will be appreciated.
-
@YuraB
That worked perfectly, Thanks for all the help. Your a Legend.
I was not aware one could place conditions within a scrape.
(tagname="td" AND class="cluster-name")
I am assuming this has to be hard coded?
-
Hi Thanks for the response
The code is working 95%!
What it does is it grabs the first row of the ignore list
<td class="cluster-name" style="text-align: left">
<a href="http://subdomain.domin.com/blueprints/2088/nodes/40937">buy beer making kits online</a></td>Any idea why this is happening? -
Hi Guys
There is a number of rows in a table.
How do I scrape the url into one list & the anchor text into another list
The rows I want to scrape contain a <span title="Number of active keywords">(*)</span>.
Rows to be ignored don't have the span <span title="Number of active keywords">(*)</span>
(20) is a number that changes
Scrape URL
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
<td class="cluster-name" style="text-align: left">
<a href="http://subdomain.domin.com/blueprints/2088/nodes/40938">beer kits</a><span title="Number of active keywords">(20)</span><td class="cluster-name" style="text-align: left">
<a href="http://subdomain.domin.com/blueprints/2088/nodes/40939">buy beer kits</a><span title="Number of active keywords">(15)</span><td class="cluster-name" style="text-align: left">
<a href="http://subdomain.domin.com/blueprints/2088/nodes/40940">beer kits recipes</a><span title="Number of active keywords">(5)</span><td class="cluster-name" style="text-align: left">
<a href="http://subdomain.domin.com/blueprints/2088/nodes/40941">beer kits</a><span title="Number of active keywords">(18)</span>------------------------------------------------------------------------------------------------------------------------------------------------------------------------Ignore URL
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
<td class="cluster-name" style="text-align: left">
<a href="http://subdomain.domin.com/blueprints/2088/nodes/40937">buy beer making kits online</a></td><td class="cluster-name" style="text-align: left"><a href="http://subdomain.domin.com/blueprints/2088/nodes/40937">buy beer making hobs online</a></td><td class="cluster-name" style="text-align: left"><a href="http://subdomain.domin.com/blueprints/2088/nodes/40937">buy beer making yeast online</a></td>------------------------------------------------------------------------------------------------------------------------------------------------------------------------I have tried the following but cant seem to get it to workclear list(%Silo_article_keyword_list)set(#Silo_article_keyword_list, $nothing, "Global")
set(#Silo_article_keyword_list, $trim($replace regular expression($find regular expression($scrape attribute(<class="cluster-name">, "outerhtml"), "(?:<td.*?<span title=\"Number of active keywords\">\\(\\d+\\)</span>.*?<a href=\"(.*?)\".*?</td>)|(?:<td.*?<a href=\"(.*?)\".*?<span title=\"Number of active keywords\">\\(\\d+\\)</span>.*?</td>)"), "([^aA-zZ\\s]+)", $nothing)), "Global")
add list to list(%Silo_article_keyword_list, $list from text(#Silo_article_keyword_list, $new line), "Don\'t Delete", "Global") -
Thanks guys for the feed back
Matt
-
Hi Guys
Does anyone know how to insert a screenshot into a mysql database blob field?
So basically the idea is ubot goes to page does the screen grab and then inserts it into the database.
Is this possible?
-
Hi Guys thanks for the response
The application is saas based and you need to login to access the page. So cant give reference url
If you wrapped the <ol> her html head and body you will basically get the page output
Oricoun I will try your suggestion. Ubotbuddy, would love to see what you do
Regards
Matt
-
Hi Guys
Been knocking me head against a wall for a few days and cannot figure this out.
I have the following page I want to scrape attached.
There are 10 <li></li> tags holding ranking information I need.
Each <li></li> tag is separated by a <hr> tag.
I am wanting to scrape the following below form each <li></li> tag and want to add it to table highlighted in orange so I can insert into a database.
How does one scrape this data so all rows stay precise and don't get mixed up?
Thanks
Matt
<li>
<b>www.clipinhair.co.za</b> - creation date unknown <span title="Page Rank">(PR: 2)</span> <span title="Domain Authority">(DA: 19)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 281,927</li>
<li>Ranking Multiplier - 0.434</li>
<li>Pages Indexed - 90</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.clipinhair.co.za/</td>
<td>2</td>
<td>32</td>
<td>
hair extensions (1)<br>
</td>
</tr>
</tbody></table>
</li>Here is the complete file: getting attaching file errors
==========================================
<ol>
<li>
<b>www.clipinhair.co.za</b> - creation date unknown <span title="Page Rank">(PR: 2)</span> <span title="Domain Authority">(DA: 19)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 281,927</li>
<li>Ranking Multiplier - 0.434</li>
<li>Pages Indexed - 90</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.clipinhair.co.za/</td>
<td>2</td>
<td>32</td>
<td>
hair extensions (1)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>www.runwayhair.co.za</b> - creation date unknown <span title="Page Rank">(PR: 1)</span> <span title="Domain Authority">(DA: 18)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 79,456</li>
<li>Ranking Multiplier - 0.122</li>
<li>Pages Indexed - 43</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.runwayhair.co.za/</td>
<td>1</td>
<td>31</td>
<td>
hair extensions (2)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>www.i-hairextensions.co.za</b> - creation date unknown <span title="Page Rank">(PR: 0)</span> <span title="Domain Authority">(DA: 18)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 56,272</li>
<li>Ranking Multiplier - 0.087</li>
<li>Pages Indexed - 30</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.i-hairextensions.co.za/</td>
<td>0</td>
<td>31</td>
<td>
hair extensions (3)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>www.bidorbuy.co.za</b> - creation date unknown <span title="Page Rank">(PR: 5)</span> <span title="Domain Authority">(DA: 64)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 40,183</li>
<li>Ranking Multiplier - 0.062</li>
<li>Pages Indexed - 45,300,000</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.bidorbuy.co.za/search/hair+extensions</td>
<td>-1</td>
<td>37</td>
<td>
hair extensions (4)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>www.glamorhair.co.za</b> - creation date unknown <span title="Page Rank">(PR: 0)</span> <span title="Domain Authority">(DA: 12)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 32,365</li>
<li>Ranking Multiplier - 0.050</li>
<li>Pages Indexed - 3,050</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.glamorhair.co.za/</td>
<td>0</td>
<td>22</td>
<td>
hair extensions (5)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>www.frontrow.co.za</b> - creation date unknown <span title="Page Rank">(PR: 0)</span> <span title="Domain Authority">(DA: 12)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 26,570</li>
<li>Ranking Multiplier - 0.041</li>
<li>Pages Indexed - 124</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.frontrow.co.za/</td>
<td>0</td>
<td>26</td>
<td>
hair extensions (6)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>hairextensionsjhb.co.za</b> - creation date unknown <span title="Page Rank">(PR: 2)</span> <span title="Domain Authority">(DA: 10)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 22,496</li>
<li>Ranking Multiplier - 0.035</li>
<li>Pages Indexed - 0</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://hairextensionsjhb.co.za/</td>
<td>2</td>
<td>24</td>
<td>
hair extensions (7)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>www.glamourize.co.za</b> - creation date unknown <span title="Page Rank">(PR: 1)</span> <span title="Domain Authority">(DA: 19)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 19,865</li>
<li>Ranking Multiplier - 0.031</li>
<li>Pages Indexed - 21</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.glamourize.co.za/</td>
<td>1</td>
<td>28</td>
<td>
hair extensions (8)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>www.groupon.co.za</b> - creation date unknown <span title="Page Rank">(PR: 6)</span> <span title="Domain Authority">(DA: 41)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 19,800</li>
<li>Ranking Multiplier - 0.030</li>
<li>Pages Indexed - 52,100</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.groupon.co.za/coupons/beauty/hairdresser/hair-extension</td>
<td>1</td>
<td>1</td>
<td>
hair extensions (10)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
<li>
<b>www.divadivinehair.co.za</b> - creation date unknown <span title="Page Rank">(PR: 2)</span> <span title="Domain Authority">(DA: 30)</span>
<ul>
<li>Google ORV<sup>real</sup> - R 18,836</li>
<li>Ranking Multiplier - 0.029</li>
<li>Pages Indexed - 125</li>
</ul>
<table class="info_table" style="margin-left: 5%; width: 95%">
<tbody><tr>
<th>Page</th>
<th title="Page Rank">PR</th>
<th title="Page Authority">PA</th>
<th>Terms (Rank)</th>
</tr>
<tr>
<td>http://www.divadivinehair.co.za/</td>
<td>2</td>
<td>41</td>
<td>
hair extensions (9)<br>
</td>
</tr>
</tbody></table>
</li>
<hr>
</ol> -
Found a work around
I only used multiply.
I removed the $eval() and it worked
-
Thanks for the reply.
This is so annoying
so why was it working before the update to 5.7.
I working with competing pages and I use the the competing pages number x 10 as a ceiling for me to do analysis.
I have never had this issue in the last 4 years and now that I need the functionality at a critical point in a project the update stuffs up the math functions.
Is there a way to calculate the number I need using JavaScript?
-
Hi Kev
Any work arounds?
-
Hi Guys
Anyone having an issue with the eval multiply reults?
I get odd results like "8.73e+006"
The eval math function is not working.
A basic math sum like 873000 * 10 = 8.73e+006
It should be 8730000Im sure it not the code. Started happening since 5.7 update
Is it just me or is there an actual bug?
if($comparison($scrape attribute($element offset(<style="text-align: right">, 0), "innertext"), "=", $nothing)) {
then {
set(#Psys_ptkw_cmp, 0, "Global")
set(#Psys_ptkw_cmp, 2000000, "Global")
set(#Psys_ptkw_cmp, $replace($trim(#Psys_ptkw_cmp), ",", $nothing), "Global")
set(#Psys_ptkw_ceiling, $nothing, "Global")
set(#Psys_ptkw_ceiling, $eval($multiply(#Psys_ptkw_cmp, #Psys_ptkw_ceiling)), "Global")
}
else {
set(#Psys_ptkw_cmp, 0, "Global")
set(#Psys_ptkw_cmp, $scrape attribute($element offset(<style="text-align: right">, 0), "innertext"), "Global")
set(#Psys_ptkw_cmp, $replace($trim(#Psys_ptkw_cmp), ",", $nothing), "Global")
set(#Psys_ptkw_ceiling, $nothing, "Global")
set(#Psys_ptkw_ceiling, $eval($multiply(#Psys_ptkw_cmp, #Psys_Drill_ceiling)), "Global")
}
} -
I figured out why things where running so slow.
The logging was on and as soon as i turned logging off the bot flew as expected.
Thanks blumi for the pointers earlier
-
The indexes and changes suggested shaved of 10 seconds, thanks for the tip
-
Thanks B for the advise I will try this.
-
-
Oh forgot to say this is running on my local machine which is pretty fast and there is only 7 rows in table
-
Hi Guys so I have been playing around with the database command and setting variables from the retrieved dataset.
I created a command that calls a single record and put it in the table. It then takes the 19 columns from the table and sets each one as a variable to use.
This process takes 1 minute 23 seconds to do.
The call to the mysql database is a view and via the query editor it takes 0.003s.
Does this seem right?
Query editor 0.003s ubot to complete task 1m 23s
I think this is a very slow way of doing things.
Is there a faster way of setting the variables?
Even the debugger is running slow.
I'm on win 8, ubot 5 latest version, dev
Am I doing this correct?
Can anyone suggest a faster method of doing this with examples?
Thanks
Matt
-
-
-
HI Experts
I need some assistance.
Been sitting for hours trying to figure out how to do this.
I have 3 tables with ids. (id="table_1")
I need to check all the check boxes in say id="table_2" but not in id="table_1" or id="table_3" before a form is submitted.
Does anyone know how this can be done?
Any help would be appreciated
Thanks in advance for any suggestions
example
<table class="info_table" id="table_1" style="margin-bottom: 1em; display: none">
<tr>
<td><input name="themes[84412]" type="hidden" value="f"><input id="themes_84412" name="themes[84412]" onclick="check_total_silos(this);" type="checkbox" value="t"></td>
<td></tr>
</table>
<table class="info_table" id="table_2" style="margin-bottom: 1em; display: none">
<tr>
<td><input name="themes[84413]" type="hidden" value="f"><input id="themes_84413" name="themes[84413]" onclick="check_total_silos(this);" type="checkbox" value="t"></td>
<td></tr>
<tr>
<td><input name="themes[84415]" type="hidden" value="f"><input id="themes_84415" name="themes[84415]" onclick="check_total_silos(this);" type="checkbox" value="t"></td>
<td></tr>
<tr>
<td><input name="themes[84418]" type="hidden" value="f"><input id="themes_84418" name="themes[84418]" onclick="check_total_silos(this);" type="checkbox" value="t"></td>
<td></tr>
</table>
<table class="info_table" id="table_3" style="margin-bottom: 1em; display: none">
<tr>
<td><input name="themes[84414]" type="hidden" value="f"><input id="themes_84414" name="themes[84414]" onclick="check_total_silos(this);" type="checkbox" value="t"></td>
<td></tr>
</table>
-
HI Blumi40
Could you maybe provide an example?
-
Hi Thanks for the reponse.
Now that UBot can connect to a database it has really caught my attention. It would be great if they can expand on making db commands more efficient so processes can easily be implemented so we can bulk move lots of data more easily.
Loving where its going though
-
HI Guys
I have been trying to figure out how to do you insert a .csv file into a database with the databse query command?
This has me stumped.
Could someone post a code sample here just to see how the loop works to create the insert statements for each row?
Any help would be appreciated
Regards
Matt
Why are decimals dropped when inserted into Mysql table?
in Scripting
Posted
OK figured this one out
When you create the insert statement you must wrap the number with the FORMAT function
example
FORMAT(column_name,2)