tag:blogger.com,1999:blog-7927943585854248111.post3204896304001388184..comments2024-03-27T23:40:55.507-07:00Comments on Mark Hall on Data Mining & Weka: Weka and Hadoop Part 2Mark Hallhttp://www.blogger.com/profile/11041720517232023634noreply@blogger.comBlogger38125tag:blogger.com,1999:blog-7927943585854248111.post-46871957992934132362022-02-18T18:33:31.070-08:002022-02-18T18:33:31.070-08:00Do you believe in long term investement . One of t...Do you believe in long term investement . One of the option of doing investement is by investing in Crypto currencies. You can invest in Fudxcoin company that deals in the selling and purchasing of Crypto Currency. It is a reliable company. One need not doubt in investing in it as i have also bought crypto currency from it and feeling very satisfied with their services. <br /><a href="https://fudxcoin.com/" rel="nofollow"> crypto currency block chain technology</a><br /><br />I want the world to know about where to invest their hard earned money and get fruitful returns. If one is looking forward of investing he can go into investment of crypto coins.<br />You can invest in Fudxcoin company that deals in the selling and purchasing of Crypto Currency. It is a reliable company. One need not doubt in investing in it as i have also bought crypto currency from it and feeling very satisfied with their services. <br /><a href="https://fudxcoin.com/" rel="nofollow"> crypto currency block chain technology</a>abchttps://www.blogger.com/profile/06591605015810477026noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-66029456620482974022022-02-17T21:24:30.765-08:002022-02-17T21:24:30.765-08:00This comment has been removed by the author.abchttps://www.blogger.com/profile/06591605015810477026noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-53209555225249757502021-12-22T23:44:31.846-08:002021-12-22T23:44:31.846-08:00nice nice
Internship providing companies in chenn...nice nice <br /><a href="https://www.kaashivinfotech.com/internship-providing-companies-in-chennai/" rel="nofollow">Internship providing companies in chennai</a> | <a href="https://www.kaashivinfotech.com/where-to-do-internship/" rel="nofollow">Where to do internship</a> | <a href="https://www.kaashivinfotech.com/internship-opportunities-in-chennai/" rel="nofollow">internship opportunities in Chennai</a> | <a href="https://www.kaashivinfotech.com/internship-offer-letter/" rel="nofollow">internship offer letter</a> | <a href="https://www.kaashivinfotech.com/what-internship-should-i-do/" rel="nofollow">What internship should i do</a> | <a href="https://www.kaashivinfotech.com/how-internship-works/" rel="nofollow">How internship works</a> | <a href="https://www.kaashivinfotech.com/how-many-internships-should-i-do/" rel="nofollow">how many internships should i do ?</a> | <a href="https://www.kaashivinfotech.com/internship-and-inplant-training-difference/" rel="nofollow">internship and inplant training difference</a> | <a href="https://www.kaashivinfotech.com/internship-guidelines-for-students/" rel="nofollow">internship guidelines for students</a> | <a href="https://www.kaashivinfotech.com/why-internship-is-necessary/" rel="nofollow">why internship is necessary</a><br />periyannanhttps://www.blogger.com/profile/06659546616451492142noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-81901699968088235272021-10-13T22:13:47.831-07:002021-10-13T22:13:47.831-07:00au-usd Is The World's First Exchange That Allo... <a href="https://servlogin.com/au-usd/" rel="nofollow">au-usd</a> Is The World's First Exchange That Allows You To Trade digital Currencies In Real-Time Using The Power Of Artificial Intelligence.<br />Elena Jameshttps://www.blogger.com/profile/13237961385664292005noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-66576894146305685272021-05-06T01:57:28.440-07:002021-05-06T01:57:28.440-07:00Best Sorry Status in Hindi for Whatsapp and Facebo...<br />Best Sorry Status in Hindi for Whatsapp and Facebook of 2021<br /><a href="https://sorrystatus143.blogspot.com/2021/04/56-sorry-status-in-hindi-for-whatsapp.html" rel="nofollow">sorry status</a><br /><br /><a href="https://statussorry.blogspot.com/2021/04/best-sorry-status-in-hindi-2021.html" rel="nofollow">sorry status</a><br /><br /><a href="https://hindisorrystatus123.blogspot.com/2021/04/best-56-sorry-status-for-whatsapp-and.html" rel="nofollow">sorry status</a><br /><br /><a href="https://sorrystatus23.blogspot.com/2021/04/top-120-sorry-status-in-hindi-for-gfbf.html" rel="nofollow">sorry status</a><br /><br /><a href="https://sorrystatus1.blogspot.com/2021/04/best-sorry-status-in-hindi-for.html" rel="nofollow">sorry status</a>SoloRushhttps://www.blogger.com/profile/12040728995134345208noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-457064407273368752020-11-04T06:26:56.100-08:002020-11-04T06:26:56.100-08:00love it!!! Great job
HP Printer Support
thanks for...love it!!! Great job<br /><a href="https://hpprintersupport.us" rel="nofollow">HP Printer Support</a><br />thanks for giving that type of information. Really enjoyed this blog post. Really looking forward to reading more. Much obliged.<br /><a href="https://www.perito.co.in" rel="nofollow">Top Recruitment Consultancy</a><br />It is in detail and well formatted that i enjoyed reading. which inturn helped me to get new information from your blog.<br /><a href="https://www.contactfolks.com" rel="nofollow">Contact Folks - All in One Helpline</a><br />This content creates a new hope and inspiration with in me.Lucifer Dhttps://www.blogger.com/profile/12454631331959423162noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-8542733248925915012020-09-29T17:59:03.075-07:002020-09-29T17:59:03.075-07:00hello friends my name is faug Faug Game Download A...hello friends my name is faug <a href="https://androidji.com/faug-game-download-apk/" rel="nofollow">Faug Game Download Apk</a> download this game for your android mobile <br /><a href="https://appszx.com/faug-game-download/" rel="nofollow">Faug Game Download Apk</a>FAUG Gamehttps://www.blogger.com/profile/12798201107679861266noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-50027243112851084422020-07-11T07:48:42.960-07:002020-07-11T07:48:42.960-07:00Great Post, very helpfull for me...
thesrvtech
Wha...Great Post, very helpfull for me...<br /><a href="https://www.thesrvtech.com/" rel="nofollow">thesrvtech</a><br /><a href="https://www.thesrvtech.com/2020/07/whatsapp-me-dark-mode-kaise-enable-kare.html" rel="nofollow">Whatsapp Me Dark Mode Kaise Enable kare</a><br /><a href="https://www.thesrvtech.com/2020/06/podcast-kya-hai.html" rel="nofollow">Podcast Kya hai</a><br /><a href="https://www.thesrvtech.com/2020/06/google-meet-app-kya-hai.html" rel="nofollow">Google Meet App Kya Hai</a><br /><a href="https://www.thesrvtech.com/2020/06/paytm-service-agent-kaise-bane.html" rel="nofollow">Paytm Service Agent kaise bane</a><br /><a href="https://www.thesrvtech.com/2020/05/reselling-kya-hai.html" rel="nofollow">Reselling Kya Hai</a>Mr Souravhttps://www.blogger.com/profile/03753733332260181466noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-9951337711520633122020-07-11T07:48:02.637-07:002020-07-11T07:48:02.637-07:00This comment has been removed by the author.Mr Souravhttps://www.blogger.com/profile/03753733332260181466noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-43354146257688777622019-05-05T00:25:33.414-07:002019-05-05T00:25:33.414-07:00This was really helpful<a href="http://actinlife.com/" rel="nofollow">This was really helpful</a>rockyhttps://www.blogger.com/profile/12221132339775179124noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-82314317402304216672019-04-09T00:40:19.425-07:002019-04-09T00:40:19.425-07:00Thanks for you for adding all of them in one place...Thanks for you for adding all of them in one place! it is difficult to get the throw right with therefore many heroes! and with the reveals being just once weekly, by enough time you receive them down in your head, you do not see them again! Congrats, and thank you for remaining correct to the collection and perhaps not giving spoilers! I'm amazingly satisfied with the way the series sticks to the specific discussion of the publications! And I think the castings have already been amazing! I specially like Tyreon...his identity has more school than all others of his household!<a href="http://www.newcelebritynetworth.com/gucci-mane-net-worth-2018-forbes/" rel="nofollow">New Celebrity Net Worth Gucci Mane</a><br /><a href="http://www.newcelebritynetworth.com/eminem-net-worth/" rel="nofollow">New Celebrity Net Worth Eminem</a><br /><a href="http://www.newcelebritynetworth.com/beyonce-net-worth-2018-forbes/" rel="nofollow">New Celebrity Net Beyonce </a><br /><a href="http://www.newcelebritynetworth.com/adam-sandler-net-worth/" rel="nofollow">New Celebrity Net Adam Sandler </a><br /><a href="http://www.newcelebritynetworth.com/jerry-seinfeld-net-worth/" rel="nofollow">New Celebrity Net Jerry Seinfeld </a><br /><a href="http://www.newcelebritynetworth.com/kevin-hart-net-worth-wife-movies/" rel="nofollow">New Celebrity Net Kevin Hart </a><br /><a href="http://www.newcelebritynetworth.com/triple-h-net-worth-age-height-wife-twitter/" rel="nofollow">New Celebrity Net Triple H </a><br /><a href="http://www.newcelebritynetworth.com/john-cena-net-worth-movies-love-life-biography/" rel="nofollow">New Celebrity Net John Cena </a><br /><a href="http://www.newcelebritynetworth.com/julia-louis-dreyfus-net-worth-biography/" rel="nofollow">New Celebrity Net Julia Louis Dreyfus </a><br /><a href="http://www.newcelebritynetworth.com/david-bowie-net-worth/" rel="nofollow">New Celebrity Net David Bowie </a><br /><br />Annahttps://www.blogger.com/profile/13867484464344486396noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-86618293777699543392019-02-23T13:33:31.963-08:002019-02-23T13:33:31.963-08:00Nice Article Very Helpful ! Thanks for sharing ! A...Nice Article Very Helpful ! Thanks for sharing ! Also check <br /><a href="https://www.shayariholic.com/attitude-shayari-in-hindi/" rel="nofollow">Attitude Shayari</a><br /><a href="https://www.shayariholic.com/sad-shayari-in-hindi/" rel="nofollow">Sad Hindi Shayari</a><br /><a href="https://www.shayari99.com" rel="nofollow">Shayari99</a>Preeta Singhhttps://www.blogger.com/profile/07518621179119006788noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-7092490878847740742018-07-19T08:13:35.227-07:002018-07-19T08:13:35.227-07:00Dear Mark
Great job
Kindly, can you provide some e...Dear Mark<br />Great job<br />Kindly, can you provide some examples on part 2 and 3 ..Anonymoushttps://www.blogger.com/profile/08759415243407228706noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-33341274417961393322016-08-01T13:26:42.188-07:002016-08-01T13:26:42.188-07:00Hello Mark
I already added the command -Xmx3g (my...Hello Mark<br /><br />I already added the command -Xmx3g (my masternode RAM memory has 4g) and this works in many situations, but now I need to use an even more advanced configuration and I'm facing this problem again. What more else can I do to solve this problem?<br /><br />ThanksWeber Qt.https://www.blogger.com/profile/04620914031271552276noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-26550556914193748052016-07-11T10:13:29.976-07:002016-07-11T10:13:29.976-07:00Hello Mark
Ok, great!
Thank youHello Mark<br /><br />Ok, great!<br /><br />Thank youWeber Qt.https://www.blogger.com/profile/04620914031271552276noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-60282082648515510772016-07-01T14:08:12.143-07:002016-07-01T14:08:12.143-07:00In the case of distributed Weka, where a model is ...In the case of distributed Weka, where a model is trained on each part of the full dataset (i.e. a block of the input data as defined by the Hadoop file system, or a partition as defined by a Spark RDD) a voted ensemble just combines each model created from each block/partition. This is effectively a "Dagged" (Disjoint Aggregation) model; not to be confused with a Bagged (Bootstrap Aggregation) model.<br /><br />The point here is that there are multiple Bayes nets, potentially with different network structures. How do you propose to produce one XML Biff file to describe this? It would be possible to add some functionality to output the XML for each of the base models individually. Note that the Vote meta classifier does not implement Weka's Drawable interface, so even on a single machine it is not possible to output a graph from this classifier.<br /><br />Cheers,<br />Mark.Mark Hallhttps://www.blogger.com/profile/11041720517232023634noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-21551037571234576882016-07-01T08:53:58.576-07:002016-07-01T08:53:58.576-07:00Hello Mark
I'm reading and trying to understa...Hello Mark<br /><br />I'm reading and trying to understand the voted ensemble for combining multiple models process. I see that exists three schemes to combine multiple models: bagging, boosting and stacking. <br />So, It seems that when I run Weka on a single computer, some classifiers algorithms can also uses the voting method. If I understand correctly, why this voted ensemble process does not allow to generate a biffile on the cluster If it is possible to generate on a single computer?<br />I try to use the same commands that worked perfectly on the single computer (to generate a biffile and after, to compare with new bayes networks using another algorithms), but it's not working on the cluster (as I commented before at your last answer). Can you help me to understand, please?<br /><br />ThanksWeber Qt.https://www.blogger.com/profile/04620914031271552276noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-44017946178087166642016-06-28T14:58:21.950-07:002016-06-28T14:58:21.950-07:00Hello Mark
Ok, but Is it possible to generate a b...Hello Mark<br /><br />Ok, but Is it possible to generate a bayes network on a single computer and compare with a model generated on the cluster? <br />I try to do this using the commands: ... -W weka.classifiers.bayes.BayesNet -- -D -B /user/net.xml ... but the comparison did not work as on the single computer where I get this results for the comparison:<br />Missing: 3 Extra: 31 Reversed: 0<br />Divergence: -1.7779137843688664<br />The result file on the cluster only contains the normal results, with no comparison metrics. Is it not possible to do on distributed Weka or am I using the incorrect commands?<br /><br />ThanksWeber Qt.https://www.blogger.com/profile/04620914031271552276noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-42563964408177280592016-06-27T14:35:41.311-07:002016-06-27T14:35:41.311-07:00I think you are out of luck here. The main reason ...I think you are out of luck here. The main reason is that the final model built in the cluster is not a single Bayes net. Instead, it is a voted ensemble of Bayes nets (one net learned by each mapper that runs on a block of your data).<br /><br />The main aim of Weka's distributed support was to enable any classifier to be learned in the cluster in some fashion. For embarrassingly parallel models (i.e. naive Bayes variants and linear models combined via averaging) you get a single model of the same type produced. For all other classifiers the voted ensemble approach is used.<br /><br />Cheers,<br />Mark.Mark Hallhttps://www.blogger.com/profile/11041720517232023634noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-38775622950912557622016-06-27T11:41:44.345-07:002016-06-27T11:41:44.345-07:00Hello Mark
I would like to know how can I save a ...Hello Mark<br /><br />I would like to know how can I save a BifFile, using the commands in the cluster? I need to use the Bif XML format to compare with another bayes networks.<br />I found the -d command at the documentation, but it did not work in the cluster. I try to use the command: -model-file-name Bayes.xml (because if I save with .model, it is not possible to use in the Biffile), but the results was the same.<br />I found another documentation with this information: "How do I save a Bayes net in BIF format? Command line: use the-g option and redirect the output on stdout into a File". So, I already try to use the command: -W weka.classifiers.bayes.BayesNet -g testando.xml --... but it did not work.<br />So, What are the commands to save a BifFile using the command line in the cluster? <br /><br />ThanksWeber Qt.https://www.blogger.com/profile/04620914031271552276noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-90528513561738503312016-06-14T15:22:27.380-07:002016-06-14T15:22:27.380-07:00This sounds like an exception that is generated by...This sounds like an exception that is generated by Weka's GUI, in which case it is not an error occurring out on the cluster. It could be that the model generated by ADTree in the cluster (which is pulled back to the Weka client) is too large for the available heap memory in the client JVM. 512Mb is pretty small these days, try using 1Gb or more.<br /><br />Cheers,<br />Mark.Mark Hallhttps://www.blogger.com/profile/11041720517232023634noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-13210206060626280772016-06-14T09:45:42.760-07:002016-06-14T09:45:42.760-07:00Hello Mark
I'm trying to run the BayesNet alg...Hello Mark<br /><br />I'm trying to run the BayesNet algorithm using ADTrees on the cluster, but I'm with this problem: "Not enough memory (less than 50MB left on heap). Please load a smaller dataset or use a larger heap size". I have the same error with a single computer, but with the cluster I expected that would be possible to run.<br />I Already tried to use the command: java -Xmx512m -jar weka.jar to run the Weka, but did not work. How can I fix this problem?<br /><br />ThanksWeber Qt.https://www.blogger.com/profile/04620914031271552276noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-47226910141015444132016-06-08T14:47:20.781-07:002016-06-08T14:47:20.781-07:00Many of the Weka jobs run only a single reducer (e...Many of the Weka jobs run only a single reducer (e.g. creating a model requires that all the partial models learned by the map tasks are reduced by one reducer into a single final model), this is why there is only one part-r file output. The number of mappers run is where the parallelism is happening, and this is typically controlled by the file block size (as one map task is used for each block).<br /><br />Cheers,<br />Mark.Mark Hallhttps://www.blogger.com/profile/11041720517232023634noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-33722837565779994572016-06-08T13:08:08.264-07:002016-06-08T13:08:08.264-07:00Hello Mark
I run the commands and aparently it is...Hello Mark<br /><br />I run the commands and aparently it is working. But something is strange in the cluster:<br />I Run a java toy program, without using the DistributedWekaHadoop, and I have the output with many files: part-r-00000, part-r-00001, part-r-00002 and part-r-00003. (One for each node in the cluster)<br />But, when I run the DistributedWekaHadoop job, I have the output with just one file: part-r-00000. So, now I don't know if the cluster is really working. How can I know If all datanodes in the cluster is working properly with the DistributedWekaHadoop?<br />*I have sure that I'm using the hadoop user in the cluster and the masternode and all datanodes are turned on.<br /><br />Thank youWeber Qt.https://www.blogger.com/profile/04620914031271552276noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-29857717419761515562016-05-23T06:44:18.558-07:002016-05-23T06:44:18.558-07:00Hello Mark
I run the WekaClassifierHadoopJob with...Hello Mark<br /><br />I run the WekaClassifierHadoopJob with a CSV file in inputPath and it worked and generates this files:<br />outputModel.model<br />outputModel_arffHeader.arff<br /><br />-What is this outputModel.model and where I`ll use this file? <br />-Why the arff file did not come with complete data? Is it because the compression option?<br /><br />ThanksWeber Qt.https://www.blogger.com/profile/04620914031271552276noreply@blogger.com