tag:blogger.com,1999:blog-7927943585854248111.post1790610497747172840..comments2024-03-27T23:40:55.507-07:00Comments on Mark Hall on Data Mining & Weka: Weka and SparkMark Hallhttp://www.blogger.com/profile/11041720517232023634noreply@blogger.comBlogger72125tag:blogger.com,1999:blog-7927943585854248111.post-72282201252606404002023-02-19T23:11:56.317-08:002023-02-19T23:11:56.317-08:00I am getting this error on my Windows machine for ... I am getting this error on my Windows machine for the 4th workflow.: java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'Arijithttps://www.blogger.com/profile/05456603036949861292noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-22369193886831316672022-05-18T02:36:26.622-07:002022-05-18T02:36:26.622-07:00I desperately looking for a blog like yours, full ...I desperately looking for a blog like yours, full of information written in simple and understandable language. I always support bloggers like you who do not post only on that topic that make money. Keep it up and a big thumb to you and your work. I also have a request for you nowadays people are obsessed with organic words. Before buying any product they seek organic, especially in foods. Would you like to cover a suggested topic <a href="https://pusht.in/whole-or-sabut-organic-masoor-dal/" rel="nofollow">pusht 100% organic whole masoor dal</a>Ravi Kumarhttps://www.blogger.com/profile/06915651124990515731noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-18051597466064914302022-01-03T22:11:24.844-08:002022-01-03T22:11:24.844-08:00Would you be interested in trading links or maybe ...Would you be interested in trading links or maybe guest writing a blog post or vice-versa?<br /><a href="https://kitsonlinetrainings.com/course/oracle-rac-online-training" rel="nofollow">oracle rac online training </a><br /><a href="https://kitsonlinetrainings.com/course/oracle-rac-online-training" rel="nofollow">oracle rac training </a>KITS Technologieshttps://www.blogger.com/profile/01255736173821596606noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-28199799083286278092021-09-27T03:35:03.558-07:002021-09-27T03:35:03.558-07:00Trade Stocks, Forex, And Bitcoin Anywhere In The W...Trade Stocks, Forex, And Bitcoin Anywhere In The World: <a href="https://servlogin.com/robofx/" rel="nofollow">roboforex login</a> Is The Leading Provider Of Software That Allows You To Trade On Your Own Terms. Whether You Are Operating In The Forex, Stock, Or Cryptocurrency Markets, Use roboforex login Software And Anonymous Digital Wallet To Connect With The Financial World.: roboforex login Is A Currency Trading Company That Allows You To Trade Stocks, Forex, And Cryptocurrency.<br />John Paulsonhttps://www.blogger.com/profile/02423526617581444734noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-3189999046546862972020-11-19T23:32:25.688-08:002020-11-19T23:32:25.688-08:00Very nice article,Thank you for sharing it.
Keep u...Very nice article,Thank you for sharing it.<br />Keep updating...<br /><br /><a href="https://onlineitguru.com/servicenow-online-training.html" rel="nofollow">Big Data and Hadoop Online Training</a>Veera Blogspothttps://www.blogger.com/profile/14710488178692992760noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-1659184682976986212020-11-18T01:02:44.399-08:002020-11-18T01:02:44.399-08:00Having RDDs referenceable for the duration that th...Having RDDs referenceable for the duration that the Spark context is alive makes it possible to have a tighter coupling between Spark job steps in the Knowledge Flow. The success and failure connection types introduced in distributedWekaHadoop can now be used to carry data, such as the context and references to various RDD datasets that are in play. <a href="http://www.thecollectionmarts.com/product-category/bedding/" rel="nofollow">bedsheets buy online</a> , <a href="http://www.thecollectionmarts.com/product-category/comforter-set/" rel="nofollow">premium bed sheets</a> , <a href="http://www.thecollectionmarts.com/product-category/high-quality-bedsheets/" rel="nofollow">queen size fitted bed sheets</a> , <a href="http://www.thecollectionmarts.com/product-category/bridal-set/" rel="nofollow">bridal bed covers</a> , <a href="http://www.thecollectionmarts.com/product-category/export-quality-duvet-covers/" rel="nofollow">cotton duvet sets</a> , <a href="http://www.thecollectionmarts.com/product-category/razai-set/" rel="nofollow">vicky razai factory address</a> , <a href="http://www.thecollectionmarts.com/product-category/sofa-cover/" rel="nofollow">sofa cover sofa cover</a> , <a href="http://www.thecollectionmarts.com/product-category/velvet-bedsheets/" rel="nofollow">velvet duvet cover</a>Darren Demershttps://www.blogger.com/profile/08050776248828465230noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-86402674799532117152020-11-16T21:40:37.447-08:002020-11-16T21:40:37.447-08:00software testing company in India
software testing...<a href="https://w3softech.com" rel="nofollow">software testing company in India</a><br /><a href="https://w3softech.com" rel="nofollow">software testing company in Hyderabad</a><br />Thanks for sharing such an informative post.<br />Great article , keep sharing<br /><br />kirankumarpaitahttps://www.blogger.com/profile/01593714001281024596noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-8730985823891147062020-01-28T23:31:31.616-08:002020-01-28T23:31:31.616-08:00As the growth of Big data engineering automation ,...As the growth of<a href="https://www.indiumsoftware.com/big-data-services/" rel="nofollow"> Big data engineering automation </a>, it is essential to spread knowledge in people. This meetup will work as a burst of awareness. Alfred Avinahttps://www.blogger.com/profile/05278500525526860375noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-86961605444328496542019-12-04T21:52:16.329-08:002019-12-04T21:52:16.329-08:00The details captured by the data migration develo...The details captured by the <a href="https://migrationsuggestion.data.blog/2019/07/29/various-types-of-data-migration-and-its-benefits-need-to-know/" rel="nofollow"> data migration development services </a> help in the migration of files folder excellently. The steps explained by your services are beneficial, which helped me in the migration of the folder files easily without getting interrupted at any point.<br />Nikishahttps://www.blogger.com/profile/00589954961755559760noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-3609036726450754252019-11-04T22:32:47.044-08:002019-11-04T22:32:47.044-08:00Good Blog
Sanjary kids is the best playschool, pre...Good Blog<br />Sanjary kids is the best playschool, preschool in Hyderabad, India. Start your play school,preschool in Hyderabad with sanjary kids. Sanjary kids provides programs like Play group,Nursery,Junior KG,Serior KG,and Teacher Training Program.<br /><a href="http://www.sanjarykids.com/" rel="nofollow">best preschool in hyderabad </a><br /><a href="http://www.sanjarykids.com/" rel="nofollow">preschool teacher training </a><br /><a href="http://www.sanjarykids.com/" rel="nofollow">playschools in hyderabad </a><br /><a href="http://www.sanjarykids.com/" rel="nofollow">preschool teacher training in hyderabad </a>Chandra Sekhar Reddyhttps://www.blogger.com/profile/14721695642222569714noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-11193757484996616832018-05-10T14:38:47.480-07:002018-05-10T14:38:47.480-07:00The key to this is to take a look at the command l...The key to this is to take a look at the command line options for the job (or the listOptions() method in WekaClassifierMapTask). If you run:<br /><br />java weka.Run .WekaClassifierSparkJob -h<br /><br />One of the options is:<br /><br />-W<br /> The fully qualified base classifier to use. Classifier options<br /> can be supplied after a '--'<br /><br />So, your setClassifierMapTaskOptions() call needs to actually take the following string:<br /><br />"-W weka.classifiers.functions.LinearRegression -- -S 0 -R 1.0E-8 -num-decimal-places 4"<br /><br />A similar string will allow you to specify Gaussian processes.<br /><br />Cheers,<br />Mark.<br /><br />Mark Hallhttps://www.blogger.com/profile/11041720517232023634noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-2116622980953401102018-03-16T03:26:59.951-07:002018-03-16T03:26:59.951-07:00Dear Mark,
I am using DistributedWekaSpark via Jav...Dear Mark,<br />I am using DistributedWekaSpark via Java.<br />I am trying to run WekaClassifierSparkJob and a WekaClassifierEvaluationSparkJob for Linear Regression with the code bellow:<br /><br />WekaClassifierSparkJob job = new WekaClassifierSparkJob();<br /> job.setClassAttribute("last");<br /> job.setDebug(false);<br /> job.setModelFileName(classifier.getName()+".model");<br /> job.setNumIterations(1);<br /> job.setRandomizeAndStratify(false);<br /> job.setWriteRandomlyShuffledSplitsToOutput(false);<br /> job.setClassifierMapTaskOptions("LinearRegression -S 0 -R 1.0E-8 -num-decimal-places 4");<br /> job.setDataset(previousJob.getDatasets().next().getKey(), previousJob.getDataset(previousJob.getDatasets().next().getKey()));<br /> job.setCachingStrategy(previousJob.getCachingStrategy());<br /> job.getSparkJobConfig().setOutputDir(Constants.SPARK_OUTPUT);<br /> job.runJobWithContext(previousJob.getSparkContext());<br /> WekaClassifierEvaluationSparkJob job2 = new WekaClassifierEvaluationSparkJob();<br /> job2.setDebug(false);<br /> job2.setOutputSubdir(classifier.getName());<br /> job2.setDataset(previousJob.getDatasets().next().getKey(), previousJob.getDataset(previousJob.getDatasets().next().getKey()));<br /> job2.setCachingStrategy(previousJob.getCachingStrategy());<br /> job2.getSparkJobConfig().setOutputDir(Constants.SPARK_OUTPUT);<br />job2.runJobWithContext(job.getSparkContext());<br />if(job2.getJobStatus().equals(DistributedJob.JobStatus.FINISHED)){<br /> System.out.println(classifier.getName()+": "+job2.getText());<br /> }<br />Although this seems to work, if I run the same job for Gaussian Processes I get the same results.<br />So, I guess I am not configuring the jobs correctly.<br />How can I configure them to get the same results as when I run the jobs from Weka KnowledgeFlow?<br /><br />Thanks in advance,<br />MariosMarioshttps://www.blogger.com/profile/01542930597505728374noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-20705554749039562018-02-25T23:10:15.131-08:002018-02-25T23:10:15.131-08:00There is nothing built in to distributed Weka to h...There is nothing built in to distributed Weka to handle kerberos I'm afraid. However, the general approach (not specific to Pentaho data integration) at:<br /><br />https://help.pentaho.com/Documentation/5.2/0P0/0W0/030/040<br /><br />might be usable.<br /><br />Cheers,<br />Mark.Mark Hallhttps://www.blogger.com/profile/11041720517232023634noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-84293445729836311332018-02-24T19:15:13.224-08:002018-02-24T19:15:13.224-08:00This comment has been removed by the author.Anonymoushttps://www.blogger.com/profile/07331987463129558456noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-48040363320899326332018-02-23T10:15:12.491-08:002018-02-23T10:15:12.491-08:00Mark,
Any thoughts on Arshaq's question above...Mark,<br /><br />Any thoughts on Arshaq's question above?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-71610883295342657272018-02-23T09:59:30.656-08:002018-02-23T09:59:30.656-08:00Arshaq,
Did you resolve this? I'm getting th...Arshaq,<br /><br />Did you resolve this? I'm getting the same error.<br /><br />ThanksAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-14541352269527470792018-02-19T18:26:58.011-08:002018-02-19T18:26:58.011-08:00This comment has been removed by the author.Anonymoushttps://www.blogger.com/profile/07331987463129558456noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-37299417453152445922018-01-19T17:13:26.374-08:002018-01-19T17:13:26.374-08:00Thank you very much
Thank you very much<br />Anonymoushttps://www.blogger.com/profile/07331987463129558456noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-27316512160711100552018-01-08T21:08:42.876-08:002018-01-08T21:08:42.876-08:00you may open weka explorer and load any dataset fi...you may open weka explorer and load any dataset file to enable other tabs. Now go to cluster tab, under the result list area, right click which will show you load model object file. There you can locate the model file and see it in cluster output window. Ankit Desaihttps://www.blogger.com/profile/12231593294261414550noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-46567450373506302222017-11-22T16:00:53.817-08:002017-11-22T16:00:53.817-08:00Thank you very much , another question please, i r...Thank you very much , another question please, i ran distributed kmeans in local mode and i got a model extension file, how i can view the file <br />Anonymoushttps://www.blogger.com/profile/07331987463129558456noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-84470378060177892622017-11-20T12:37:34.257-08:002017-11-20T12:37:34.257-08:00Hi Fray,
Packages are installed in ${user.home}/w...Hi Fray,<br /><br />Packages are installed in ${user.home}/wekafiles/packages. Inside this directory you should find distributedWekaBase and distributedWekaSpark (or distributedWekaSparkDev, depending on which one you installed). Each directory contains a src folder.<br /><br />Cheers,<br />Mark.Mark Hallhttps://www.blogger.com/profile/11041720517232023634noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-669017356306258022017-11-20T05:05:14.114-08:002017-11-20T05:05:14.114-08:00This comment has been removed by the author.Anonymoushttps://www.blogger.com/profile/07331987463129558456noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-20976165133846959452017-08-31T16:43:42.469-07:002017-08-31T16:43:42.469-07:00Hello Mark,
I am trying to use the distributedWek...Hello Mark,<br /><br />I am trying to use the distributedWekaSpark package. Everything is running fine while running the process using files in local system. But when I try to use a file from kerberose enabled hdfs ,its giving me following errors. Is there a way to connect to kerberized cluster?<br /><br />ArffHeaderSparkJob$266157602|SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]<br />weka.core.WekaException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]<br /> at weka.knowledgeflow.steps.AbstractSparkJob.runJob(AbstractSparkJob.java:294)<br /> at weka.knowledgeflow.steps.AbstractSparkJob.start(AbstractSparkJob.java:221)<br /> at weka.knowledgeflow.StepManagerImpl.startStep(StepManagerImpl.java:1020)<br /> at weka.knowledgeflow.BaseExecutionEnvironment$3.run(BaseExecutionEnvironment.java:440)<br /> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)<br /> at java.util.concurrent.FutureTask.run(FutureTask.java:266)<br /> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)<br /> at java.lang.Thread.run(Thread.java:748)<br /><br />Thanks<br />Arshaqhttps://www.blogger.com/profile/15848663211692549125noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-29745778463205505412017-06-08T02:04:44.044-07:002017-06-08T02:04:44.044-07:00Yes, I replaced the libraries and I'm using Or...Yes, I replaced the libraries and I'm using Oracle JVM. But I guess, it's because I use Spark 2.1. I recently seen that it works with older versions. Thanks for the reply though. I'll try with older versions.Kerem Okhttps://www.blogger.com/profile/16021304425782409399noreply@blogger.comtag:blogger.com,1999:blog-7927943585854248111.post-49217688099514248772017-06-03T05:22:56.528-07:002017-06-03T05:22:56.528-07:00Have you replaced the Spark libraries in ~/wekafil...Have you replaced the Spark libraries in ~/wekafiles/packages/distributedWekaSpark/lib with the spark assembly jar that comes with the Spark distribution being used to run your cluster? Also make sure that you are using an Oracle JVM to run both Weka and Spark.<br /><br />Cheers,<br />Mark.Mark Hallhttps://www.blogger.com/profile/11041720517232023634noreply@blogger.com