tag:blogger.com,1999:blog-32358620461509948712024-02-19T08:36:26.195+01:00oj! BlogJava programming, performance tuning and ojAlgo related stuff.apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-3235862046150994871.post-44662018927878455102019-03-19T14:38:00.001+01:002019-03-19T14:38:56.561+01:00New ojAlgo web siteojAlgo has a new web site, and it's essentially a blog. All content from this blog has been moved/copied there, and future posts will only appear at the ojAlgo web site.<br />
<br />
<a href="http://www.ojalgo.org/">http://www.ojalgo.org</a><br />
<br />
<br />apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.com2tag:blogger.com,1999:blog-3235862046150994871.post-39950050056048557612019-02-18T08:58:00.001+01:002019-02-18T09:18:26.423+01:00Oracle's JVMs HotSpot, Graal CE & Graal EEBenchmarking the HotSpot, Graal CE and Graal EE doing matrix multiplication<br />
<br />
The HotSpot JVM will eventually be replaced by <a href="https://www.graalvm.org/">Graal</a> which is about to be released at v1.0. Graal comes in 2 flavours – a free community edition (CE) and a commercial enterprise edition (EE). The EE has some performance optimisations that the CE doesn't. In particular it has auto-vectorization.<br />
<br />
This is a follow-up to the earlier post <a href="https://ojalgo.blogspot.com/2019/02/quick-test-to-compare-hotspot-and-openj9.html">Quick test to compare HotSpot and OpenJ9</a>. This time the main objective is to compare the Graal CE and Graal EE. The general message on the performance of Graal CE seems to be that it is roughly on par with HotSpot – better at some things and worse at others, but about the the same on average. Presumably Graal EE will perform better than HotSpot.<br />
<div>
<ul>
<li>JDK 1.8.0_161, Java HotSpot(TM) 64-Bit Server VM, 25.161-b12</li>
<li>JDK 1.8.0_192, GraalVM 1.0.0-rc12, 25.192-b12-jvmci-0.54 GraalVM CE</li>
<li>JDK 1.8.0_192, GraalVM 1.0.0-rc12, 25.192-b12-jvmci-0.54 GraalVM EE</li>
</ul>
<div>
<br />
Let's test those JVMs using code that will definitely benefit from auto-vectorization, matrix multiplication, and to test that using 3 commonly used pure Java linear algebra libraries:<br />
<ul>
<li>Apache Commons Math (ACM) v3.6.1 </li>
<li>Efficient Java Matrix Library (EJML) v0.37.1 </li>
<li>ojAlgo v47.0.1-SNAPSHOT </li>
</ul>
<div>
<br />
The test performs matrix-matrix multiplication on square dense matrices of various sizes: 100, 150, 200, 350, 500, 750 and 1000. The libraries have made different choices on how to store the elements and how to implement multiplication. In particular ojAlgo is multithreaded, and the others are not. The goal here is not to determine which library is the fastest, but rather to get an idea about what happens when you change the JVM.</div>
<br />
The chart below shows the speed (throughput, ops/min) for the various library/JVM combinations for each of the different matrix sizes.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7XRRutWnInTLaHbkzyUxPI0b4uB_yjn4oJ4fb7_QomffgyBF9-PMYhRJ1VaZLmz-eHyV9Io1yUNgOKXQplm22TjPfQstgC11D1LLRBPbnRo8hq-LenOLy3Zx2aDSkmuXJXtcV4FictLA/s1600/OracleJVMs.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="681" data-original-width="713" height="610" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7XRRutWnInTLaHbkzyUxPI0b4uB_yjn4oJ4fb7_QomffgyBF9-PMYhRJ1VaZLmz-eHyV9Io1yUNgOKXQplm22TjPfQstgC11D1LLRBPbnRo8hq-LenOLy3Zx2aDSkmuXJXtcV4FictLA/s640/OracleJVMs.png" width="640" /></a></div>
<br />
<h3>
Interpretation</h3>
The chart speaks for itself and the results are as expected. Graal CE is typically slower than HotSpot, and Graal EE faster. The difference between the CE and EE versions of Graal is significant.<br />
<br />
How much will Oracle charge for the EE version?<br />
<br />
In the previous post <a href="https://ojalgo.blogspot.com/2019/02/quick-test-to-compare-hotspot-and-openj9.html">Quick test to compare HotSpot and OpenJ9</a> you can see how Graal CE compares to <a href="https://www.eclipse.org/openj9/">OpenJ9</a>.<br />
<br />
<h3>
Test details</h3>
This test was executed on a Google Cloud Platform Compute Engine: n1-highmem-4 (4 vCPUs Intel Skylake Xeon, 26 GB memory).<br />
<br />
The benchmark code is here: <a href="https://github.com/optimatika/ojAlgo-linear-algebra-benchmark">https://github.com/optimatika/ojAlgo-linear-algebra-benchmark</a>
<br />
<br />
The raw results (*.log and *.csv files) can be found here: <a href="https://github.com/optimatika/ojAlgo-linear-algebra-benchmark/tree/develop/results/2019/02/oracles-jvms-hotspot-graal-ce-graal-ee">https://github.com/optimatika/ojAlgo-linear-algebra-benchmark/tree/develop/results/2019/02/oracles-jvms-hotspot-graal-ce-graal-ee</a><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
</div>
</div>
apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.com0tag:blogger.com,1999:blog-3235862046150994871.post-12618585968810695122019-02-07T22:17:00.000+01:002019-02-18T09:15:49.541+01:00Quick test to compare HotSpot and OpenJ9Benchmarking the HotSpot and OpenJ9 JVM:s doing matrix multiplication<br />
<div>
<br /></div>
<div>
AdoptOpenJDK comes in two flavours – one with the usual HotSpot JVM and one with OpenJ9. <br />
<ul>
<li>JDK 11.0.2, OpenJDK 64-Bit Server VM, 11.0.2+9 </li>
<li>JDK 11.0.2, Eclipse OpenJ9 VM, openj9-0.12.1 </li>
</ul>
<div>
After completing the first round of executions I decided to also add Graal.</div>
<div>
<ul>
<li>JDK 1.8.0_192, GraalVM 1.0.0-rc12, 25.192-b12-jvmci-0.54</li>
</ul>
</div>
Unfortunately I could only find builds based on OpenJDK 8, but the Graal JVM is the latest possible built just a couple of days ago.<br />
<br />
Which is faster? Most likely the answer to that depends on what you're testing. I decided to test matrix multiplication, and to test that using 3 commonly used pure Java linear algebra libraries.<br />
<ul>
<li>Apache Commons Math (ACM) v3.6.1 </li>
<li>Efficient Java Matrix Library (EJML) v0.37.1 </li>
<li>ojAlgo v47.0.1-SNAPSHOT </li>
</ul>
The test performs matrix-matrix multiplication on square dense matrices of various sizes: 100, 150, 200, 350, 500, 750 and 1000. The libraries have made different choices on how to store the elements and how to implement multiplication. In particular ojAlgo is multithreaded, and the others are not. The goal here is not to determine which library is the fastest, but rather to get an idea about what happens when you change the JVM.<br />
<br />
The chart below shows the speed (throughput, ops/min) for the various library/JVM combinations for each of the different matrix sizes.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicKX6wY_IVQ1yQ3ChzK7gCs06RJ7jIPljtAI31hy_InZHkKw9zdbg5fySwMPrt6j7X2EMVyDaztY_mNZNu25ks2svqmf080ItzS84gP6WU-oKCnf4jgki2dc9PSLVayO1yyVIxUgFf7MU/s1600/HotSpotOpenJ9Graal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="946" data-original-width="990" height="609" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicKX6wY_IVQ1yQ3ChzK7gCs06RJ7jIPljtAI31hy_InZHkKw9zdbg5fySwMPrt6j7X2EMVyDaztY_mNZNu25ks2svqmf080ItzS84gP6WU-oKCnf4jgki2dc9PSLVayO1yyVIxUgFf7MU/s640/HotSpotOpenJ9Graal.png" width="640" /></a></div>
<br />
<h3>
Interpretation</h3>
<b>OpenJ9:</b> Switching from HotSpot to OpenJ9 makes ojAlgo faster, ACM slower and EJML fluctuates between roughly the same performance and much slower. It seems OpenJ9 is less stable in its results than HotSpot. Apart from the fluctuations with EJML it widens the gap between the slowest and the fastest code. OpenJ9 makes the fast even faster and the slow even slower.<br />
<br />
<b>Graal:</b> Graal was consistently slower than HotSpot (half speed). This is the community edition (CE) of Graal. I've learned that (full) support for vectorization will only be in the enterprise edition (EE). I'm guessing that the EE will give performance similar to that of HotSpot.<br />
<br />
The speed differences shown here are significant! Regardless of library and matrix size, performance could be halved or doubled by choosing another JVM. Looking at combinations of libraries and JVM:s there is an order of magnitude in throughput to be gained by choosing the right combination.<br />
<br />
This will not translate to an entire system/application being this much slower or faster – matrix multiplication is most likely not the only thing it does. But, perhaps you should test how whatever you're working on performs with a different JVM.<br />
<h3>
</h3>
<h3>
Test details</h3>
This test was executed on a Google Cloud Platform Compute Engine: n1-highmem-4 (4 vCPUs Intel Skylake Xeon, 26 GB memory).<br />
<br />
The benchmark code is here: <a href="https://github.com/optimatika/ojAlgo-linear-algebra-benchmark">https://github.com/optimatika/ojAlgo-linear-algebra-benchmark</a>
<br />
<br />
The raw results (*.log and *.csv files) can be found here: <a href="https://github.com/optimatika/ojAlgo-linear-algebra-benchmark/tree/develop/results/2019/02/quick-test-to-compare-hotspot-and-openj9">https://github.com/optimatika/ojAlgo-linear-algebra-benchmark/tree/develop/results/2019/02/quick-test-to-compare-hotspot-and-openj9</a><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
</div>
apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.com0tag:blogger.com,1999:blog-3235862046150994871.post-70310580198832466612018-09-30T16:03:00.001+02:002018-10-02T21:19:54.063+02:00Neural Network Basics<div>
A <a href="https://ojalgo.blogspot.com/2018/09/introducing-artificial-neural-networks.html">previous post</a> introduced ojAlgo's Artificial Neural Network feature. It did so by presenting a fully functional program that trained and evaluated a network model to categorise handwritten digits using the <a href="http://yann.lecun.com/exdb/mnist/">MNIST</a> data set. That example then included all necessary pre and post processing as well as code that would generate the actual image (*.png) files. All of that may have obscured just how simple it is to work with ojAlgo's neural networks.
</div>
<br />
<div>
The code below is not complete – it simply outlines the basic steps involved to build, train and use a neural network with ojAlgo.
</div>
<br />
<div>
<script src="https://gist.github.com/apete/b3278dc2f8c2db6a00369c211ba321db.js?file=OutlineNeuralNetworkUsage.java"></script>
</div>
<br />
<br />
<div>
Now have a look at that <a href="https://ojalgo.blogspot.com/2018/09/introducing-artificial-neural-networks.html">fully functional example</a>...
</div>
<br />
<br />apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.com2tag:blogger.com,1999:blog-3235862046150994871.post-73604342492564335072018-09-16T15:42:00.000+02:002018-09-22T17:52:17.017+02:00Introducing Artificial Neural Networks with ojAlgoWith v46 ojAlgo got support for building artificial neural networks. Here's an example of what you can do with it.<br />
<br />
The MNIST database is a large image database of handwritten digits that is commonly used for training and testing in the field of machine learning / image categorisation. Information regarding this dataset and various results achieved is widely published.<br />
<ul>
<li><a href="http://yann.lecun.com/exdb/mnist/">http://yann.lecun.com/exdb/mnist/</a></li>
<li><a href="https://en.wikipedia.org/wiki/MNIST_database">https://en.wikipedia.org/wiki/MNIST_database</a></li>
</ul>
<div>
A correctly modelled/trained neural network should be able to achieve a 5% error rate on this dataset. Most/all published results are better than that. The largest, most advanced, models have managed 0,35%. That's almost unbelievably good. ojAlgo currently doesn't have all the features required to build that kind of model. The model in the program listed below gets about 2.2% error rate. Here are some sample digits/images from the data set.<br />
<br />
<div style="text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqzwFHmwJg9hXDl2V5xDa415cHqUui8w1cwDz6ckBfOcyLrO7JNhNdTpOkD4CCN3OeCREVnD-9KxIEcHK2Cj1kVIUFLvJ06Wml0dm6UpDLc5itNd933hhC4l8IhJHstpdRn-2nSg9rYrg/s1600/100000_5.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqzwFHmwJg9hXDl2V5xDa415cHqUui8w1cwDz6ckBfOcyLrO7JNhNdTpOkD4CCN3OeCREVnD-9KxIEcHK2Cj1kVIUFLvJ06Wml0dm6UpDLc5itNd933hhC4l8IhJHstpdRn-2nSg9rYrg/s1600/100000_5.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguCwobJ99srR96kxnXKzrtY-ksN1QJSGGgwfrp9XAfJg_tmzwNQ2DAwyXQbgKonx9nCdrUbMuhQKWMbEzDUHJ1Hd5tn_2n5h950PitcCSCq8wWOu0hY8FBZMvvmIY97PvYnPD1p4TdK54/s1600/100001_0.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguCwobJ99srR96kxnXKzrtY-ksN1QJSGGgwfrp9XAfJg_tmzwNQ2DAwyXQbgKonx9nCdrUbMuhQKWMbEzDUHJ1Hd5tn_2n5h950PitcCSCq8wWOu0hY8FBZMvvmIY97PvYnPD1p4TdK54/s1600/100001_0.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiS2b1tH_gzxta4MjefTGkz8-UVI-yjgdoXcJiVWzXru2yJEp2DAD3AZYvhbv2PDR2UemdfYrLHJpt-unG345_Rzt5Y3tWgRRe1MOomgd8qBxA2jasIYDnMhg9tXN6q2YKc4nN_5whsPjI/s1600/100002_4.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiS2b1tH_gzxta4MjefTGkz8-UVI-yjgdoXcJiVWzXru2yJEp2DAD3AZYvhbv2PDR2UemdfYrLHJpt-unG345_Rzt5Y3tWgRRe1MOomgd8qBxA2jasIYDnMhg9tXN6q2YKc4nN_5whsPjI/s1600/100002_4.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrovu4h0H9xNjkhDgfq71wHAgYjmhsIJL4U9FrSAEnaJ9XdTJB2ZR6kZaj-jTnGyjj6FH9dxiY63Gghvc7KWtEnoVX76GOwKz1mzDse6ufQyRaLNGkxxBkk-foPdiNqHPH5aPy3ym7SNM/s1600/100003_1.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrovu4h0H9xNjkhDgfq71wHAgYjmhsIJL4U9FrSAEnaJ9XdTJB2ZR6kZaj-jTnGyjj6FH9dxiY63Gghvc7KWtEnoVX76GOwKz1mzDse6ufQyRaLNGkxxBkk-foPdiNqHPH5aPy3ym7SNM/s1600/100003_1.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYCxhXYCcj2a0kEPhgsIZpfDtydxyh3w9vqMhf47cYuCwWTNEnXCpbyd7RsEFDV0wMaYCPVheWtNrsFyh5Q4UTGOMK0xvECENwijA0-lwTvH1zvfAxGagdRaKzrERLLaZi4QXI8j8XGDc/s1600/100004_9.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYCxhXYCcj2a0kEPhgsIZpfDtydxyh3w9vqMhf47cYuCwWTNEnXCpbyd7RsEFDV0wMaYCPVheWtNrsFyh5Q4UTGOMK0xvECENwijA0-lwTvH1zvfAxGagdRaKzrERLLaZi4QXI8j8XGDc/s1600/100004_9.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTP2Ma42wovT8C8_erJUpbW4fW_SplNUMMW3132nlO9KaIgnS3iwqkxlU7CwkoHkEbeGNaKVC1fyQVvVmr8ZJG0hmW7qt9tvmjhmQ4AhvT6AFczbLPBdRy34mxlY9uKbOabfMkCI9xS6U/s1600/100005_2.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTP2Ma42wovT8C8_erJUpbW4fW_SplNUMMW3132nlO9KaIgnS3iwqkxlU7CwkoHkEbeGNaKVC1fyQVvVmr8ZJG0hmW7qt9tvmjhmQ4AhvT6AFczbLPBdRy34mxlY9uKbOabfMkCI9xS6U/s1600/100005_2.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkYb0DrsDUCVgNmd4C6t-Hdw0Xe-TYBqLLBPXYijHZexzAHUcaTavUj9jTHJhHfPZIpT8tPI1mYKLB_mNBTjH0PtdiqrGjkKu_Ei4pMqPnc1SBq0YqXqXR_4_o0iyK51pzP9FENo57bBs/s1600/100006_1.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkYb0DrsDUCVgNmd4C6t-Hdw0Xe-TYBqLLBPXYijHZexzAHUcaTavUj9jTHJhHfPZIpT8tPI1mYKLB_mNBTjH0PtdiqrGjkKu_Ei4pMqPnc1SBq0YqXqXR_4_o0iyK51pzP9FENo57bBs/s1600/100006_1.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRvcOvQI4efjN9bJhXR6S3UYp2mQB40-4-G7slTUL5sGExRAN_Asm987wBpWMTSW9hgAPldqUf2JkjD9U2jwupjMUDWMozFO1MOAlymiiYUotwsQH53Wgxwaxa6bDZuHCmEIYvnkl63GU/s1600/100007_3.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRvcOvQI4efjN9bJhXR6S3UYp2mQB40-4-G7slTUL5sGExRAN_Asm987wBpWMTSW9hgAPldqUf2JkjD9U2jwupjMUDWMozFO1MOAlymiiYUotwsQH53Wgxwaxa6bDZuHCmEIYvnkl63GU/s1600/100007_3.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHa8x5Ai4gf_kSxQx8OHjNc0HCOizqclkZf8FskfuTWo4hgb6a4CMfJ7QSP8bW_kEEx7YH42SYe7ys0NaJ1nc0eqdykOecACGJXlI1bel431XlKMhtILYmRYxSUNvXqnIiVCOaUazdKg0/s1600/100008_1.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHa8x5Ai4gf_kSxQx8OHjNc0HCOizqclkZf8FskfuTWo4hgb6a4CMfJ7QSP8bW_kEEx7YH42SYe7ys0NaJ1nc0eqdykOecACGJXlI1bel431XlKMhtILYmRYxSUNvXqnIiVCOaUazdKg0/s1600/100008_1.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzqWPC7e5IX84UmZ_rNz1p4ASxJKVzfijU0CCIyFRl5FcAOjC_jAA4_lhhikv0K59908EQKyV5xfBQnTF_-7hYtKDOw3qhuvuJO4-KjyPlu8bFruIvWCvKtUOevG_vgUMbA7uIHbpibHY/s1600/100009_4.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzqWPC7e5IX84UmZ_rNz1p4ASxJKVzfijU0CCIyFRl5FcAOjC_jAA4_lhhikv0K59908EQKyV5xfBQnTF_-7hYtKDOw3qhuvuJO4-KjyPlu8bFruIvWCvKtUOevG_vgUMbA7uIHbpibHY/s1600/100009_4.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEga4n_EpVmr7YkGUODGtFtFLa8HakjCULHLci0vVGhH8js7gaov8xhvma6Mr1OtZsexGC8lBHbtRzAd_7Iu_P5-T8UyJsuLbBQxH8DV1SuOIn9mdrzL6RyEGU-Gk4DE9joO-9_L1114heA/s1600/100010_3.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEga4n_EpVmr7YkGUODGtFtFLa8HakjCULHLci0vVGhH8js7gaov8xhvma6Mr1OtZsexGC8lBHbtRzAd_7Iu_P5-T8UyJsuLbBQxH8DV1SuOIn9mdrzL6RyEGU-Gk4DE9joO-9_L1114heA/s1600/100010_3.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9M1QvoX7uez9v0gQ7CZfLTbA2wfGq2xezcR64qMUPuq0NWQGXhCeWBAS4eZZduTxECtRM2wG2e3JgprHcrk1JGZqkxGCmCMA1aSFQR0dbqeQb4-blkDWZA6DVb7lJjCPrDdxsol5cGRE/s1600/100011_5.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9M1QvoX7uez9v0gQ7CZfLTbA2wfGq2xezcR64qMUPuq0NWQGXhCeWBAS4eZZduTxECtRM2wG2e3JgprHcrk1JGZqkxGCmCMA1aSFQR0dbqeQb4-blkDWZA6DVb7lJjCPrDdxsol5cGRE/s1600/100011_5.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpsBZfPSPr4HN-SdAPOljj7saPXKl5neuLCFRQXXf7ciqozt1ulhJRzAo8nzebOh9mvv9Z_PseKLh0PU0aCys2DhrjzUpoTrsOOA6mKMsyUrHh7wHSniOGsu038_32s1-XhoDHBWoog4A/s1600/100012_3.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpsBZfPSPr4HN-SdAPOljj7saPXKl5neuLCFRQXXf7ciqozt1ulhJRzAo8nzebOh9mvv9Z_PseKLh0PU0aCys2DhrjzUpoTrsOOA6mKMsyUrHh7wHSniOGsu038_32s1-XhoDHBWoog4A/s1600/100012_3.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHBddj_Irul4AeI2l0cy1ZBCE5SxSN4PAMOGnvnc-KssWEtLRnzxdusNKxCdP71ttVn30BDev_OgK5U0zBgqXmbZoPPL474Ua_k1_hBk5Lnfu8z1pn536EVFDc_n29VbySb5FXgacyaP0/s1600/100013_6.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHBddj_Irul4AeI2l0cy1ZBCE5SxSN4PAMOGnvnc-KssWEtLRnzxdusNKxCdP71ttVn30BDev_OgK5U0zBgqXmbZoPPL474Ua_k1_hBk5Lnfu8z1pn536EVFDc_n29VbySb5FXgacyaP0/s1600/100013_6.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDkaUTxQw-9fp5izov0x8v32eU8jjIqfz2tmjJ7xfYpwf-jiD_RhUDXm7uzFhjiE_mvHQRX-SjMTQWM-EGfdTdhZiLx3z3yJJa30q2fQNWJ9MvaWEDx5FEfqi1dvYT6zmdy6cJmBiZnMo/s1600/100014_1.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDkaUTxQw-9fp5izov0x8v32eU8jjIqfz2tmjJ7xfYpwf-jiD_RhUDXm7uzFhjiE_mvHQRX-SjMTQWM-EGfdTdhZiLx3z3yJJa30q2fQNWJ9MvaWEDx5FEfqi1dvYT6zmdy6cJmBiZnMo/s1600/100014_1.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKnTAQvYXBTTnraYnrh3KCH996kZ9Xx-swCMCxV2T5v3oUKC64BKMc_X-ZpKkjSES6p7UEgRhxCp8lSLC-kw62okCRSmrn1gMtFzBb9N1iOhrEEvR_2Xq9-PNwRR2Pfsre9astDGRF0jg/s1600/100015_7.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKnTAQvYXBTTnraYnrh3KCH996kZ9Xx-swCMCxV2T5v3oUKC64BKMc_X-ZpKkjSES6p7UEgRhxCp8lSLC-kw62okCRSmrn1gMtFzBb9N1iOhrEEvR_2Xq9-PNwRR2Pfsre9astDGRF0jg/s1600/100015_7.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNQzJJVrVMusDF2LmTMcmwhJvrwznhrIyJJ_rshK320egAkZ7XrvV7OcWSeq3meG4e_895u_-e4d_PCKbcqpv7dwkam2aU-UM87_pgnfdpqmSB8Ex4xE5NCzPT0maPrwYEsQZsGfvffjM/s1600/100016_2.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNQzJJVrVMusDF2LmTMcmwhJvrwznhrIyJJ_rshK320egAkZ7XrvV7OcWSeq3meG4e_895u_-e4d_PCKbcqpv7dwkam2aU-UM87_pgnfdpqmSB8Ex4xE5NCzPT0maPrwYEsQZsGfvffjM/s1600/100016_2.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYBfCq7VEqF-T7DKslHzDn2OE_6S4S84Drod3gjIS-EDGvjx_4B2GehS1dBmX9Uvr9ziqpxIUMlqv89awRTtu_AT9w2OHqbYy57IC6s2YB1FQbNmeGFjcIqZqnsRm9hPfWYD7wvhVBMRs/s1600/100017_8.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYBfCq7VEqF-T7DKslHzDn2OE_6S4S84Drod3gjIS-EDGvjx_4B2GehS1dBmX9Uvr9ziqpxIUMlqv89awRTtu_AT9w2OHqbYy57IC6s2YB1FQbNmeGFjcIqZqnsRm9hPfWYD7wvhVBMRs/s1600/100017_8.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg458LOMhxi8jTQwLd8pNzCNhk_sNq8HnSK8mzj031oWcUekebl5sALjCVvJMfdSbitwwt6ta0mnDTROiYp1sCex4czztfGQWhlS0Vw5KMJrygOw4iIT-NehYbwte0GPhwFks5nwzplSY4/s1600/100018_6.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg458LOMhxi8jTQwLd8pNzCNhk_sNq8HnSK8mzj031oWcUekebl5sALjCVvJMfdSbitwwt6ta0mnDTROiYp1sCex4czztfGQWhlS0Vw5KMJrygOw4iIT-NehYbwte0GPhwFks5nwzplSY4/s1600/100018_6.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_0TlKPtgZNmRgxzt7tDay58MSMCWuRK7LP6-aFQrNQXPMKKOeTBMPzk1yYEJP83jriI0B9wYGhe0czp4oyEVwRdWTYcKMrJ5iGob-lHRAMJBFNLJoXsEmCCAl_OqXCymWAJ0kLB8augI/s1600/100019_9.png"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_0TlKPtgZNmRgxzt7tDay58MSMCWuRK7LP6-aFQrNQXPMKKOeTBMPzk1yYEJP83jriI0B9wYGhe0czp4oyEVwRdWTYcKMrJ5iGob-lHRAMJBFNLJoXsEmCCAl_OqXCymWAJ0kLB8augI/s1600/100019_9.png" /></a></div>
<br />
<br />
The program below (with its dependency on ojAlgo) can do the following:<br />
<br />
<ul>
<li>Read/parse the files containing the image data and labels.</li>
<li>Generate the actual images so that you can inspect them. The example images above are generated with that code.</li>
<li>Print images to console (to sanity check results)</li>
<li>Model and train feedforward neural networks:</li>
<ul>
<li>Any number of layers</li>
<li>Any number of input/output nodes per layer</li>
<li>Choose between 5 different activator and 2 different error/loss functions </li>
</ul>
</ul>
<div>
<br />
The main benefit of using ojAlgo is how easy it is to do this and get good results. Download the example code below (you also need ojAlgo v46.1.1 or later) and run it, and start modifying the network structure, learning rate and other things. (You also need to download the data files, and update the various paths in the programs.)</div>
</div>
<br />
<div>
<script src="https://gist.github.com/apete/b3278dc2f8c2db6a00369c211ba321db.js?file=TrainingANN.java"></script>
</div>
apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.com0tag:blogger.com,1999:blog-3235862046150994871.post-40840837597497558792018-09-14T20:13:00.000+02:002018-09-14T20:13:30.815+02:00ojAlgo ExamplesAll ojAlgo example code previously published on the ojAlgo wiki has now been moved to a GitHub Gist. On this page you see the complete contents of that gist. (Never mind the eclipse and git files at the top. Scroll down the page.) When/if anything is added to that gist this page is automatically updated.<br />
<br />
<h3>Code</h3>
<br />
<div>
<script src="https://gist.github.com/apete/b3278dc2f8c2db6a00369c211ba321db.js"></script>
</div>apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.com0tag:blogger.com,1999:blog-3235862046150994871.post-57052388071492399822018-05-15T13:44:00.000+02:002018-05-21T12:24:11.696+02:00New Java Matrix Benchmark Results ComingI'm in the process of executing the Java Matrix Benchmark on a Google Cloud Platform Compute Engine (n1-highmem-4 (4 vCPUs, 26 GB memory) with Intel Skylake processors). Actually I'm doing 3 separate benchmark executions using 3 different JVM:s on otherwise identical machines: <br />
<ol>
<li>Java(TM) SE Runtime Environment (build 1.8.0_161-b12) </li>
<li>Java(TM) SE Runtime Environment (build 9.0.4+11) </li>
<li>Zing Runtime Environment for Java Applications (build 1.8.0-zing_18.04.0.0-b2) </li>
</ol>
When all is done it'll be very interesting to compare the results between the JVM:s.<br />
<br />
The results for all pure Java libraries running on Oracle 8 are already complete. Here's the summary performance chart for that:<span style="font-family: inherit;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3BZvx2iZm-vtpGtZWX9o2pbiP4EFXSv1O3adU3UJcKcgZy0KFIi74uaZ8jOda5yE6H-xokmMF4BSK4SiO0GH5cBD4-nlGWm0Vt6Kk7gBbWIg8agH_39MPmOIApSIuuADphwz3mwQVCUI/s1600/summary_oracle8.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="937" data-original-width="1354" height="441" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3BZvx2iZm-vtpGtZWX9o2pbiP4EFXSv1O3adU3UJcKcgZy0KFIi74uaZ8jOda5yE6H-xokmMF4BSK4SiO0GH5cBD4-nlGWm0Vt6Kk7gBbWIg8agH_39MPmOIApSIuuADphwz3mwQVCUI/s640/summary_oracle8.png" width="640" /></a></div>
<br />
<div>
<br /></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
It'll be another 2 weeks or so before everything is done. By then there will be results for these library and JVM combinations:<br />
<table style="width: 100%;">
<tbody>
<tr><th><br /></th><th><br /></th><th>Oracle 8</th><th>Oracle 9</th><th>Zing 8</th></tr>
<tr><td><a href="https://dst.lbl.gov/ACSSoftware/colt/" target="_blank">Colt</a></td><td>1.2.0</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="http://commons.apache.org/proper/commons-math/" target="_blank">Apache Commons Math</a></td><td>3.6.1</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="http://ejml.org/" target="_blank">EJML</a></td><td>0.33</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="https://math.nist.gov/javanumerics/jama/" target="_blank">JAMA</a></td><td>1.0.3</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="http://jblas.org/" target="_blank">jblas</a> (using native code)</td><td>1.2.4</td><td><div style="text-align: center;">
<br /></div>
</td><td><div style="text-align: center;">
X</div>
</td><td><br /></td></tr>
<tr><td><a href="http://la4j.org/" target="_blank">la4j</a></td><td>0.6.0</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="https://github.com/andreas-solti/matrix-toolkits-java" target="_blank">MTJ</a> (pure Java)</td><td>1.0.7</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="https://github.com/andreas-solti/matrix-toolkits-java" target="_blank">MTJ-N</a> (using native code)</td><td>1.0.7</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
<br /></div>
</td></tr>
<tr><td><a href="http://ojalgo.org/" target="_blank">ojAlgo</a></td><td>45.0.0</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="https://sites.google.com/site/piotrwendykier/software/parallelcolt" target="_blank">Parallel Colt</a></td><td>0.9.4</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="https://ujmp.org/" target="_blank">UJMP</a> (pure Java)</td><td>0.3.0</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td></tr>
<tr><td><a href="https://ujmp.org/" target="_blank">UJMP-N</a> (using native code)</td><td>0.3.0</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
X</div>
</td><td><div style="text-align: center;">
<br /></div>
</td></tr>
</tbody></table>
<br />
<div>
The jblas library (using native code) crashed the benchmark twice running on different JVM:s. As part of the plan for how to recover and resume the executions it was decided to not re-run the libraries using native code on the all the instances – to save time. The native code libraries are not the most interesting when comparing JVM:s.<br />
<br /></div>
<div>
Further, and more detailed, results will be published here, at the ojAlgo wiki and/or at the Java Matrix Benchmark web site:<br />
<ul style="font-family: sans-serif;">
<li style="font-family: serif;"><a href="https://ojalgo.blogspot.com/" title="https://ojalgo.blogspot.com">https://ojalgo.blogspot.com</a></li>
<li style="font-family: serif;"><a href="https://github.com/optimatika/ojAlgo/wiki/Java-Matrix-Benchmark" title="https://github.com/optimatika/ojAlgo/wiki/Java-Matrix-Benchmark">https://github.com/optimatika/ojAlgo/wiki/Java-Matrix-Benchmark</a></li>
<li style="font-family: serif;"><span style="color: #0000ee;"><a href="https://lessthanoptimal.github.io/Java-Matrix-Benchmark/" title="https://lessthanoptimal.github.io/Java-Matrix-Benchmark/">https://lessthanoptimal.github.io/Java-Matrix-Benchmark/</a></span></li>
</ul>
</div>
apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.com0tag:blogger.com,1999:blog-3235862046150994871.post-83303188591512927102018-03-06T22:12:00.000+01:002019-02-01T15:07:42.585+01:00Matrix multiplication on different JVM:s<div>
<br /></div>
Most developers are aware of the possibility to use either a JVM from Oracle or "just" OpenJDK, but few consider using any of the other alternatives. There are more alternatives! <br />
<br />
<div>
<a href="https://en.wikipedia.org/wiki/List_of_Java_virtual_machines">https://en.wikipedia.org/wiki/List_of_Java_virtual_machines</a><br />
<br />
What if choosing one of the alternatives could significantly improve the performance of your application?<br />
<br />
I frequently see discussions and benchmarks comparing different libraries (that solve the same problem). Comparing Java performance to that of native code is also common, but I don't think I've ever seen a benchmark comapring different JVM:s. I decided to do this - I decided to run the Java Matrix Benchmark using both the Oracle JVM and Zing from Azul Systems.<br />
<br />
I haven't done it yet, but as part of the preparations, I've executed some smaller tests. Below you'll find the results from benchmarking matrix multiplication using 3 different pure Java linear algebra libraries on both the Oracle JVM and Azul's Zing.<br />
<br />
<div style="text-align: center;">
<a href="http://ojalgo.org/downloads/multiplication_o8_z8_o9_1000.png"><img border="0" height="416" src="https://blogger.googleusercontent.com/img/proxy/AVvXsEjIHydEobFS5bs7dRWqGvmhR8CBx6_KIUUzxoyFYKzzIWJxsEws6gvQGtzuJ_qVr9tGQ4ZVamS1AT-Nh2UM5A7OwGvmHqiNvE-BLNw59V_j3ls_2hJL2-Nd0A8arGf8oFFIJx2FwgwYoQBMm4a-nIuwB5RRPF7NmewnzPQN=" width="640" /></a></div>
<br />
This chart shows the relative performance on matrix multiplication of 3 different Java linear algebra libraries, using 2 different JVM:s. The x-axis shows the (square) matrix size, and the y-axis the relative performance.<br />
<h3>
</h3>
<h3>
</h3>
<h3>
<br /></h3>
<h3>
Which 3 Java linear algebra libraries? </h3>
<ul>
<li>ojAlgo v45.0.0-SNAPSHOT </li>
<li>EJML v0.33 </li>
<li>Apache Commons Math (ACM) v3.6.1</li>
</ul>
<div>
ojAlgo is the only multi-threaded library among these.</div>
<h3>
</h3>
<h3>
</h3>
<h3>
<br /></h3>
<h3>
Which 2 JVM:s? </h3>
<ul>
<li>JDK 1.8.0_161, VM 25.161-b12 (Oracle 8) </li>
<li>JDK 1.8.0-zing_17.12.1.0, VM 1.8.0-zing_17.12.1.0-b1-product-azlinuxM-X86_64 (Zing 8) </li>
</ul>
<h3>
</h3>
<h3>
</h3>
<h3>
<br /></h3>
<h3>
Relative to what?</h3>
Actually a 4:th linear algebra library was benchmarked – Matrix Toolkits Java (MTJ) v1.0.7. It is based on netlib-java which use native code libraries if available. The machine in question had some plain/default version of ATLAS installed which was used. (just did 'sudo yum install atlas'). Since the actual matrix multiplication is done in native code the performance was essentially identical between the different JVM:s. This was used as a reference.<br />
<div>
<br /></div>
<div>
"1.0" on the y-axis represents the speed of MTJ/netlib-java/ATLAS. Then for the other library and JVM combinations you can see how much faster or slower they where.<br />
<h3>
</h3>
<h3>
</h3>
<h3>
<br /></h3>
<h3>
Interpreting the results</h3>
These are my conclusions:<br />
<ol>
<li>The choice of JVM does make a difference (it did in this case) and Oracle is not necessarily the best option.</li>
<li>Performance differences between the JVM:s are of the same magnitude as between different code libraries.</li>
<li>Depending on which code library you use, the effects of changing JVM:s vary – you need to look at the individual combinations.</li>
<li>ojAlgo is the only multi-threaded library here. This allows it to be more than 4 times faster than the other libraries (with the larger matrices). In those cases there is also a significant difference between Oracle and Zing – it's another 4x speed improvement. This would be vectorisation happening in the JVM. Modern JVM:s have support to automagically utilise SIMD instructions. This feature is brittle, it doesn't always happen! It seems Zing agreed with ojAlgo and managed to vectorise the code, where Oracle did not.</li>
<li>Java is fast enough to sometimes be faster than native. In this case the native code library is not the fastest possible. I believe ATLAS, that was used in this case, can be tuned to perform better than it did here, and there are other alternatives (perhaps to purchase an Intel MKL license). But, unless you do something to ensure the best possible native performance, a good Java library may be faster (as shown here).</li>
</ol>
<div>
Look at that chart again. For the larger matrix sizes there is almost a <b>100x speed increase</b> between the slowest and the fastest library/JVM combination.</div>
<h3>
</h3>
<h3>
</h3>
<h3>
<br /></h3>
<h3>
What kind of machine was this run on?</h3>
The benchmark was executed on a Google Cloud Platform n1-standard-8 (8 vCPUs, 30 GB memory) with Intel Skylake processors.<br />
<h3>
</h3>
<h3>
</h3>
<h3>
<br /></h3>
<h3>
Full execution output logs</h3>
</div>
<div>
The same benchmark was executed 3 times using different JVM:s.<br />
<ul>
<li><a href="http://ojalgo.org/downloads/oracle8_multiply.log">Oracle 8</a> </li>
<li><a href="http://ojalgo.org/downloads/zing8_multiply.log">Zing 8</a> </li>
<li><a href="http://ojalgo.org/downloads/oracle9_multiply.log">Oracle 9</a> </li>
</ul>
<h3>
</h3>
<h3>
</h3>
<h3>
<br /></h3>
<h3>
The benchmark software</h3>
<a href="https://github.com/optimatika/ojAlgo-linear-algebra-benchmark" target="_blank">https://github.com/optimatika/ojAlgo-linear-algebra-benchmark</a></div>
</div>
<!-- Blogger automated replacement: "https://images-blogger-opensocial.googleusercontent.com/gadgets/proxy?url=http%3A%2F%2Fojalgo.org%2Fdownloads%2Fmultiplication_o8_z8_o9_1000.png&container=blogger&gadget=a&rewriteMime=image%2F*" with "https://blogger.googleusercontent.com/img/proxy/AVvXsEjIHydEobFS5bs7dRWqGvmhR8CBx6_KIUUzxoyFYKzzIWJxsEws6gvQGtzuJ_qVr9tGQ4ZVamS1AT-Nh2UM5A7OwGvmHqiNvE-BLNw59V_j3ls_2hJL2-Nd0A8arGf8oFFIJx2FwgwYoQBMm4a-nIuwB5RRPF7NmewnzPQN=" -->apetehttp://www.blogger.com/profile/07480681674118054617noreply@blogger.com0