13.08.2014 ALGLIB Project publishes technical report on StrongNet
Our technical report proposes two innovations in the design of neural networks: (a) strong neurons – highly nonlinear/nonsmooth neurons with multiple outputs and (b) mostly unsupervised architecture – backpropagation-free design with all layers except for the last one being trained in a completely unsupervised setting.
The new neural design (called StrongNet) was tested on a well-known MNIST benchmark. This demonstrated that StrongNet is able to capture structural information about images in the "mostly unsupervised" setting. Three-layer StrongNet has a test set error on MNIST as low as 1.1%. It outperforms 11 out of 14 two/three-layer networks cited on the MNIST homepage (as of Aug 2014).
read full report (PDF)
12.03.2014 New article - "ALGLIB+MKL: first results"
In November (2013) we presented new product - ALGLIB 3.8.1 (Commercial Edition), the first ALGLIB version which was shipped with Intel MKL - high-performance numerical library by Intel. It was first release which featured C# interface to native computational core, thus both C++ and C# users were able to use MKL as part of ALGLIB.
MKL-capable ALGLIB is a significant step in ALGLIB development, result of technical partnership with Intel Numerics Russia. Now, after 4 months with new ALGLIB, we decided to publish an article which discusses benefits MKL brings to ALGLIB.
25.11.2013 ALGLIB 3.8.2 is released
This release was made in less than 3 weeks since previous one, and includes only one improvement in ALGLIB for C++ – better, smoother integration with Intel MKL. Only C++ version of Commercial Edition was changed, and only this change was made, so if you use Free Edition (or anything other than C++), you may ignore this release.
Experimental support for Intel MKL was introduced in ALGLIB 3.8.1, which was released in the beginning of November. Both C++ and C# versions of ALGLIB started to use MKL. ALGLIB for C# was favorably received by customers, but C++ users reported about two inconveniences. Both of them due to the fact that MKL-capable version of ALGLIB for C++ was delivered as precompiled static library (binary addition to the main source code distribution).
First, several users reported about linking problems with combined ALGLIB+MKL static library – distribution of precompiled C++ libraries has a lot of tricky issues. Second, some of our users already own license for MKL, and they wanted to use MKL and ALGLIB together in one application – but they wanted ALGLIB to use their own installation of MKL.
As result, in new ALGLIB for C++ we changed distribution model. ALGLIB itself is distributed as source-only, and again you should compile it yourself – with your own compiler, own options, and perfect compatibility with your build environment. If you want to use Intel MKL, you have two options here...
First, you may link with special lightweight Intel MKL distribution, which is shipped with ALGLIB for C++ as DLL file. In this case you do not have to buy separate license for MKL, but this MKL distribution can be used only as part of the ALGLIB. Second option is to link ALGLIB with your own MKL installation (you should have license for Intel MKL). In this case you may use Intel MKL both directly – and as part of ALGLIB.
Both options involve no precompiled static libraries. In the first case the only library you should use is import library, which is portable across different compilers. In the second case you should just add one more C file (ALGLIB-MKL interface) to your application, so again there should be no compatibility issues.
07.11.2013 ALGLIB 3.8.1 is released
ALGLIB 3.8.1 is released! This new release of ALGLIB features two major improvements in Commercial Edition: 1) C# interface to highly optimized native computational core, 2) experimental integration with Intel MKL.
1. Prior to 3.8.1, ALGLIB for C# was 100% NET application. Being pure NET assembly, it was portable and easy to integrate into existing NET projects. However, on some important computational problems (linear algebra, large scale data mining) C# performance was 2x-4x times slower than that of optimized native code.
Starting from 3.8.1, ALGLIB for C# includes both pure NET computational core – and its native companion, highly optimized native core with C# interface. Both computational cores provide same functionality, and even more – they provide 100% identical C# interface. You can easily plug HPC core into existing applications which use previous versions of ALGLIB. As for ALGLIB 3.8.1, HPC core can be used only under Windows systems. However, we plan to introduce support for LSB-compliant Linux systems in the next release. One more limitation – HPC core is not included in ALGLIB for VB.NET, although in next ALGLIB release VB users will have same level of performance as C# programmers.
2. Second important improvement is integration with Intel MKL, which can be accessed from C++ and C#. Intel MKL is used internally to accelerate linear algebra algorithms. We obtained developer's license from Intel which allows us to ship precompiled ALGLIB binaries linked with Intel MKL.
C++ version of ALGLIB now includes precompiled static library
alglibxx_hpc.lib (ALGLIB core),
mkl4alglibxx.lib (MKL imports) and
mkl4alglibxx.dll (MKL core).
When you link your application with these precompiled static/dynamic libraries,
ALGLIB will use MKL internally to provide same interface –
but with 2x-4x (depending on hardware being used) increase in linear algebra performance.
As for C#, this version of ALGLIB includes native computational core (as we told before).
It is already linked with Intel MKL, so you don't need additional steps in order to use MKL from C#.
We want to make several notes here. First, due to licensing restrictions, you can not use Intel MKL directly. You can use ALGLIB functionality which is internally accelerated by MKL's functions, but you can not access these functions directly. We do not provide you MKL header files because we do not have right to do so. Second, because MKL support is in the experimental stage, for now we ship only Windows libraries. Linux binaries are planned to be included in the next release of ALGLIB.
Finally, during implementation of this subproject we received several important suggestions and ideas from developers of local Intel's office, and we would like to thank them (especially Gennady Fedorov and Sergey Maidanov) for their help.
3. These improvements can be used only by commercial users who own license for new multi-threaded branch of ALGLIB. If you bought your license before release of 3.8.0 and didn't upgraded your license yet, it is good time to consider an upgrade. Upgrade is done for free, the only thing we need from you is a formal confirmation that you agree with new licensing terms.
In addition to improvements mentioned above, we made several minor fixes in both Commercial and Free editions of ALGLIB and fixed one bug in the MinBLEIC optimizer (of course, bug fixes were included in all editions – commercial and free).
07.08.2013 ALGLIB 3.8.0 is released
ALGLIB 3.8.0 was released today! This release features a lot of performance-related improvements, new features and bug fixes. Both editions of ALGLIB (Free and Commercial one) share following improvements:
ALGLIB Commercial Edition features following performance-related improvements (our former customers should upgrade their license in order to access these improvements):
Both editions of ALGLIB share following bug fixes:
02.08.2013 ALGLIB Project announces changes in product line!
ALGLIB Project is going to release ALGLIB 3.8.0 in several days and we are glad to announce changes in our product line!
Prior to ALGLIB 3.8.0 we had single product - ALGLIB Standard Edition - which was distributed under two licenses, open source and commercial. Starting from 3.8.0, we have product line of two related products - ALGLIB Free Edition and ALGLIB Commercial Edition. Both editions provide same functionality, i.e. they both can solve same set of mathematical problems. However, they differ in their performance.
Free Edition is a single-threaded library with no extensive low-level optimization being applied. It is descendant of pre-3.8.0 ALGLIB Standard Edition, which had no multithreading support. Commercial Edition provides multithreaded acceleration for many important functions, and includes better optimized computational core. Functions whose performance is different in FE and CE are explicitly labelled as such in ALGLIB Reference Manual.
ALGLIB Free Edition is still licensed under GPL.
ALGLIB Commercial Edition, obviously, is distributed under commercial licensing terms.
We had to change commercial licensing terms for this new edition,
new licensing terms can be downloaded as PDF from our website.
Most important changes are:
1) different terms for distribution of ALGLIB CE source code,
2) special provisions for future inclusion of third-party binary components (Intel MKL - planned for 3.8.2).
IMPORTANT! Special note for commercial users who bought pre-3.8.0 ALGLIB and have active support-and-maintenance agreement. ALGLIB CE is a new product, related to, but different from ALGLIB SE, which you paid for. We offer you free upgrade from Standard Edition to new multithreaded Commercial Edition, but in order to do that you have to accept new licensing terms. New performance-related code is not compatible with former ALGLIB license, so accepting new license is the only way to get new ALGLIB CE.
Upgrading your license is easy - just write to email@example.com, tell us your agreement number and confirm that you are ready to upgrade to new license. We can not perform such upgrade without your consent, because it will be done by entering into new agreement.
If you decide not to upgrade, you still will be able to receive new versions of ALGLIB (within your support period), but it will be single-threaded ALGLIB without new performance-related improvements. However, it will have full set of functions - the only difference from new ALGLIB CE will be its single-threaded performance and lack of planned SSE/AVX/IntelMKL extensions.
18.01.2013 ALGLIB 3.7.0 is released
ALGLIB 3.7.0 was released today, with new algorithms and fixes:
Commercial version of ALGLIB 3.7.0 will be delivered to customers with active support/maintenance agreement within few days. Good luck with ALGLIB!
04.07.2012 ALGLIB 3.6.0 is released
ALGLIB 3.6.0 was released today, with new algorithms and fixes:
Commercial version of ALGLIB 3.6.0 will be delivered to customers with active support/maintenance agreement within few days. Good luck with ALGLIB!
22.05.2012 New articles for ALGLIB User Guide
We've updated ALGLIB User Guide - online book about ALGLIB and numerical analysis algorithms in general. Update includes three articles on interpolation and data analysis algorithms:
We continue our work on bringing you more information - and developing new algorithms. Good luck with ALGLIB!
27.03.2012 ALGLIB 3.5.0 is released
New version of ALGLIB is released - ALGLIB 3.5.0, with many improvements and new algorithms:
Commercial version of the new release will be delivered to customers with active support/maintenance agreement within few days. Good luck with ALGLIB!
03.01.2012 Commercial licensing policy changed - more rights for our customers
ALGLIB Project announces introduction of new version of the ALGLIB Commercial License Agreement. New agreement gives same freedoms as previous one, but additionally it allows our customers to:
These options can be used for no additional fee by any customer, from large company to one-developer startup. We believe that such license covers most use cases. However, if you feel that for some reason our commercial license does not suit your needs, feel free to contact us and describe your situation. We are always ready to discuss new licensing options.
04.10.2011 We've moved into a new office!
ALGLIB Project - a company which supports ALGLIB - has moved to the new location. Our new office is located at the 12th floor of the "Capital-Nizhny" office building. Thus, our mailing address has changed to 603006, Russian Federation, Nizhny Novgorod, Maxim Gorky street, 117, room 1210. Our legal address remains unchanged.
17.06.2011 ALGLIB 3.4.0 is released
New version of ALGLIB is released - ALGLIB 3.4.0. This release includes many improvements:
We've also fixed several bugs:
28.04.2011 Commercial licensing/pricing policy changed
ALGLIB Project announces following changes in the commercial licensing policy. First, we've introduced new version of the commercial license agreement. It gives same freedoms as previous one, but additionally it:
We've also introduced set of new licensing options targeted at large businesses and government entities:
Finally, we've changed our pricing model. New pricing model features differentiated price, which depends on the number of developers working with ALGLIB. Thus, it has some traits of per-developer licensing - but it is not per-developer licensing because there is no need to pay additional fees every time you add one mode developer.
There is unlimited company-wide license and there is a set of discounts based on the number of developers actually using ALGLIB. For example, there are discounts for companies with 1, 3, 5 or 8 developers. Additional discounts are provided to companies from developing countries or companies which need only minor fraction of ALGLIB functionality. These discounts are applied at the moment of purchase and are not subject to recalculation/revocation in the future (except for the situation when you understand that you need more developers right after the purchase). We've discussed details of our new licensing/pricing model in our FAQ. Speaking short, we give discount for small companies, but we do not charge them when they begin to grow.
We hope that our new licensing model will make ALGLIB accessible both to small and large companies. In case you feel that your company needs special handling (i.e. some special kind of discount), feel free to contact us.
06.03.2011 ALGLIB 3.3.0 is released
New release of ALGLIB - now in five programming environments (C++, C#, CPython, IronPython, VB.NET). This release contains following improvements:
lsfitunit) now support optimization/fitting with boundary constraints
29.12.2010 ALGLIB for IronPython is released
We are ready to announce last release of the Release Month - ALGLIB for IronPython!
Similar to its CPython counterpart, ALGLIB for IronPython is a wrapper - but wrapper around computational core written in C#. ALGLIB for IronPython - is a 100% NET library, which includes only safe code and which is compatible with both 32-bit and 64-bit architectures. And, as far as we know, ALGLIB is a first major numerical analysis library, which is fully compatible with IronPython.
This release is the last one for 2010, and our Release Month is over. Good luck to all ALGLIB users in the upcoming 2011!
25.12.2010 One more release - ALGLIB for CPython
Release Month continues with one more branch of ALGLIB! ALGLIB for CPython is an automatically generated wrapper for computational core written in C++. This branch of ALGLIB combines performance of C++ with flexibility of Python.
One of the most interesting features of new ALGLIB is 100% compatibility with both branches of Python - 2.x and 3.x, which allows to use package with any version of interpreter since 2.5. It makes ALGLIB one of the few numerical packages which can be used with Python 3.
In order to achieve such portability, ALGLIB for CPython uses
ctypes to access computational core, which is implemented as external shared library.
As result, it can be used with CPython only - other implementations of Python don't support
ctypes fully enough.
However, Release Month is not over yet :) in a several days we will introduce one more version of ALGLIB, for one more programming language from (spoiler!) Python family.
23.12.2010 ALGLIB for VB.NET is released
New branch of ALGLIB is released - ALGLIB for VB.NET! From now - in open source and commercial editions.
ALGLIB for VB.NET is an automatically generate wrapper for ALGLIB for C#.
All ALGLIB functionality can be accessed through one VB module -
which automatically calls appropriate C# functions and converts data structures between VB.NET and C#.
And you don't have to study C# in order to use ALGLIB and to setup C# compiler -
C# computational core is distributed in the precompiled form (although you still have sources).
20.12.2010 ALGLIB 3.2.0 is released
We announce new version of ALGLIB - ALGLIB 3.2.0. This release includes many improvements and new algorithms:
31.10.2010 ALGLIB 3.1.0 is released
We are happy to announce new release - ALGLIB 3.1.0. This release includes following improvements:
We also significantly improved ALGLIB documentation, especially 'Optimization' è 'Interpolation' sections of ALGLIB User Guide. Many new examples were added, cross-links between ALGLIB User Guide and ALGLIB Reference Manual were implemented.
30.09.2010 Final release of ALGLIB 3.0.0 for C++ and C#
Final release of ALGLIB 3.0.0 is ready - both C++ and C# versions. This release contains some minor tweaks for better portability as well as some new functionality. We've fixed some minor bugs in ALGLIB 3.0.0.RC1 (and discovered some bugs in GCC and Sun Studio during our testing).
C# version features significant changes in the interface provided by the library. As with C++ version, 104 .cs files were merged into 12 packages.
Function names were changed from
alglib.functionname. Inconvenient reverse communication
interface used by optimizers and fitting functions was replaced by more convenient version which accepts delegates.
There is also new functionality in the ALGLIB 3.0 in comparison with Release Candidate 1: better step selection for CG-based algorithms, grid conversion and differentiation using cubic splines, internal sorting algorithm improvements.
01.09.2010 New release - ALGLIB 3.rc1 for C++
Totally new version of ALGLIB is released - ALGLIB 3.0.0.rc1. This new version contains several major changes which makes it very different from previous releases.
First, it contains many backward incompatible changes. Names/parameters of some functions were changed. Obsolete constructions which were there because of backward compatibility only are now gone. See Change Log for more information. Sounds complex? Not really - if you can compile your code with new ALGLIB, everything will work.
Second, there was major restructurization of code which was made after discussion with ALGLIB users. ALGLIB 2.6.0 contained 104 units, and our users find it inconvenient to have 104 files in their project. Now, in ALGLIB 3.x all units are merged into 12 packages with simple dependency structure. So you have to change names of headers/units you use.
Third, project idea was changed a bit. Before 3.x core concept was automatic translation from pseudocode to each of target languages. FreePascal ALGLIB was 100% FreePascal, VBA ALGLIB was 100% VBA. It was very beatiful idea, but the more we wanted from ALGLIB, the harder it was to implement it in all languages. So from 3.0 we've changed our focus from automatic translation to automatic interface generation. Automatic translation is still used, but we translate to several "rich" languages only: C, C++, C#, multiple precision C++. Other languages will get interface to precompiled C version.
Such large-scale changes need some time, so now we are only able to introduce C++ version. During September we will introduce C# (which is almost ready) and Python versions (former will be implemented in pure C#, while latter will be interface to precompiled C version). Python interface was announced long before this release. It was planned to be introduced in June, but because of switch to ALGLIB 3.x with its backward incompatible changeds we've decided to make it available after first 3.x release.
There is also new functionality in the ALGLIB 3.0: nonlinear solver, efficient restart functions for optimizers, improved error handling in many functions. Inconvenient reverse communication interface used by optimizers and fitting functions was replaced by more convenient version which accepts function pointers (however, you still can use reverse communication interface).
01.06.2010 ALGLIB 2.6.0 is released
New ALGLIB release features:
1. two bugfixes (see. Change Log).
2. improved spline interpolation algorithms: added support for Catmull-Rom spline; added support for periodic boundary conditions.
3. new algorithms for parametric spline interpolation in 2D and 3D are introduced. Two types of curves are supported: with non-periodic and periodic boundary conditions, i.e. open curves and loops.
Now several words about the future. New release of ALGLIB is almost the same as ALGLIB 2.5.0 - there were no new algorithms except for improved spline interpolation. The reason for this is active work under new extension of ALGLIB package - interface between ALGLIB and Python. Two interfaces are planned to be released - one for double precision ALGLIB, another for multiple precision ALGLIB. At least one of them should be ready in June, with another following several weeks later.
12.04.2010 ALGLIB 2.5.0 is released
New ALGLIB release features:
1. two bugfixes (see Change Log).
2. new optimization algorithms: nonlinear conjugate gradient method and active set algorithm for bound constrained optimization.
3. API of L-BFGS and Levenberg-Marquardt algorithms was slightly changed: see #0000326 and #0000327 for more info. These changes are backward incompatible. However, it should be easy to modify existing programs to work with new interface.
10.03.2010 ALGLIB 2.4.0 is released
1. New ALGLIB release contains improved versions of several linear algebra algorithms: QR decomposition, matrix inversion, condition number estimation.
2. New algorithms implemented: complex QR and LQ decompositions, condition number estimation for triangular matrices, exact and approximate nearest neighbor search using kd-trees, new multidimensional scattered data interpolation/fitting algorithm with O(N·logN) complexity (modified Shepard's method with fast k-NN queries).
3. Minor fixes in one of the units (see Change Log).
30.01.2010 ALGLIB 2.3.0 is released
1. New ALGLIB release features improved linear algebra subroutines (mostly in C++ version). Performance of C++ implementation of Level 1 BLAS was significantly improved. A number of cache oblivious Level 2 and Level 3 BLAS algorithms was implemented. Triangular factorization algorithms (LU decomposition and Cholesky decomposition) were rewritten in new cache-oblivious manner. Now they are relying on Level 3 ALGLIB BLAS and their performance was significantly improved. New condition number estimation algorithm is much more faster than the previous version (partly due to reduction of unnecessary matrix copying).
2. New algorithms were added: a set of linear solvers for real/complex/SPD/HPD matrices.
3. Minor fixes (see Change Log for more info).
Plans for the near future: improve performance of orthogonal factorization algorithms (QR/QL/LQ/RQ, bi/tridiagonal reduction) and rewrite them in cache-oblivious manner; further improve C++ version performance using SSE intrinsics.
18.01.2010 ALGLIB 2.2.1 is released
ALGLIB 2.2.1 is a small bugfix-only release. This release fixes incorrect assertion in minlm unit.
December, 2009. ALGLIB 2.2.0.
1. One of the most important changes is new script-based compilation/testing system. Now ALGLIB may be compiled using Bash-script
build (or batch file with similar name - for Windows users). One more script -
check - may be used to test ALGLIB (or individual units).
2. ALGLIB was tested under wide variety of platforms and compilers, package portability was significantly improved. As of December, 2009, ALGLIB was tested under x86 and x86_64, under Windows and Linux, with Microsoft compilers, GCC, Mono and FreePascal. More wide testing is expected - both with new compilers and new hardware/software platforms.
3. Now ALGLIB supports Pascal in two fashions - proprietary Delphi compiler and open-source FreePascal compiler. ALGLIB for FreePascal is a different set of source codes, which targets slightly different language standard (FPC default mode). Although there are minor issues with current FPC release, FreePascal looks like an interesting compiler to support because of its open sourceness and cross-platform capabilities.
4. A lot of sources introduced: new constrained/unconstrained linear least squares fitting codes, non-linear least squares fitting sources, rational/polynomial fitting in barycentric form, fitting by splines, high quality random number generator, random matrix generator. See http://bugs.alglib.net/changelog_page.php for complete list of changes.
October, 2009. ALGLIB 2.1.2.
September, 2009. ALGLIB 2.1.1.
ALGLIB 2.1.1 - is a minor update to ALGLIB 2.1.0, bugfix-only release. The only change is a bug in the neural networks package being fixed. Major improvements will wait till 2.2.0 release, which is expected to be released in November, 2009.
September, 2009. ALGLIB 2.1.0.
The most important in the version 2.1.0 is a change of license. The decision to change the license was not easy. The project was distributed under BSD for about two years. But over time it became clear that GPL is more suitable for further project development. So starting from version 2.1.0, ALGLIB is distributed under GPL 2+ (GPL version 2 or later). License change, of course, affects only 2.1.0 and following versions. Users of previous versions may continue to use ALGLIB and distribute derivative code under less strict, BSD-compatible terms.
About other changes:
Backward incompatible changes:
July, 2009. ALGLIB 2.0.1.
ALGLIB 2.0.1 is a small bug fix release (see bug tracker for more information). The most important changes: improved performance of rational interpolation algorithm; improved compatibility with tools/compilers which checks for uninitialized variables. The next release is scheduled for August/September; new integration sources will be introduced as well as FFT/FHT/FST/FCT sources and fast convolution/correlation sources.
April, 2009. Site updates.
For a long time - almost a year - the project web-site has not been updated. Such a long break does not mean that the project development stopped. For most of the time we were actively developing new source codes. Now it's my pleasure to announce the following news to the ALGLIB users:
Welcome the new ALGLIB 2.0!
Many ALGLIB users told us about their need to download all source codes in one zip archive. Also many users told us that absence of version numbers confuses them and prevents them from tracking changes in the projects. In fact, the project has been developing for a long time in a non-traditional way, as a collection of modules, where the users only chosen the ones they needed. This approach has its advantages, for instance, it allows for creation of compact programs. However there are drawbacks too: the changes in multitude of modules are very hard to track.
This is why the source code access mechanism was changed. Now users can download the entire project in a single zip archive, while still having the possibility to download modules one by one. Individual modules still don't have version numbers. However now the entire release has a number that changes with each alteration. The current version has the number 2.0.
Please note that the version numbers do not begin with 1 as the project has undergone a serious development since 1999, only a part of which was seen to English-speaking users (the project has been originally developed as a Russian language site). Version 2.0 contains all changes made to ALGLIB during the years.
A thought about the future...
Here I will set the priority tasks for the nearest future:
03 June, 2008. Last changes:
14 April, 2008. Last changes:
09 August, 2007. Last changes:
31 March, 2007. Last changes:
03 June, 2006. Welcome to the ALGLIB.NET website!
The main news for today is opening of the English-language version of the site, which you are visiting now. Nearly seven years ago, when this site only appeared, hardly anyone could imagine what this website will turn into, a number of years later. For this time, the website revealed only its best sides in the Russian segment of the global network. Now it is time to keep moving ahead. Due to the translation of this information into English, this site has become available to a larger amount of people than before. I hope that you will find all necessary information on this site and will come back for more!
Sergey Bochkanov, Author of ALGLIB.
ALGLIB® - numerical analysis library, 1999-2014.