NSB Entropy Estimation

The nsb-entropy project is devoted to implementation and practical use of the NSB algorithm for estimation of entropy and related information-theoretic quantities from undersampled discrete data. Originally, the algorithm was presented in

More details about the algorithm can be found in this collection of theory papers.

There are two implementations of the method (C++ and MatLab/Octave). The MatLab/Octave code (thanks to Fernando Montani for porting the original Octave source to MatLab/Octave compatible version) is pretty basic in its functionality, works cross-platform, but is not actively being developed anymore due to its slow speed as well as inability of Octave and MatLab to index discrete structures that can exist in a space with cardinality of more than 2^32 (which we routinely encounter in linguistics or neuroscience applications). The C++ code is much faster and is currently being developed. It has all the bells and whistles invented after the initial creation of algorithms and can transparently deal with complicated data files. As a cost, one needs to compile the code and install additional software libraries (GMP and GSL), both available under the GPL.

Both implementations can be downloaded from the project development/download pages, hosted by SourceForge. They are distributed under GPL v.2, the Free Software License.

The following is being requested from all users of either branches of the software:

If you are willing to help with the ongoing development, please contact Ilya Nemenman. Currently, the most important contribution you can make is to port the software to Windows and/or Unix-like operating systems besides Mac OS X, which is the native OS for the project.

Thanks for your interest!

Ilya Nemenman


SourceForge.net Logo