Showing posts with label intel mkl. Show all posts
Showing posts with label intel mkl. Show all posts

25 July 2015

615. SIESTA on debian jessie with intel mkl and ifort

It's pretty similar to what I described in http://verahill.blogspot.com.au/2015/07/614-siesta-with-mpi-on-debian-jessie.html, with the main differences being the SCALAPACK, FC, BLACS and BLAS settings in arch.make

I presume that (the sadly no longer free for non-US academics) MKL was set up as shown here: http://verahill.blogspot.com.au/2013/06/465-intel-mkl-math-kernel-library-on.html

I haven't run all the tests on the build yet, but most of the ones that I tried worked, with the exception of the benzene test which came out with "Failure to converge standard eigenproblem", which is described here: http://departments.icmab.es/leem/siesta/Documentation/Manuals/manual-2.0/node47.html and isn't due to the build parameters.

NOTE: this doesn't work on mkl version 2013.sp1.3.174 as the blacs openmpi lib is missing. It does work on 2013.3.163, which is the version I used below. I have no idea why the libraries supplied with mkl are so different.

Follow
sudo apt-get install libopenmpi-dev
sudo mkdir /opt/siesta
sudo chown $USER /opt/siesta
cd /opt/siesta
wget http://departments.icmab.es/leem/siesta/CodeAccess/Code/siesta-3.2-pl-5.tgz
tar xvf siesta-3.2-pl-5.tgz
cd siesta-3.2-pl-5/Obj
sh ../Src/obj_setup.sh
../Src/./configure --enable-mpi

Edit arch.make:
# # This file is part of the SIESTA package. # # Copyright (c) Fundacion General Universidad Autonoma de Madrid: # E.Artacho, J.Gale, A.Garcia, J.Junquera, P.Ordejon, D.Sanchez-Portal # and J.M.Soler, 1996- . # # Use of this software constitutes agreement with the full conditions # given in the SIESTA license, as signed by all legitimate users. # .SUFFIXES: .SUFFIXES: .f .F .o .a .f90 .F90 SIESTA_ARCH=x86_64-unknown-linux-gnu--unknown FPP= FPP_OUTPUT= FC=ifort RANLIB=ranlib SYS=nag SP_KIND=4 DP_KIND=8 KINDS=$(SP_KIND) $(DP_KIND) FFLAGS=-g -O2 FPPFLAGS= -DMPI -DFC_HAVE_FLUSH -DFC_HAVE_ABORT LDFLAGS= ARFLAGS_EXTRA= FCFLAGS_fixed_f= FCFLAGS_free_f90= FPPFLAGS_fixed_F= FPPFLAGS_free_F90= BLAS_LIBS=-L/opt/intel/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_sequential LAPACK_LIBS=dc_lapack.a liblapack.a BLACS_LIBS=/opt/intel/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.a SCALAPACK_LIBS=-L/opt/intel/mkl/lib/intel64 -lmkl_scalapack_lp64 COMP_LIBS=dc_lapack.a liblapack.a libblas.a NETCDF_LIBS= NETCDF_INTERFACE= MPI_LIBS= -L/usr/lib/openmpi/lib -lmpi -lmpi_f90 -lmpi_f77 LIBS=$(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS) $(NETCDF_LIBS) $(MPI_LIBS) -lpthread #SIESTA needs an F90 interface to MPI #This will give you SIESTA's own implementation #If your compiler vendor offers an alternative, you may change #to it here. MPI_INTERFACE=libmpi_f90.a MPI_INCLUDE=/usr/lib/openmpi/include #Dependency rules are created by autoconf according to whether #discrete preprocessing is necessary or not. .F.o: $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $< .F90.o: $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $< .f.o: $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) $< .f90.o: $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) $<
Then build:
make

You can edit Tests/test.mk to make sure that it's pointing to your siesta executable (or make a symlink to siesta in /opt/siest/siesta-3.2-pl-5/) and that it uses mpirun and the appropriate number of threads.

Then run make in Tests/ to run all the tests.

27 June 2013

465. The Intel MKL (Math Kernel Library) on Linux (Debian) -- for free

I've been living under the impression that the Intel MKL weren't free.

In all fairness, since I'm using AMD almost exclusively and the ACML is free -- and OpenBlas has worked fine on my i5-2400-based node -- I haven't had enough motivation to really dig into this. However, as part of a post on compiling GAMESS US, Kirill Berezovsky mentions in passing that you can get both the Intel MKL and ifortran for free for non-commercial use.

To be fair, that's free in the Windows sense, not in the Linux sense. You've still got enough restrictions to make Stallman weep, but it's free enough that we have a fighting chance at evaluation the software -- and use it if it's good enough.

See here for ACML: http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html
See here for OpenBlas: http://verahill.blogspot.com.au/2013/05/423-openblas-on-debian-wheezy.html

Anyway, this is about MKL:


0. Register a request
Go to http://software.intel.com/en-us/non-commercial-software-development
Click on what you want to download -- in this case the Intel MKL. Fill out the form and hit submit. Note that you'll need to enable cookies for this to work.


[while I was at it I got the intel parallel studio xe as well -- that's 2 Gb though, so don't get it unless you want it and have a lot of bandwidth to spare]


1. Download
You'll get an email with a link. Click it:


The MKL file is 609 MB, which is a bit bigger than the ACML files which tend to be around 67-70 Mb (each).

Note: the email you get has a serial number. From what I understand it's valid for one year if you want to download updates and new releases. However, nowhere does it say that any installed software will expire, so I presume that you can continue using the MKL libraries indefinitely.


2. Install
In my case the file is called l_mkl_11.0.4.183.tgz and it was downloaded to ~/Downloads
mkdir ~/tmp
cd ~/tmp
cp ~/Downloads/l_mkl_11.0.4.183.tgz .
tar xvf l_mkl_11.0.4.183.tgz
cd l_mkl_11.0.4.183
sudo sh install.sh
Step no: 1 of 7 | Welcome -------------------------------------------------------------------------------- Welcome to the Intel(R) Math Kernel Library 11.0 Update 4 for Linux* installation program. The Flagship of HPC Math Libraries. This library contains highly optimized, extensively threaded, mathematical functions for engineering, scientific, and financial applications that require maximum performance. -------------------------------------------------------------------------------- You will complete the steps below during this installation: Step 1 : Welcome Step 2 : License Step 3 : Activation Step 4 : Intel(R) Software Improvement Program Step 5 : Options Step 6 : Installation Step 7 : Complete -------------------------------------------------------------------------------- Step no: 1 of 7 | Options > Missing Optional Pre-requisite(s) -------------------------------------------------------------------------------- There are one or more optional unresolved issues. It is highly recommended to resolve them all before you continue the installation. You can fix them without exiting from the installation and re-check. Or you can quit from the installation, fix them and run the installation again. -------------------------------------------------------------------------------- Missing optional pre-requisites -- unsupported OS -------------------------------------------------------------------------------- 1. Skip missing optional pre-requisites [default] 2. Show the detailed info about issue(s) 3. Re-check the pre-requisites h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: 1 [..] 13. THIRD PARTY PROGRAMS. The Materials may include third party programs or materials that are governed by the third party's license terms, including without limitation, open source software. The license terms associated with such third party programs or materials govern your use of same, and Intel is not liable for such third party programs or materials. * Other names and brands may be claimed as the property of others -------------------------------------------------------------------------------- Do you agree to be bound by the terms and conditions of this license agreement? Type "accept" to continue or "decline" to back to the previous menu: accept Step no: 3 of 7 | Activation -------------------------------------------------------------------------------- If you have purchased this product and have the serial number and a connection to the internet you can choose to activate the product at this time. Activation is a secure and anonymous one-time process that verifies your software licensing rights to use the product. Alternatively, you can choose to evaluate the product or defer activation by choosing the evaluate option. Evaluation software will time out in about one month. Also you can use license file, license manager, or remote activation if the system you are installing on does not have internet access activation options. -------------------------------------------------------------------------------- 1. I want to activate my product using a serial number [default] 2. I want to evaluate my product or activate later 3. I want to activate either remotely, or by using a license file, or by using a license manager h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: 1 Note: Press "Enter" key to back to the previous menu. Please type your serial number (the format is XXXX-XXXXXXXX): -------------------------------------------------------------------------------- Activation completed successfully. -------------------------------------------------------------------------------- Press "Enter" key to continue: Step no: 4 of 7 | Intel(R) Software Improvement Program -------------------------------------------------------------------------------- Help improve your experience with Intel(R) software Participate in the design of future Intel software. Select 'Yes' to give us permission to learn about how you use your Intel software and we will do the rest. - No Personal contact information is collected - There are no surveys or additional follow-up emails by opting in - You can stop participating at any time Learn more about Intel(R) Software Improvement Program http://software.intel.com/en-us/articles/software-improvement-program With your permission, Intel may automatically receive anonymous information about how you use your current and future Intel software. -------------------------------------------------------------------------------- 1. Yes, I am willing to participate and improve Intel software. (Recommended) 2. No, I don't want to participate in the Intel(R) Software Improvement Program at this time. b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection: 2 Step no: 5 of 7 | Options -------------------------------------------------------------------------------- You are now ready to begin installation. You can use all default installation settings by simply choosing the "Start installation Now" option or you can customize these settings by selecting any of the change options given below first. You can view a summary of the settings by selecting "Show pre-install summary". -------------------------------------------------------------------------------- 1. Start installation Now 2. Change install directory [ /opt/intel/composer_xe_2013.4.183 ] 3. Change components to install [ All ] 4. Show pre-install summary h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: 1 Step no: 6 of 7 | Installation -------------------------------------------------------------------------------- Each component will be installed individually. If you cancel the installation, components that have been completely installed will remain on your system. This installation may take several minutes, depending on your system and the options you selected. -------------------------------------------------------------------------------- Installing Intel Math Kernel Library 11.0 Update 4 on IA-32 component... done -------------------------------------------------------------------------------- Installing Intel Math Kernel Library 11.0 Update 4 on Intel(R) 64 component... done -------------------------------------------------------------------------------- Finalizing installation... done -------------------------------------------------------------------------------- Press "Enter" key to continue Step no: 7 of 7 | Complete -------------------------------------------------------------------------------- Thank you for installing and for using the Intel(R) Math Kernel Library 11.0 Update 4 for Linux*. Support services start from the time you install or activate your product, so please create your support account now in order to take full advantage of your product purchase. Your Subscription Service support account provides access to free product updates interactive issue management, technical support, sample code, and documentation. To create your support account, please visit the Subscription Services web site https://registrationcenter.intel.com/RegCenter/registerexpress.aspx?clientsn=NBJ N-87S2RM3P To get started using Intel(R) Math Kernel Library 11.0 Update 4 located in /opt/intel/composer_xe_2013.4.183 visit: install-dir/Documentation/en_US/mkl/get_started.html. -------------------------------------------------------------------------------- q. Quit [default] -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [q]:

3. Usage
I'll demonstrate using nwchem. I've repeated the build instructions over and over on this blog, but here it goes again:

sudo apt-get install build-essential gfortran python2.7-dev libopenmpi-dev openmpi-bin
sudo mkdir /opt/nwchem -p
sudo chown $USER:$USER /opt/nwchem
cd /opt/nwchem
wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.3.revision1-src.2013-05-28.tar.gz -O Nwchem-6.3.revision1-src.2013-05-28.tar.gz
tar xvf Nwchem-6.3.revision1-src.2013-05-28.tar.gz
cd nwchem-6.3-src.2013-05-28/
export NWCHEM_TOP=`pwd`
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr

export BLASOPT="-L/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/ -lmkl_sequential -lmkl_core -lmkl_intel_ilp64"
export LIBRARY_PATH="$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/"

export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
export ARMCI_NETWORK=SOCKETS

cd $NWCHEM_TOP/src

make clean
make nwchem_config
make FC=gfortran 1> make.log 2>make.err

cd $NWCHEM_TOP/contrib
export FC=gfortran
./getmem.nwchem


I had a look at http://www.ichec.ie/support/tutorials/mkl.pdf to better understand what the different libraries do. Still not quite clear though, so I might not be getting optimal performance.


4. Performance
I basically repeated the test job shown here: http://verahill.blogspot.com.au/2013/05/430-briefly-crude-comparison-of.html

The machine has 16 gb ram and an intel i5-2400 cpu (4 cores)

ACML: Total times cpu: 1644.3s wall: 1655.5s
MKL:    Total times  cpu:     1550.0s     wall:     1563.8s

It certainly looks good. While 100 seconds in less than two minutes, and 1550 is 94 %  of 1644, it's a difference of about one day per fortnight, which doesn't make much of a difference on a small single-user cluster, it would make a bit of cumulative difference on our 1400 core multi-user cluster.