Installation
This documentation provides installation instructions for HARP.
HARP is a toolset for ingesting, processing and inter-comparing satellite data against correlative data. The toolset is composed of a set of command line tools and a library of analysis functions.
Supported platforms
HARP is designed to run on most Unix-based operating systems (including Linux and macOS) and Windows. The platforms that are supported include at least Linux, Windows, and macOS.
What you will need
The HARP package : You can choose between a source installation (
harp-x.y.z.tar.gz
) or a binary installation (harp-x.y.z-win64.msi
(Windows only)). Note that x.y.z stands for the version number of the package. The source package contains the source code for the HARP components (C Library and command line tools) together with all documentation. You will need this package if you want to use HARP on a Unix-based system. For Windows you will only need this package if you want to have access to the HARP source (if, for instance, you want to extend/improve HARP). The binary package, which is only available for Windows, contains pre-built versions of everything from the source distribution (without the sources) and all documentation. If you do not have the HARP package, you can download it from GitHub:The following items are needed if you use the HARP source distribution:
A C compiler : Most Unix platforms come with their own C compiler so this should not be a problem. For macOS you should make sure you have installed the Developer Tools.
A recent version of CODA. You will need a version of CODA that is compiled using the same HDF4 and HDF5 libraries as you will use for building HARP. CODA can be downloaded from https://github.com/stcorp/coda/releases/latest
A recent version of HDF4. You will also need the additional libraries libjpeg, zlib, and szlib depending on whether HDF4 was build against them. In order to prevent conflicts of the HDF4 library with the netCDF interface that HARP uses, make sure you build the HDF4 software from source and use the following options when configuring the package:
./configure --enable-shared --disable-fortran --disable-netcdfMake sure that after installation the location of the libraries is in your shared library path (e.g. update your
LD_LIBRARY_PATH
environment variable).A recent version of HDF5 (version 1.8.0 or higher). You will also need the additional libraries zlib, and szlib depending on whether HDF5 was build against them.
If you plan on using the Python interface you need either Python 2 (version 2.7 or higher) or Python 3 (version 3.4 or higher). You will also need the Python packages
numpy
(version 1.4 or higher) andcffi
(version 1.5.2 or higher).
Using GitHub
HARP is also available as source code directly from GitHub, but this is only recommended to be used by users who want to co-develop HARP and it will only work on Unix-based systems. You will need to have additional software installed and need to peform additional steps if you want to build HARP from the GitHub git respository.
- Additional software that you will need:
- autoconf + automake + libtool
- flex + bison
When you clone and checkout the HARP git repository make sure to also initialize the git submodules:
$ git clone https://github.com/stcorp/harp.git
$ cd harp
$ git submodule init
$ git submodule update
This will pull in several entries in the definitions subdirectory.
Then, run the bootstrap script in the root of the source directory:
$ ./bootstrap
After that you can follow the steps as usual for building the source package.
Note that the CMake build scripts that come with HARP can also be used to build HARP, but they do not allow regeneration of automatically generated files (such as the documentation), so they are not recommended for performing co-development of HARP.
Building the source package (Unix)
To build the source package, make sure to download the official HARP source package, which is named harp-x.y.z.tar.gz. Don’t use the packages called ‘Source code’ from GitHub as these are contents of the git respository (see ‘Using GitHub’ above).
The HARP source package comes with both an autotools and a CMake build system. For Unix-based systems the recommended approach is to use the autotools based system (using the configure script), which is what is described below. Building with CMake is also supported, but not documented here.
The following steps will guide you through the process of building the HARP library and executables:
Go to the directory that contains the downloaded harp-x.y.z.tar.gz file and unpack this package:
% gzip -d harp-x.y.z.tar.gz % tar xf harp-x.y.z.tarMake sure you replace
x.y.z
with the appropriate version number.You should now have a
harp-x.y.z
directory. Go to this directory:% cd harp-x.y.zNext you will have to execute the
configure
script that checks what system you are on and sets up the build environment. There are several options you can pass to this script. The most important ones are:
--prefix <HARP installation directory>
:By default, if you perform an installation of the HARP package (see further below on how to do this) all files are installed in subdirectories under
/usr/local
. Executables will appear under/usr/local/bin
, libraries under/usr/local/lib
and all data files (e.g. documentation) under/usr/local/share/harp
. However, installing the files into the default places requires you to have administrator privileges, which you may not have. In order to install HARP in a different location where you do have permission to copy files to, you can use this option to provide a different installation directory. For instance, you can use--prefix=$HOME/harp
to install the software in the subdirectoryharp
of your home directory.CODA_LIB=<CODA library directory>
:The configure script will automatically try to locate the CODA library in the default locations for libraries (
/usr/local/lib
is usually not considered a default location!). If./configure
complains that it can’t find thecoda
library files, pass this option to./configure
with the location of this library file.CODA_INCLUDE=<CODA include file directory>
:The configure script will automatically try to locate the CODA include files in the default locations for include files (
/usr/local/include
is usually not considered a default location!). If./configure
complains that it can’t find thecoda.h
include file, pass this option to./configure
with the location of this include file.HDF4_LIB=<HDF4 library directory>
:The configure script will automatically try to locate the HDF4 libraries in the default locations for libraries (
/usr/local/lib
is usually not considered a default location!). If./configure
complains that it can’t find thedf
ormfhdf
library files, pass this option to./configure
with the location of these library files.HDF4_INCLUDE=<HDF4 include file directory>
:The configure script will automatically try to locate the HDF4 include files in the default locations for include files (
/usr/local/include
is usually not considered a default location!). If./configure
complains that it can’t find thehdf.h
ormfhdf.h
include files, pass this option to./configure
with the location of these include files.HDF5_LIB=<HDF5 library directory>
:The configure script will automatically try to locate the HDF5 library in the default locations for libraries (
/usr/local/lib
is usually not considered a default location!). If./configure
complains that it can’t find thehdf5
library files, pass this option to./configure
with the location of this library file.HDF5_INCLUDE=<HDF5 include file directory>
:The configure script will automatically try to locate the HDF5 include files in the default locations for include files (
/usr/local/include
is usually not considered a default location!). If./configure
complains that it can’t find thehdf5.h
include file, pass this option to./configure
with the location of this include file.--enable-python
:By default CODA is built without the Python interface. Use this option to enable building of the interface to Python. If you enable the Python interface then make sure you have installed the numpy and cffi packages for Python.
PYTHON=<Python executable>
:If you want to build the Python interface you should also use this option to tell the configuration script where your Python executable is located (e.g. PYTHON=/usr/bin/python). Make sure that you provide an absolute path for the Python executable (i.e. the path should start with a ‘/’).
--enable-idl
:By default HARP is built without the IDL interface. Use this option to enable building of the interface to IDL.
IDL=<IDL root directory>
:If you want to build the IDL interface you should also use this option to tell the configuration script where you have installed IDL. The <IDL root directory> is the root directory of your IDL installation. It should contain for instance the bin directory with the idl executable and an
external
directory containing the fileexport.h
. Also make sure that you provide an absolute path for the IDL root directory (i.e. starting with a/
).--enable-matlab
:By default HARP is built without the MATLAB interface. Use this option to enable building of the interface to MATLAB.
MATLAB=<MATLAB root directory>
:If you want to build the MATLAB interface you should also use this option to tell the configuration script where you have installed MATLAB. The <MATLAB root directory> is the root directory of your MATLAB installation. It should contain for instance the
bin
directory with the matlab and mex executables (or symbolic links to them) and anextern/include
subdirectory containing the filemex.h
. Also make sure that you provide an absolute path for the MATLAB root directory (i.e. starting with a/
).You should now call the configure script with the options that are appropriate for you. For instance, if you want to install HARP in the
harp
subdirectory of your home directory then your invocation of configure would be:% ./configure --prefix=$HOME/harpIf this completes successfully, you are now able to build the library by executing the
make
command:% makeIf everything was successful, you are now ready to install the package (this installation step is not optional! You will have to install HARP before you can use it). If you encountered any problems while building HARP and if you need help contact us (see the Feedback section at the bottom for contact information).
In order to make use of the HARP library and tools, you should install HARP as build in the previous step. Please make sure you have provided the appropriate installation directory option (
--prefix=<installdir>
) to the configure script, as explained in the previous section, if you do not want to install HARP in its default location/usr/local
. After running the configure script, issue the following command:% make installIf you enabled the IDL interface then you will have to add
<CODA installdir>/lib/coda/idl
to yourDLM_PATH
. You do this by setting anIDL_DLM_PATH
environment variable and add this setting to your system shell startup script (if you don’t now how to set environment variables or add these to your shell startup script, please ask your system administrator). The variable should be set to<IDL_DEFAULT>
appended with the HARP DLM directory. If, for instance, you have installed HARP un/usr/local
then you should set theIDL_DLM_PATH
environment variable to<IDL_DEFAULT>:/usr/local/lib/harp/idl
.Note that you should ideally not use the IDL
pref_set
function to set theIDL_DLM_PATH
. The HARP DLM file will still load, but HARP will not be able to find the coda format definition files or udunits2 xml files this way. You can work around this by setting the coda definition path and udunits2 xml path explicitly instead of having HARP/CODA automatically determine this location based on theIDL_DLM_PATH
. See points 8 and 9 below for how to set the environment variables globally. You can also set the environment variables inside IDL before you call any HARP or CODA functions using e.g.:IDL> SETENV, 'CODA_DEFINITION=/path/to/codadefs/' IDL> SETENV, 'UDUNITS2_XML_PATH=/path/to/udunits2.xml'If you enabled the MATLAB interface then you should create a
startup.m
file in amatlab
directory in your home directory (~/matlab/startup.m
) that contains the line:addpath <HARP installdir>/lib/harp/matlabMake sure that the CODA definition path is correctly set. If you followed the instructions normally, then HARP and CODA should be able to find the coda definition files (.codadef files) inside the installation path automatically. If you are getting ‘unsupported product’ errors then you may have to set the coda definition path explicitly. You can do this by setting the
CODA_DEFINITION
environment variable and have it point to the directory that contains the .codadef files. On Linux this would be e.g.:export CODA_DEFINITION=/usr/local/share/coda/definitionsInformation on how to set environment variables on macOS can be found here.
Also make sure that you do not already have a CODA_DEFINITION environment variable set that points to a different location than where the .codadef files that come with HARP are located.
Make sure that the udunits2 xml path is correctly set. If you followed the instructions normally, then HARP should be able to find the udunits2 xml files inside the installation path automatically. If you are getting ‘invalid unit’ errors then you may have to set the udunits2 xml path explicitly. You can do this by setting the
UDUNITS2_XML_PATH
environment variable and have it point to theudunits2.xml
file inside the installation path. On Linux this would be e.g.:export UDUNITS2_XML_PATH=/usr/local/share/harp/udunits2.xmlInformation on how to set environment variables on macOS can be found here.
Installing the binary package (Windows)
To install the binary package of HARP for Windows just run the executable which will guide you through the installation process. Make sure to install HARP in a directory that does not have any spaces in its path. Otherwise you might encounter problems with e.g. the Python interface.
For Python make sure that the bin
directory of your HARP installation is
in your PATH
setting. And then make sure to add the Lib/site-packages
directory to your PYTHONPATH
(or copy the harp python package that is
inside it to a location that is already in your Python module path).
For IDL you will have to add <HARP installdir>\idl
to your DLM_PATH
.
You do this by setting an IDL_DLM_PATH
environment variable.
On Windows open the ‘System’ control panel of your Windows operating system
and goto the ‘Advanced’ tab. Then click on ‘Environment Variables’ and create a
new system variable with the name IDL_DLM_PATH
and value
<IDL_DEFAULT>;C:\HARP\lib
.
If you have installed HARP in a location different from C:\HARP
then
replace this part in the value by the installation directory you chose when
installing HARP.
Note that you should ideally not use the IDL pref_set
function to set the
IDL_DLM_PATH
. The HARP DLM file will still load, but HARP will not be able
to find the coda format definition files or udunits2 xml files this way.
You can work around this by setting the coda definition path and udunits2
xml path explicitly instead of having HARP/CODA automatically determine this
location based on the IDL_DLM_PATH
.
This is done by setting the CODA_DEFINITION
and UDUNITS2_XML_PATH
environment variables. This can be done outside IDL by setting the environment
variable globally, but you can also do this inside IDL before you call any HARP
or CODA functions using:
IDL> SETENV, 'CODA_DEFINITION=/path/to/codadefs/'
IDL> SETENV, 'UDUNITS2_XML_PATH=/path/to/udunits2.xml'
For MATLAB you will have to start MATLAB and goto the ‘Set Path...’ menu item
in the ‘File’ menu. Here you should add the directory C:\HARP\matlab
. If
you have installed HARP in a location different from C:\HARP
then replace
this part of the directory by the installation directory you had chosen when
you installed HARP.
Note: The binary installer for Windows comes with DLLs for all dependencies, which are CODA, HDF4, HDF5, and their dependencies.
Building the source package (Windows)
To build the source package, make sure to download the official HARP source
package, which is named harp-x.y.z.tar.gz
. Don’t use the packages called
‘Source code’ from GitHub as these are contents of the git respository (see
‘Using GitHub’ above).
The official and only supported build system on Windows is CMake.
You will need to have builds of CODA, HDF4, HDF5 (and their dependencies)
available as well in order to build HARP. The locations of include files
and libraries for the third-party dependencies can be set using
<package>_INCLUDE_DIR
and <package>_LIBRARY_DIR
CMake options.
The generation of the Windows binary installer is done using CPack and WIX. So, in order to recreate the Windows binary installer, you will also have to make sure that you have CMake (3.0 or later) and WIX installed.
Documentation location
Both the source and binary HARP packages come with documentation in HTML.
For the source package you can access the documentation from within the
unpacked HARP source package directory by going to the doc/html
subdirectory and opening the file index.html in your browser. If you perform
an install of the source package all documentation will also be installed. You
can find the documentation under the subdirectory share/harp/doc/html
of
your installation directory (/usr/local
by default).
Feedback
If you encounter any problems while trying to build, install or run one or more components of the HARP package you can send an e-mail to:
If you are using the source package on a Unix based system, please provide a
copy of the config.log file that is created when you run ./configure
and a
copy of the output of make
with your e-mail.
Apart from problems, we would also appreciate to hear from you if you have any ideas, suggestions, or comments that may help us to improve the quality or functionality of HARP.