Difference between revisions of "GDL"

From SharedSkies
Jump to navigation Jump to search
 
(6 intermediate revisions by the same user not shown)
Line 18: Line 18:
as superuser (su), also  to the /usr/local/src tree on your Linux system:
as superuser (su), also  to the /usr/local/src tree on your Linux system:


cp gdl.zip /usr/local/src
  cp gdl.zip /usr/local/src


You are ready to compile the code.  .
You are ready to compile the code.  .
Line 28: Line 28:




cd /usr/local/src  
  cd /usr/local/src  


unzip gdl.zip
  unzip gdl.zip




This will create a "gdl-master" directory in /usr/local/src containing the files you will compile.
This will create a "gdl-master" directory in /usr/local/src containing the files you will compile.


These are some of the required packages on a new installation of Gnu Data Language:


cd gdl-master


mkdir build
*cmake
*graphicsmagick development
*pyplot and subpackages
*libgraphicsmagick++-devel
*libgeotiff2
*geotiff-devel
*netcdf-devel
*eigen3-devel
*tirpc


cd build
*pslib build and install from source
[http://pslib.sourceforge.net/ http://pslib.sourceforge.net/]


cmake -DCMAKE_BUILD_TYPE=Release -DGRIB=OFF  ../
*udunits build and install from source
[https://www.unidata.ucar.edu/software/udunits/ https://www.unidata.ucar.edu/software/udunits/]


If libraries are missing, try to resolve them using yast.  You will need pslib and udunits built from source because they are  not distributed in Tumbleweed.  A Google search will return the site for download the latest versions of the source code which will come with build instructions.  Install them in /usr/local.  The gdl build routine will find them.  Also, if you have a local version of Python in /usr/local then gdl will use it.  Turn off GRIB because the code is obsolete and the new ECC code that replaces it does not currently build on Opensuse.
Now try to build and add other packages as needed.


Edit CMakeCache.txt and set these flags:
  cd gdl-master
  mkdir build
  cd build
  cmake -DCMAKE_BUILD_TYPE=Release -DGRIB=OFF ../


//CXX compiler
We turn off GRIB because the functionality is not needed in astrophysics and it requires external code that does not compile readily.
CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++


//Flags used by the CXX compiler during all build types.
CMAKE_CXX_FLAGS:STRING=-ltirpc -lutil


Try to build the software.
  cmake -DCMAKE_BUILD_TYPE=Release -DGRIB=OFF -DWXWIDGETS=OFF -DGEOTIFF=off -DNETCDF=OFF -DHDF=off -DLIBPROJ4=OFF -DUDUNITS2=OFF -DGLPK=OFF -DSHAPELIB=OFF  ../


make
will turn off features used by geosciences and less often by physics and astronomy.  Each of these has a library requirement not present by default in Opensuse, and requiring installation of a package or from source.


The gdl source code is complex and has many dependencies.  There may be missing libraries to content with that can be added as needed using YAST.   
The default installation directory top level is /usr/localTo specify another one,  add


  -DCMAKE_INSTALL_PREFIX=/path/to/install/dir


After a successful make operation, install the binaries and library with
to the cmake installation command.


Once this runs, scan the output to be sure it will build as intended, that the necessary packages and libraries are installed,  and that the functions needed for your  GDL are set ON. 


make install
On Opensuse also edit  the CMakeCache.txt in build for these entries


  //CXX compiler
  CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++


Prepare a location for the procedures that you will add:
and for these libraries


  //Flags used by the CXX compiler during all build types.
  CMAKE_CXX_FLAGS:STRING=-ltirpc -lutil


cd /usr/local/
Centos may only require the library addition with -lutil.


mkdir gdl
Compile, check, and install by default in /usr/local/share


cd gdl
  make
  make check
  make install
 
The GDL source code is complex and has many dependencies.  There may be missing libraries to contend with that that will not be apparent until the make process is tried. They can be added as needed using YAST.


cp -p -r /usr/local/src/gdl/src/pro ./


== Add new procedures ==
== Add new procedures ==
Line 93: Line 112:




Copy the .pro files from these sources into one or more separate directories under /usr/local/gdl. For example, we would use something like this:
Copy the .pro files from these sources into one or more separate directories under /usr/local/share/gnudatalanguage/lib/ For example, we would use something like this:




cd /usr/local/gdl
  cd /usr/local/share/gnudatalanguage/lib/


mkdir astro
  mkdir astro


mkdir tmp
  mkdir tmp


cd tmp
  cd tmp


cp /home/john/gdl/archive/astron.tar.gz ./
  cp /home/john/gdl/archive/astron.tar.gz ./


tar xvzf astron.tar.gz
  tar xvzf astron.tar.gz


cd pro
  cd pro


cp *.pro /usr/local/gdl/astro/
  cp *.pro /usr/local/share/gnudatalanguage/lib/astro/


cd /usr/local/gdl
  cd /usr/local/share/gnudatalanguage/lib/


rm -r tmp
  rm -r tmp




and repeat this for each of the libraries you want to add, copying only the *.pro files into gdl.  They can all go into gdl/astro, or if you anticipate updating them individually, into a separate directory for each one.
and repeat this for each of the libraries you want to add, copying only the *.pro files into gdl.  They can all go into astro, or if you anticipate updating them individually, into a separate directory for each one.


When you are done, assign the ownership of the gdl directory to root, or to a trusted user:
When you are done, assign the ownership of the gdl directory to root, or to a trusted user:


cd /usr/local
  cd /usr/local/share/
 
  chown -R root.root gnudatalanguage


chown -R root.root gdl
to assure accessibility of all files and assure read access to the library:
 
  cd /usr/local/share/gnudatalanguage
  chmod a+r -R lib


to assure accessibility of all files and limit access for writing.


== Set environment variables ==
== Set environment variables ==
Line 133: Line 156:




export GDL_PATH=/usr/local/gdl/pro
  export GDL_PATH=$GDL_PATH:/usr/local/share/gnudatalanguage/lib/cmsvlib/
 
  export GDL_PATH=$GDL_PATH:/usr/local/share/gnudatalanguage/lib/astro/
GDL_PATH=$GDL_PATH:/usr/local/gdl/cmsvlib
  export GDL_PATH=$GDL_PATH:/fullpathto/gdl/programs/
 
  export GDL_PATH=$GDL_PATH:/fullpathto/exofast/
GDL_PATH=$GDL_PATH:/usr/local/gdl/astro
  export EXOFAST_PATH=/fullpathto/exofast/
 
GDL_PATH=$GDL_PATH:/usr/local/gdl/buie
 
GDL_PATH=$GDL_PATH:/usr/local/gdl/textoidl
 
GDL_PATH=$GDL_PATH:/usr/local/gdl/extra


export EXOFAST_PATH=/home/john/gdl/exofast


GDL_PATH=$GDL_PATH:$EXOFAST_PATH
We include here access to Jason Eastman's EXOFAST, assuming it may be installed in an indivdual user directory rather than the full gdl library location.  In that case, these changes would be in that user's "hidden" .profile  file.
 
export GDL_STARTUP=/home/john/gdl/gdlstartup
 
 
We include here access to Jason Eastman's EXOFAST, assuming it may be installed in an indivdual user directory.  In that case, these changes would be in that user's "hidden" .profile  file.


== Run EXOFAST under gdl ==
== Run EXOFAST under gdl ==




To run gdl you simply issue on the command line "gdl" and you will be in a command line environment for the Gnu Data Language.  To exit and return  to the shell, "exit".  GDL commands are identical to IDL, and most are supported in the recent code.   
To run gdl you simply issue on the command line "gdl" and you will be in a command line environment for the Gnu Data Language.  To exit and return  to the shell, "exit".  GDL commands are identical to IDL, and most are supported in the recent code.  If the EXOFAST library is in the search path the compiling will be automatic.  One way to do this is to have a named directory "exofast" which is a softlink to the version you want to run, and add that to your .profile as noted above.  This allows some flexibility in testing or using different versions without changing the environment.




To run a process
To run a process


gdl -e 'myprocess'
  gdl -e 'myprocess'


is all that's needed.  The myprocess.pro file must be in the current working directory or the path.  Note that ".pro" is not on the command line.
is all that's needed.  The myprocess.pro file must be in the current working directory or the path.  Note that ".pro" is not on the command line.
Line 169: Line 180:
To run and test EXOFAST, install it in your user home directory under your own "gdl" with the appropriate addition to your .profile as noted above. Then
To run and test EXOFAST, install it in your user home directory under your own "gdl" with the appropriate addition to your .profile as noted above. Then


cd /home/user/gdl/exofast
  cd /fullpathto/exofast
 
Test it with this command:
 
gdl -e "exofast, rvpath='hat3.rv',tranpath='hat3.flux',pname='HAT-P-3b',band='Sloani',/circular,/noslope,/specpriors,minp=2.85,maxp=2.95"
 
There are modifications to the exofast procedures and additions to GDL  needed to run  within the GDL framework.  A list of known ones as of 2012-07-30 is provided below.  For help with finding solutions, a [http://aramis.obspm.fr/~coulais/IDL_et_GDL/Matrice_IDLvsGDL_intrinsic.html list of routines supported in GDL] is available.
 
== EXOFAST under GDL: problems and solutions ==
 
 
The stable version 0.9.2 of GDL, EXOFAST will run the fitting with the libraries given above, addition of a few missing procedures,  and changes edited into specific exofast procedure files which we will note here.  It will fail because of missing intrinsics in the plotting.  The workarounds are to edit the original exofast procedures to avoid these calls, or to try the CVS version.
 
At this time the CVS version fails to properly read the exofast data files.  This appears to have been caused by a very recent change to the line parsing routines which we presume will be corrected once the errors can be documented and reported.  In the meantime it is possible to run EXOFAST using the stable release once additional procedures are included and a few changes are made to the exofast code.
 
While the fitting runs and appears to work, the plotting fails because of the missing procedures and incomplete intrinsics.  This is a soluble problem since the plotting can be rewritten to use the minimum set of features currently available in gdl, or to export the plotting to external programs.
 
 
'''Missing procedures'''
 
 
*chisqr_cvf.pro -- is not in the libraries noted above
 
It is available on a web search.  It appears to be an ITT Visual Information Solutions contributed routine, though it derives from code published elsewhere.  It could be recoded into a gdl procedure if needed.  We add it to /usr/local/gdl/extra/.
 
 
*chisqr_pdf.pro -- is not in the librarires noted above
 
It is available on a web search.  While it appears to be an ITT Visual Information Solutions procedure, it simply calls IGAMMA which is
a GDL routine.  We add it to /usr/local/gdl/extra/.
 
 
*bisect_pdf.pro -- is not in the librarires noted above
 
It is available on a web search.  It appears to be an ITT Visual Information Solutions procedure.  We add it to /usr/local/gdl/extra/.
 
 
*oploterr.pro -- is not included in gdl. 
 
It is an ITT Visual Information Solutions procedure that is available for download from the NASA IDL support [http://idlastro.gsfc.nasa.gov/idllibsrch.html search site].  We add it to /usr/local/gdl/extra.
 
The code lines
 
oploterr, time, rv.rv - (rv.bjd - t0)*slope-gamma, rv.err, 8
 
oploterr, time, rv.rv-modelrv, rv.err, 8
 
could be replaced by oplot with different parameters, or a perhaps a simple ploterr procedure could be defined. There is a note in the GDL urgent needs file that oploterr should be easily derived from ploterr.  When these lines are not commented out and the oploterr is allowed to run, there is a warning from EXOFAST
 
''Warning: multi-page PostScript not supported yet (FIXME!)''
 
generated immediately following the OPLOTERR call.  The (FIXME!) comes from GDL.
 
 
*poly_fit.pro -- is not included ingdl.
 
It is an ITT Visual Information Solutions procedure that is available for download from the NASA IDL support [http://idlastro.gsfc.nasa.gov/idllibsrch.html search site].  We add it to /usr/local/gdl/extra.
 
 
 
 
 
 
'''Missing intrinsics'''
 
 
*USERSYM -- called by the astro library routine plotsym is not present in 0.9.2
 
This routine is internally coded and should be in the distribution. In this version it is annotated in plotting.ccp with the note "//for now usersym is a circle".  It was introduced into the CVS source code in response to a bug report. However, the CVS version now generates errors in reading the data files used with EXOFAST, making it unusable in its current form.
 
USERSYM is called by plotsym.pro, an astro library routine.  It is used to create distinctive point plotting in  routines exofast_chi2.pro and exofast_plotdist.pro.  A workaround could be to comment out the call to plotsym and edit the symbol choice to a standard symbol for the following plot so that usersym (a user-defined symbol) is not requested.
 
 
*HISTOGRAM -- keyword parameter NAN not allowed in 0.9.2 . 
 
There is a note in a development discussion group that it has been added to the CVS version but not fully tested.
 
The problem shows up in exofast_plotdist.pro with calls to histogram using the /nan keyword.  We tried editing it out from the code lines:
 
hist = histogram(pars[angular[i],*],nbins=100,locations=x,/nan)
 
hist = histogram(pars[i,sorted],nbins=100,locations=x,min=xmin,max=xmax,/nan)
 
 
*CONTOUR --keyword parameter PATH_INFO, PATH_XY and PATH_DATA_COORDS are not allowed in 0.9.2


The function is called in exofast_errell.pro and fails for the first time at this line:
Exofast currently provides examples to test and under the examples directory there are specfic test procedures, such as


contour, hist2d, xtitle=xtitle,ytitle=ytitle, levels=levels, path_info=info,path_xy=xy,/path_data_coords,/overplot,/path_double
  cd /fullpathto/exofast/examples/hat3/
  gdl -e fithat3


with the error messages


For help with finding procedures and their status in development, a [http://aramis.obspm.fr/~coulais/IDL_et_GDL/Matrice_IDLvsGDL_intrinsic.html list of routines supported in GDL] is available, but it too is not current.


CONTOUR: Keyword parameter PATH_INFO not allowed in call to: CONTOUR


CONTOUR: Keyword parameter PATH_XY not allowed in call to: CONTOUR
== EXOFASTv2 under GDL ==


CONTOUR: Keyword parameter PATH_DATA_COORDS not allowed in call to: CONTOUR
As of July 2020, the github release of EXOFASTv2 will run under GDL.  Download it from [https://github.com/jdeast/EXOFASTv2 https://github.com/jdeast/EXOFASTv2] and unzip the contents into EXOFASTv2-master.  That directory has to be in the GDL search path, which as noted above, could be with a soft link from the .profile for the user.  To test it, go to examples and try hat3.


CONTOUR: Keyword parameter PATH_DOUBLE not allowed in call to: CONTOUR
  gdl -e fithat3.pro


The output of EXOFAST for these examples will be in a fitresults directory with  files of text tables, LaTeX tex,  and PostScript graphics.  For viewing on Linux, try "gv", or convert to pdf with ps2pdf and use evince or okular.


We try without contour plotting.
There were no errors generated in a test run on our system with the latest GDL and EXOFASTv2 at this time.

Latest revision as of 22:08, 27 July 2020

The Gnu Data Language (GDL) is a useful open-source alternative to IDL on Linux. Although for new astronomical scripting applications we prefer Python and Julia, the proprietary IDL system is widely used and there are many astronomical routines written for it that are freely available. This page is a guide to the installation of GDL on an OpenSuse Linux platform in order to use the community resources available.


Download the recent source code

We assume that the base OpenSuse system has been installed with the packages that are often used to support scientific computing. There are some unusual dependencies to build a complete GDL. Additional procedures may be added after the fact since they are compiled as needed by GDL itself.

These notes are based on latest github version as of July 2020. It is an improved version over the last Sourceforge release and should be largely compatible with IDL routines.

Go to the github website:

https://github.com/gnudatalanguage/gdl

Click on the "Code" link in the top menubar the "Download" button to provide the last stable release as a zip file. Once you have it on your system, make a copy in a permanent location, and as superuser (su), also to the /usr/local/src tree on your Linux system:

 cp gdl.zip /usr/local/src

You are ready to compile the code. .

Install and compile the source code

The source tree will be in /usr/local/src/ where you made a copy of the downloaded tarfile:


 cd /usr/local/src 
 unzip gdl.zip


This will create a "gdl-master" directory in /usr/local/src containing the files you will compile.

These are some of the required packages on a new installation of Gnu Data Language:


  • cmake
  • graphicsmagick development
  • pyplot and subpackages
  • libgraphicsmagick++-devel
  • libgeotiff2
  • geotiff-devel
  • netcdf-devel
  • eigen3-devel
  • tirpc
  • pslib build and install from source

http://pslib.sourceforge.net/

  • udunits build and install from source

https://www.unidata.ucar.edu/software/udunits/

Now try to build and add other packages as needed.

 cd gdl-master
 mkdir build
 cd build
 cmake -DCMAKE_BUILD_TYPE=Release -DGRIB=OFF  ../

We turn off GRIB because the functionality is not needed in astrophysics and it requires external code that does not compile readily.


 cmake -DCMAKE_BUILD_TYPE=Release -DGRIB=OFF -DWXWIDGETS=OFF -DGEOTIFF=off -DNETCDF=OFF -DHDF=off -DLIBPROJ4=OFF -DUDUNITS2=OFF -DGLPK=OFF -DSHAPELIB=OFF   ../ 

will turn off features used by geosciences and less often by physics and astronomy. Each of these has a library requirement not present by default in Opensuse, and requiring installation of a package or from source.

The default installation directory top level is /usr/local. To specify another one, add

 -DCMAKE_INSTALL_PREFIX=/path/to/install/dir

to the cmake installation command.

Once this runs, scan the output to be sure it will build as intended, that the necessary packages and libraries are installed, and that the functions needed for your GDL are set ON.

On Opensuse also edit the CMakeCache.txt in build for these entries

 //CXX compiler
 CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/c++

and for these libraries

 //Flags used by the CXX compiler during all build types.
 CMAKE_CXX_FLAGS:STRING=-ltirpc -lutil

Centos may only require the library addition with -lutil.

Compile, check, and install by default in /usr/local/share

 make
 make check
 make install

The GDL source code is complex and has many dependencies. There may be missing libraries to contend with that that will not be apparent until the make process is tried. They can be added as needed using YAST.


Add new procedures

For astronomical use, there are additional procedures to install system-wide. Download and save in your long term archive the code from these sites:



Copy the .pro files from these sources into one or more separate directories under /usr/local/share/gnudatalanguage/lib/ For example, we would use something like this:


 cd /usr/local/share/gnudatalanguage/lib/
 mkdir astro
 mkdir tmp
 cd tmp
 cp /home/john/gdl/archive/astron.tar.gz ./
 tar xvzf astron.tar.gz
 cd pro
 cp *.pro /usr/local/share/gnudatalanguage/lib/astro/
 cd /usr/local/share/gnudatalanguage/lib/
 rm -r tmp


and repeat this for each of the libraries you want to add, copying only the *.pro files into gdl. They can all go into astro, or if you anticipate updating them individually, into a separate directory for each one.

When you are done, assign the ownership of the gdl directory to root, or to a trusted user:

 cd /usr/local/share/
 chown -R root.root gnudatalanguage

to assure accessibility of all files and assure read access to the library:

 cd /usr/local/share/gnudatalanguage
 chmod a+r -R lib


Set environment variables

In the bash shell edit the .profile directory for each user who would access gdl, or add to /etc/profile.local for everyone, to make these changes (written assuming separated directories for each added library):


 export GDL_PATH=$GDL_PATH:/usr/local/share/gnudatalanguage/lib/cmsvlib/
 export GDL_PATH=$GDL_PATH:/usr/local/share/gnudatalanguage/lib/astro/
 export GDL_PATH=$GDL_PATH:/fullpathto/gdl/programs/
 export GDL_PATH=$GDL_PATH:/fullpathto/exofast/
 export EXOFAST_PATH=/fullpathto/exofast/


We include here access to Jason Eastman's EXOFAST, assuming it may be installed in an indivdual user directory rather than the full gdl library location. In that case, these changes would be in that user's "hidden" .profile file.

Run EXOFAST under gdl

To run gdl you simply issue on the command line "gdl" and you will be in a command line environment for the Gnu Data Language. To exit and return to the shell, "exit". GDL commands are identical to IDL, and most are supported in the recent code. If the EXOFAST library is in the search path the compiling will be automatic. One way to do this is to have a named directory "exofast" which is a softlink to the version you want to run, and add that to your .profile as noted above. This allows some flexibility in testing or using different versions without changing the environment.


To run a process

 gdl -e 'myprocess'

is all that's needed. The myprocess.pro file must be in the current working directory or the path. Note that ".pro" is not on the command line.


To run and test EXOFAST, install it in your user home directory under your own "gdl" with the appropriate addition to your .profile as noted above. Then

 cd /fullpathto/exofast

Exofast currently provides examples to test and under the examples directory there are specfic test procedures, such as

 cd /fullpathto/exofast/examples/hat3/
 gdl -e fithat3


For help with finding procedures and their status in development, a list of routines supported in GDL is available, but it too is not current.


EXOFASTv2 under GDL

As of July 2020, the github release of EXOFASTv2 will run under GDL. Download it from https://github.com/jdeast/EXOFASTv2 and unzip the contents into EXOFASTv2-master. That directory has to be in the GDL search path, which as noted above, could be with a soft link from the .profile for the user. To test it, go to examples and try hat3.

 gdl -e fithat3.pro

The output of EXOFAST for these examples will be in a fitresults directory with files of text tables, LaTeX tex, and PostScript graphics. For viewing on Linux, try "gv", or convert to pdf with ps2pdf and use evince or okular.

There were no errors generated in a test run on our system with the latest GDL and EXOFASTv2 at this time.