Recent Changes - Search:
MtPleasant26m /

Hex Home

Difx web page http://astronomy.swin.edu.au/~adeller/software/difx/
Also see the google group for difx-users http://groups.google.com/group/difx-users (for which you have to have a google account, and have your membership approved before you can access it).

To checkout/install difx software

These are the notes for the installation of difx-1.5 by Simon (August 2009).

  1. Check the packages required for OS (Debian for Hex) on http://cira.ivec.org/dokuwiki/doku.php/difx/dependenciesdebian
    1. dpkg --list | grep <package name> (can be used to see if package is installed or not)
    2. If it isn't then (after setting http_proxy environment variable, so that apt-get can get through UTAS proxy server), then apt-get install <missing package>. I had to install pkg-config, openmpi-dev and libexpat-dev
  2. edit /etc/subversion/servers (to set UTAS proxy username and password. Probably a good idea to remember to remove the password once the installation is complete as well)
  3. cd ~ ; mv difx difx-old (not sure if this is 100% necessary, but the SVN checkout didn't work properly for me on top of an old installation)
  4. svn co https://svn.atnf.csiro.au/difx (you will need a username/password for the ATNF SVN repository to the difx area. Jim and Simon have these, but if you need one for yourself go to http://svn.atnf.csiro.au/cgi-bin/svnpasswd and request it, but it will take a day or so).
  5. cd difx/setup
  6. Editted setup.bash (changed export DIFXROOT=/home/observer/difx and export IPPROOT=/opt/intel/ipp/5.3.1.062/ia32, everything else looked OK).
  7. . setup.bash
  8. The PKG_CONFIG_PATH environment variable seems to need a / on the end and the setup for this distribution doesn't to that, so export PKG_CONFIG_PATH=/home/observer/difx/lib/pkgconfig/
  9. ./install-difx

In case of problems compiling etc, you can ./install-difx --clean and ./install-difx --reconf to redo the process once the error has been fixed.

Setting up the calcserver to run automatically:

As root I

  1. cd /etc/init.d
  2. cp /home/observer/difx/applications/calcserver/branches/difx-1.5/init.d/calcserver.in calcserver
  3. I edited calcserver and replaced all occurrences of @prefix@ with /home/observer/difx/
  4. I was then able to ./calcserver start
  5. /home/observer/difx/bin/checkCalcserver hex0 then told me it was running OK
  6. To have calcserver automatically start up cd /etc/ ; update-rc.d calcserver defaults

From here down are notes for the initially difx installation by Jim/Chris (May 2008)
add /usr/local/lib to /etc/ld.so.conf, then: ldconfig

install RPFITS 

   75  tar xzvf rpfits-2.17.tar.gz 
   76  ls
   77  mv rpfits rpfits-2.17
   78  cd rpfits-2.17/
   79  ls
   80  make
   81  cd linux
   82  ls
   83  make
   84  ls
   85  make -n install
   86  make install

 cd ~/difx
 cd vlba_utils
 make

 cd /home/observer/difx/mpifxcorr/trunk/src
 vi Makefile.am

bin_PROGRAMS = mpifxcorr mpifxcorr_SOURCES = mpifxcorr.cpp fxmanager.cpp core.cpp datastream.cpp visibility.cpp configuration.cpp uvw.cpp mode.cpp mk5.cpp polyco.cpp

  1. set the include path found by configure

INCLUDES = -I/usr/local/include/ -I/home/observer/difx/vlba_utils/ -I/opt/intel/ipp/5.3.1.062/ia32/include/ $(all_includes)

  1. the library search path.

mpifxcorr_LDFLAGS = $(all_libraries) noinst_HEADERS = fxmanager.h core.h datastream.h architecture.h visibility.h configuration.h uvw.h mk5.h polyco.h

mpifxcorr_LDADD = -L/opt/intel/ipp/5.3.1.062/ia32/sharedlib \ -L/usr/local/lib/ -L/home/observer/difx/vlba_utils/ -lipps -lguide -lippvm -lvlba_utils -lrpfits -lg2c

lib_LIBRARIES = libmpifxcorr.a libmpifxcorr_a_SOURCES = configuration.cpp mode.cpp core.cpp datastream.cpp polyco.cpp mk5.cpp fxmanager.cpp uvw.cpp visibility.cpp` ~

cd .. aclocal automake 

 ./configure CXX=mpicxx

make (note NO "cd src")

Install the Perl Astro package 

  192  cd /home/observer/difx/
  193  ls
  194  cd utilities/
  195  ls
  196  cd vex
  197  ls
  198  cd vexlib/
  199  ls
  200  make
  201  make -n install
  202  ls
  203  cd ../vexperl/
  204  ls
  205  vi Makefile.PL 
  206  perl Makefile.PL 
  207  make
  208  make test

su make install 

 cd ~/bin
 ln -s /home/observer/difx/utilities/vex2difx/trunk/vex2config.pl .

  215  cd
  216  ls
  217  mkdir correlations
  218  cd correlations
  219  mkdir chsb6324
  220  ls
  221  cd chsb6324/
  222  mv ~/*.skd .
  223  ls
  224  vex2config.pl chsb6324.skd

creates <blah>.input

edit the input file and check the following: 

 EXECUTE TIME (SEC): 50400
 START MJD:          54058
 START SECONDS:      82800

Need to change these if correlating a subset of the experiment

Configurations 

 NUM CONFIGURATIONS: 1        leave
 CONFIG SOURCE:      DEFAULT   leave
 INT TIME (SEC):     2        change if you want to
 NUM CHANNELS:       128      cahnge
 BLOCKS PER SEND:    1250     change in extreme cases (e.g. lots of spectral pts) for performance
 GUARD BLOCKS:       1        leave don't know what this is
 POST-F FRINGE ROT:  FALSE    TRUE is faster but avoid when high spec res, high fringe rate
 QUAD DELAY INTERP:  TRUE     leave
 WRITE AUTOCORRS:    TRUE     change
 PULSAR BINNING:     FALSE    leave (unless you're interested in pulsars)
 DATASTREAM 0 INDEX: 0        leave
 DATASTREAM 1 INDEX: 1      leave
 BASELINE 0 INDEX:   0   leave 

 # FREQ TABLE #######! leave but check it agrees with obs

 # TELESCOPE TABLE ##!    Change. Use GPS-Maser as first guess. Correlate on a fringe-finder, fringe fit and apply in production run. If monitoring GPS-Maser during hte experiment use the rate from that, it's probably better.
 TELESCOPE ENTRIES:  2
 TELESCOPE NAME 0:   HOB
 CLOCK DELAY (us) 0: -10.6
 CLOCK RATE(us/s) 0: 0.0
 TELESCOPE NAME 1:   CED
 CLOCK DELAY (us) 1: 1.05
 CLOCK RATE(us/s) 1: 0.0 

 # DATASTREAM TABLE #!           leave, except maybe Tsys (in Jy)
 DATASTREAM ENTRIES: 2
 DATA BUFFER FACTOR: 256
 NUM DATA SEGMENTS:  16
 TELESCOPE INDEX:    0
 TSYS:               1240
 DATA FORMAT:        LBAVSOP
 QUANTISATION BITS:  2
 FILTERBANK USED:    FALSE
 READ FROM FILE:     TRUE
 NUM FREQS:          1
 FREQ TABLE INDEX 0: 0
 CLK OFFSET 0 (us):  0.000       leave but this is the clock offset per freq. may be used after fringe-fit to correct for differing delays. 
 NUM POLS 0:         2
 INPUT BAND 0 POL:   R  check this is correct.
 INPUT BAND 0 INDEX: 0
 INPUT BAND 1 POL:   L
 INPUT BAND 1 INDEX: 0


 # BASELINE TABLE ###!    defines what to process
 BASELINE ENTRIES:   1
 D/STREAM A INDEX 0: 0
 D/STREAM B INDEX 0: 1    baseline 0 correlates antennas 0 and 1
 NUM FREQS 0:        1
 POL PRODUCTS 0/0:   2
 D/STREAM A BAND 0:  0    A0-B0 (RCP)
 D/STREAM B BAND 0:  0
 D/STREAM A BAND 1:  1    A1-B1 (LCP)
 D/STREAM B BAND 1:  1

For cross-products too 

 vex2config.pl -cross chsb6324.skd

and the above section will look like this: 

 POL PRODUCTS 0/0:   4
 D/STREAM A BAND 0:  0
 D/STREAM B BAND 0:  0
 D/STREAM A BAND 1:  1
 D/STREAM B BAND 1:  1
 D/STREAM A BAND 2:  0
 D/STREAM B BAND 2:  1
 D/STREAM A BAND 3:  1
 D/STREAM B BAND 3:  0


 # DATA TABLE #######     Need to edit this bit!:
 should be the number of raw data files in each directory e.g.:
 # DATA TABLE #######
 D/STREAM 0 FILES:   6
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230000.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230010.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230020.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230030.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230040.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230050.lba
 D/STREAM 1 FILES:   6
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230000.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230010.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230020.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230030.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230040.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230050.lba




 # NETWORK TABLE ####!      for eVLBI. 
 PORT NUM 0:         52100       port number
 TCP WINDOW (KB) 0:  512         TCP window size
 PORT NUM 1:         52101
 TCP WINDOW (KB) 1:  -1

get CALC from PAMHELA cluster from Chris 

 cd ~/difx
 tar xzvf <tar.gz>
 cd calc9
  259  \rm *.o
  260  ls
  261  rm libcorcalc.a 
  262  ls *.a
  263  ls *.so*
  264  make
  265  ar -r libcorcalc.a *.o

work out how to get EOPs%

edited ~/.bashrc

cd to directory with VEX file 

 vex2model_updated.pl chsb6324.skd

machines file 

 first line is head node (hex0)
 next is data stream processes:
 hex0


 hex0   head node
 hex0   data stream 0
 hex0   data stream 1
  (if e.g. data were on hex4):
 hex4   data stream 0
 hex4   data stream 1
  next are processing (core) nodes 
 hex1
 hex2
 hex3
 hex4
 hex5
 (leaving hex0 to deal with IO (data streams) and head (manager)). We could specify hex0 as another core node and use the remaining 5 threads (1+4) but in this case there's no need)

edit the threads file. Note number of spaces between text is important: 

 NUMBER OF CORES:    5
 8
 8
 8
 8
 8

These would be reduced if running IO nodes on other machines. E.g. if data on hex4: 

 8
 8
 8
 6
 8
 ... and may have used hex0 too, but anyway....

edit run.sh 

 -np 8 (where 8 is # processes, = wc -l machines)

VEX2config evlbi mode to use vsib_send and recv instead of specifying files by hand. Also, selecting three antennas only: 

 vex2config.pl -evlbi -ant Pa -ant At -ant Mp vt12a.skd

HexTests8May08 HexTests9May08 FringeTest