Recent Changes - Search:

HomePage

LBA

Mt Pleasant 26m

Ceduna 30m

Mt Pleasant 14m

Ceduna-Hobart Interferometer

Schedules

AuScope

PmWiki

edit SideBar

T 09240

Main.T09240 History

Hide minor edits - Show changes to markup

September 11, 2009, at 04:55 AM by jim Lovell -
Changed lines 137-187 from:

time2sec

cd to directory with VEX file

edit run.sh

 -np 8 (where 8 is # processes, = wc -l machines)

VEX2config evlbi mode to use vsib_send and recv instead of specifying files by hand. Also, selecting three antennas only:

 vex2config.pl -evlbi -ant Pa -ant At -ant Mp vt12a.skd 

Example

  1. on hex0:
     cd ~/correlations
     mkdir t08232 (or whatever the experiment name is)
  2. copy machines, run.sh2] and [@.threads file from a previous experiment
  3. copy .skd file across from hobart
     vex2config.pl -cross -evlbi t08232.skd
  4. edit the .input file for a fringe search / Ceduna check. e.g First 20 mins is 1921–293:
 EXECUTE TIME: 1200
 INT TIME (SEC):     0.5
 FREQ (MHZ) 0:       6640.00

remove the semi-colon from end of READ FROM FILE If pols swapped at Ceduna, for CED:

 INPUT BAND 0 POL:   L
 INPUT BAND 1 POL:   R
  1. vex2model_updated.pl t08232.skd
  2. edit machines:
 hex0
 hex2
 hex5
 hex1
 hex2
 hex3
 hex4
 hex5
  1. edit t08232.threads:
 NUMBER OF CORES:    5
 8
 7
 8
 8
 7
to:
September 11, 2009, at 04:09 AM by 131.217.62.98 -
Added lines 128-139:

Setup aips on ares:

 source /usr/local/pkg/AIPS_31DEC06/LOGIN.CSH
 $CDTST

Run aips

 aips tv=local
September 11, 2009, at 04:08 AM by 131.217.62.98 -
Changed lines 40-41 from:
 NUM CHANNELS:       1024
to:
 NUM CHANNELS:       512

NOTE: 1024 chans doesn’t seem to work!

September 11, 2009, at 01:59 AM by 131.217.62.98 -
Changed line 125 from:
to:

When the correlation is finished

September 11, 2009, at 01:31 AM by 131.217.62.98 -
Changed lines 115-116 from:
  1. hex0:
to:
  1. hex5: vsib_send -H hex5 -p 52100 *.lba but don’t press [Enter] yet!
  2. hex4: vsib_send -H hex4 -p 52101 *.lba but don’t press [Enter] yet!
  3. hex0: cd ~/correlations/t09240
  4. hex0: ./run.sh
  5. When you see this:

[ -1] INFO MPI Process 0 is running on host hex0.phys.utas.edu.au Press [Enter] in the hex5 and hex4 windows. Data should start moving and the RPF file should appear and grow.

Press Ctrl-C to stop correlation. Before re-starting, change the input file to give a different output RPF file, or delete the existing RPF file, otherwise DiFX won’t start.

September 11, 2009, at 01:23 AM by 131.217.62.98 -
Changed line 13 from:
  1. copy machines, run.sh2 and .threads file from a previous experiment. note the threads file should now be named “threads”, not “<expt_code>.threads”%
to:
  1. copy machines, run.sh2 and .threads file from a previous experiment. note the threads file should now be named “threads”, not “<expt_code>.threads”
Changed lines 16-21 from:

-cross indicates create cross-products -evlbi indicates data will be streamed using vsib_send and vsib_recv -new for the new version of DiFX

creates <blah>.input

to:
 -cross indicates create cross-products
 -evlbi indicates data will be streamed using vsib_send and vsib_recv
 -new for the new version of DiFX

This creates t09240.input

Changed lines 27-28 from:

First time with data to both telescopes is 05:04:20

to:

First time with data to both telescopes is 05:04:20. Run time2sec.pl to convert that time to seconds:

Deleted line 31:
Changed lines 85-87 from:
to:

Now make the model

 vex2model_updated.pl t09240.skd

Edit the machines file. We’re not using hex1, so it looks like this:

 hex0
 hex5
 hex4
 hex2
 hex3
 hex4
 hex5

Hobart data on hex5, Ceduna on hex4

edit the threads file. Note number of spaces between text is important:

 NUMBER OF CORES:    4
 8
 8
 7
 7

edit run.sh

 -np 7 (where 7 is # processes, = wc -l machines)

Then start correlation:

  1. Open 3 windows, one on hex0 (head node), one on hex5 (Hobart data), one on hex4 (Ceduna data)
    1. hex5: cd /data/glast_agn/t09240
    2. hex4: cd /data_fermi/t09240/cd
    3. in one of the above (if necessary), delete any initial data files where there’s no overlap
    4. hex0:
Deleted lines 118-219:

Times in seconds are from the start of the day. Need to change these if correlating a subset of the experiment

Configurations

 NUM CONFIGURATIONS: 1        leave
 CONFIG SOURCE:      DEFAULT   leave
 INT TIME (SEC):     2        change if you want to
 NUM CHANNELS:       128      cahnge
 BLOCKS PER SEND:    1250     change in extreme cases (e.g. lots of spectral pts) for performance
 GUARD BLOCKS:       1        leave don’t know what this is
 POST-F FRINGE ROT:  FALSE    TRUE is faster but avoid when high spec res, high fringe rate
 QUAD DELAY INTERP:  TRUE     leave
 WRITE AUTOCORRS:    TRUE     change (if you want to)
 PULSAR BINNING:     FALSE    leave (unless you’re interested in pulsars)
 DATASTREAM 0 INDEX: 0        leave
 DATASTREAM 1 INDEX: 1      leave
 BASELINE 0 INDEX:   0   leave 

 # FREQ TABLE #######! leave but check it agrees with obs

 # TELESCOPE TABLE ##!    Change. Use GPS-Maser as first guess. Correlate on a fringe-finder, fringe fit and apply in production run. If monitoring GPS-Maser during the experiment use the rate from that, it’s probably better.
 TELESCOPE ENTRIES:  2
 TELESCOPE NAME 0:   HOB
 CLOCK DELAY (us) 0: −10.6
 CLOCK RATE(us/s) 0: 0.0
 TELESCOPE NAME 1:   CED
 CLOCK DELAY (us) 1: 1.05
 CLOCK RATE(us/s) 1: 0.0 

 # DATASTREAM TABLE #!           leave, except maybe Tsys (in Jy)
 DATASTREAM ENTRIES: 2
 DATA BUFFER FACTOR: 256
 NUM DATA SEGMENTS:  16
 TELESCOPE INDEX:    0
 TSYS:               1240
 DATA FORMAT:        LBAVSOP
 QUANTISATION BITS:  2
 FILTERBANK USED:    FALSE
 READ FROM FILE:     TRUE
 NUM FREQS:          1
 FREQ TABLE INDEX 0: 0
 CLK OFFSET 0 (us):  0.000       leave but this is the clock offset per freq. may be used after fringe-fit to correct for differing delays. 
 NUM POLS 0:         2
 INPUT BAND 0 POL:   R  check this is correct.
 INPUT BAND 0 INDEX: 0
 INPUT BAND 1 POL:   L
 INPUT BAND 1 INDEX: 0


 # BASELINE TABLE ###!    defines what to process
 BASELINE ENTRIES:   1
 D/STREAM A INDEX 0: 0
 D/STREAM B INDEX 0: 1    baseline 0 correlates antennas 0 and 1
 NUM FREQS 0:        1
 POL PRODUCTS 0/0:   2
 D/STREAM A BAND 0:  0    A0-B0 (RCP)
 D/STREAM B BAND 0:  0
 D/STREAM A BAND 1:  1    A1-B1 (LCP)
 D/STREAM B BAND 1:  1 

For cross-products too

 vex2config.pl -cross chsb6324.skd 

and the above section will look like this:

 POL PRODUCTS 0/0:   4
 D/STREAM A BAND 0:  0
 D/STREAM B BAND 0:  0
 D/STREAM A BAND 1:  1
 D/STREAM B BAND 1:  1
 D/STREAM A BAND 2:  0
 D/STREAM B BAND 2:  1
 D/STREAM A BAND 3:  1
 D/STREAM B BAND 3:  0


 # DATA TABLE #######     Need to edit this bit if not using evlbi (vsib_send/recv) mode:
 should be the number of raw data files in each directory e.g.:

 # DATA TABLE #######
 D/STREAM 0 FILES:   6
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230000.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230010.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230020.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230030.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230040.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230050.lba
 D/STREAM 1 FILES:   6
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230000.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230010.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230020.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230030.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230040.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230050.lba


 # NETWORK TABLE ####!      for eVLBI. 
 PORT NUM 0:         52100       port number for data stream 1
 TCP WINDOW (KB) 0:  512         TCP window size
 PORT NUM 1:         52101      port number for data stream 2
 TCP WINDOW (KB) 1:  −1
Changed lines 120-124 from:
 vex2model_updated.pl chsb6324.skd 

machines file

 first line is head node (hex0)
 next is data stream processes:
to:

edit run.sh

 -np 8 (where 8 is # processes, = wc -l machines)

VEX2config evlbi mode to use vsib_send and recv instead of specifying files by hand. Also, selecting three antennas only:

 vex2config.pl -evlbi -ant Pa -ant At -ant Mp vt12a.skd 

Example

  1. on hex0:
     cd ~/correlations
     mkdir t08232 (or whatever the experiment name is)
  2. copy machines, run.sh2] and [@.threads file from a previous experiment
  3. copy .skd file across from hobart
     vex2config.pl -cross -evlbi t08232.skd
  4. edit the .input file for a fringe search / Ceduna check. e.g First 20 mins is 1921–293:
 EXECUTE TIME: 1200
 INT TIME (SEC):     0.5
 FREQ (MHZ) 0:       6640.00

remove the semi-colon from end of READ FROM FILE If pols swapped at Ceduna, for CED:

 INPUT BAND 0 POL:   L
 INPUT BAND 1 POL:   R
  1. vex2model_updated.pl t08232.skd
  2. edit machines:
Deleted lines 150-159:
 hex0   head node
 hex0   data stream 0
 hex0   data stream 1
  (if e.g. data were on hex4):
 hex4   data stream 0
 hex4   data stream 1
  next are processing (core) nodes 
 hex1
Deleted lines 151-152:
 hex3
 hex4
Changed lines 153-155 from:
 (leaving hex0 to deal with IO (data streams) and head (manager)). We could specify hex0 as another core node and use the remaining 5 threads (1+4) but in this case there’s no need)

edit the threads file. Note number of spaces between text is important:

to:
 hex1
 hex2
 hex3
 hex4
 hex5
  1. edit t08232.threads:
Added line 161:
 7
Deleted lines 163-211:
 8
 8

These would be reduced if running IO nodes on other machines. E.g. if data on hex4:

 8
 8
 8
 6
 8
 … and may have used hex0 too, but anyway….

edit run.sh

 -np 8 (where 8 is # processes, = wc -l machines)

VEX2config evlbi mode to use vsib_send and recv instead of specifying files by hand. Also, selecting three antennas only:

 vex2config.pl -evlbi -ant Pa -ant At -ant Mp vt12a.skd 

Example

  1. on hex0:
     cd ~/correlations
     mkdir t08232 (or whatever the experiment name is)
  2. copy machines, run.sh2] and [@.threads file from a previous experiment
  3. copy .skd file across from hobart
     vex2config.pl -cross -evlbi t08232.skd
  4. edit the .input file for a fringe search / Ceduna check. e.g First 20 mins is 1921–293:
 EXECUTE TIME: 1200
 INT TIME (SEC):     0.5
 FREQ (MHZ) 0:       6640.00

remove the semi-colon from end of READ FROM FILE If pols swapped at Ceduna, for CED:

 INPUT BAND 0 POL:   L
 INPUT BAND 1 POL:   R
  1. vex2model_updated.pl t08232.skd
  2. edit machines:
 hex0
 hex2
 hex5
 hex1
 hex2
 hex3
 hex4
 hex5
  1. edit t08232.threads:
 NUMBER OF CORES:    5
 8
Deleted lines 164-166:
 8
 8
 7
September 11, 2009, at 01:12 AM by 131.217.62.98 -
Changed lines 1-274 from:

Test with Agilent and SMY on 3c273

to:

Test with Agilent and SMY on 3c273

Correlation notes

512 chan delay fix pols

  1. on hex0:
     cd ~/correlations
     mkdir t09240
  2. copy machines, run.sh2 and .threads file from a previous experiment. note the threads file should now be named “threads”, not “<expt_code>.threads”%
  3. copy .skd file across from hobart
     vex2config.pl -cross -evlbi -new t09240.skd

-cross indicates create cross-products -evlbi indicates data will be streamed using vsib_send and vsib_recv -new for the new version of DiFX

creates <blah>.input

edit the input file and check the following:

 EXECUTE TIME (SEC): 
 START MJD:          
 START SECONDS:      

First time with data to both telescopes is 05:04:20

 observer@hex0:~/correlations/t09240$ time2sec.pl 05:04:20
  05:04:20 → 18260

so in the input file:

 START SECONDS:      18260

For an initial fringe check, set

 EXECUTE TIME (SEC): 300
 OUTPUT FILENAME: /home/observer/correlations/t09240/t09240_1.rpf

Lots of channels for initial search

 NUM CHANNELS:       1024

It turned out that default clocks were way out. Try these for a start:

 # TELESCOPE TABLE ##!
 TELESCOPE ENTRIES:  2
 TELESCOPE NAME 0:   HOB
 CLOCK DELAY (us) 0: −3
 CLOCK RATE(us/s) 0: 0.0
 TELESCOPE NAME 1:   CED
 CLOCK DELAY (us) 1: 1.05
 CLOCK RATE(us/s) 1: 0.0

Polarisations are swapped at Ceduna, so in the DATASTREAM TABLE, swap pols for second antenna (Ceduna):

 TELESCOPE INDEX:    1
 …
 INPUT BAND 0 POL:   L
 INPUT BAND 0 INDEX: 0
 INPUT BAND 1 POL:   L
 INPUT BAND 1 INDEX: 1
 INPUT BAND 2 POL:   R
 INPUT BAND 2 INDEX: 0
 INPUT BAND 3 POL:   R
 INPUT BAND 3 INDEX: 1

Also in the baseline table:

 # BASELINE TABLE ###!
 …
 POL PRODUCTS 0/1:   4
 D/STREAM A BAND 0:  1
 D/STREAM B BAND 0:  3
 D/STREAM A BAND 1:  3
 D/STREAM B BAND 1:  1
 D/STREAM A BAND 2:  1
 D/STREAM B BAND 2:  1
 D/STREAM A BAND 3:  3
 D/STREAM B BAND 3:  3

Lastly, the port numbers for data

 # NETWORK TABLE ####!
 PORT NUM 0:         52100
 TCP WINDOW (KB) 0:  512
 PORT NUM 1:         52101
 TCP WINDOW (KB) 1:  512

time2sec

Times in seconds are from the start of the day. Need to change these if correlating a subset of the experiment

Configurations

 NUM CONFIGURATIONS: 1        leave
 CONFIG SOURCE:      DEFAULT   leave
 INT TIME (SEC):     2        change if you want to
 NUM CHANNELS:       128      cahnge
 BLOCKS PER SEND:    1250     change in extreme cases (e.g. lots of spectral pts) for performance
 GUARD BLOCKS:       1        leave don’t know what this is
 POST-F FRINGE ROT:  FALSE    TRUE is faster but avoid when high spec res, high fringe rate
 QUAD DELAY INTERP:  TRUE     leave
 WRITE AUTOCORRS:    TRUE     change (if you want to)
 PULSAR BINNING:     FALSE    leave (unless you’re interested in pulsars)
 DATASTREAM 0 INDEX: 0        leave
 DATASTREAM 1 INDEX: 1      leave
 BASELINE 0 INDEX:   0   leave 

 # FREQ TABLE #######! leave but check it agrees with obs

 # TELESCOPE TABLE ##!    Change. Use GPS-Maser as first guess. Correlate on a fringe-finder, fringe fit and apply in production run. If monitoring GPS-Maser during the experiment use the rate from that, it’s probably better.
 TELESCOPE ENTRIES:  2
 TELESCOPE NAME 0:   HOB
 CLOCK DELAY (us) 0: −10.6
 CLOCK RATE(us/s) 0: 0.0
 TELESCOPE NAME 1:   CED
 CLOCK DELAY (us) 1: 1.05
 CLOCK RATE(us/s) 1: 0.0 

 # DATASTREAM TABLE #!           leave, except maybe Tsys (in Jy)
 DATASTREAM ENTRIES: 2
 DATA BUFFER FACTOR: 256
 NUM DATA SEGMENTS:  16
 TELESCOPE INDEX:    0
 TSYS:               1240
 DATA FORMAT:        LBAVSOP
 QUANTISATION BITS:  2
 FILTERBANK USED:    FALSE
 READ FROM FILE:     TRUE
 NUM FREQS:          1
 FREQ TABLE INDEX 0: 0
 CLK OFFSET 0 (us):  0.000       leave but this is the clock offset per freq. may be used after fringe-fit to correct for differing delays. 
 NUM POLS 0:         2
 INPUT BAND 0 POL:   R  check this is correct.
 INPUT BAND 0 INDEX: 0
 INPUT BAND 1 POL:   L
 INPUT BAND 1 INDEX: 0


 # BASELINE TABLE ###!    defines what to process
 BASELINE ENTRIES:   1
 D/STREAM A INDEX 0: 0
 D/STREAM B INDEX 0: 1    baseline 0 correlates antennas 0 and 1
 NUM FREQS 0:        1
 POL PRODUCTS 0/0:   2
 D/STREAM A BAND 0:  0    A0-B0 (RCP)
 D/STREAM B BAND 0:  0
 D/STREAM A BAND 1:  1    A1-B1 (LCP)
 D/STREAM B BAND 1:  1 

For cross-products too

 vex2config.pl -cross chsb6324.skd 

and the above section will look like this:

 POL PRODUCTS 0/0:   4
 D/STREAM A BAND 0:  0
 D/STREAM B BAND 0:  0
 D/STREAM A BAND 1:  1
 D/STREAM B BAND 1:  1
 D/STREAM A BAND 2:  0
 D/STREAM B BAND 2:  1
 D/STREAM A BAND 3:  1
 D/STREAM B BAND 3:  0


 # DATA TABLE #######     Need to edit this bit if not using evlbi (vsib_send/recv) mode:
 should be the number of raw data files in each directory e.g.:

 # DATA TABLE #######
 D/STREAM 0 FILES:   6
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230000.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230010.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230020.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230030.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230040.lba
 FILE 0/0:           /data/sb6324/ho/sb6324_Ho_323_230050.lba
 D/STREAM 1 FILES:   6
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230000.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230010.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230020.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230030.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230040.lba
 FILE 0/0:           /data/sb6324/cd/sb6324_Cd_323_230050.lba


 # NETWORK TABLE ####!      for eVLBI. 
 PORT NUM 0:         52100       port number for data stream 1
 TCP WINDOW (KB) 0:  512         TCP window size
 PORT NUM 1:         52101      port number for data stream 2
 TCP WINDOW (KB) 1:  −1

cd to directory with VEX file

 vex2model_updated.pl chsb6324.skd 

machines file

 first line is head node (hex0)
 next is data stream processes:
 hex0


 hex0   head node
 hex0   data stream 0
 hex0   data stream 1
  (if e.g. data were on hex4):
 hex4   data stream 0
 hex4   data stream 1
  next are processing (core) nodes 
 hex1
 hex2
 hex3
 hex4
 hex5
 (leaving hex0 to deal with IO (data streams) and head (manager)). We could specify hex0 as another core node and use the remaining 5 threads (1+4) but in this case there’s no need)

edit the threads file. Note number of spaces between text is important:

 NUMBER OF CORES:    5
 8
 8
 8
 8
 8

These would be reduced if running IO nodes on other machines. E.g. if data on hex4:

 8
 8
 8
 6
 8
 … and may have used hex0 too, but anyway….

edit run.sh

 -np 8 (where 8 is # processes, = wc -l machines)

VEX2config evlbi mode to use vsib_send and recv instead of specifying files by hand. Also, selecting three antennas only:

 vex2config.pl -evlbi -ant Pa -ant At -ant Mp vt12a.skd 

Example

  1. on hex0:
     cd ~/correlations
     mkdir t08232 (or whatever the experiment name is)
  2. copy machines, run.sh2] and [@.threads file from a previous experiment
  3. copy .skd file across from hobart
     vex2config.pl -cross -evlbi t08232.skd
  4. edit the .input file for a fringe search / Ceduna check. e.g First 20 mins is 1921–293:
 EXECUTE TIME: 1200
 INT TIME (SEC):     0.5
 FREQ (MHZ) 0:       6640.00

remove the semi-colon from end of READ FROM FILE If pols swapped at Ceduna, for CED:

 INPUT BAND 0 POL:   L
 INPUT BAND 1 POL:   R
  1. vex2model_updated.pl t08232.skd
  2. edit machines:
 hex0
 hex2
 hex5
 hex1
 hex2
 hex3
 hex4
 hex5
  1. edit t08232.threads:
 NUMBER OF CORES:    5
 8
 7
 8
 8
 7
September 05, 2009, at 06:46 PM by 131.217.63.235 -
Added line 1:

Test with Agilent and SMY on 3c273

Edit - History - Print - Recent Changes - Search
Page last modified on September 11, 2009, at 04:55 AM