LBA Yg

Pre-setup things to do or check before setting up for LBA observations:

Make sure you are part of the LBA chat on Mattermost. Any sudden change in scheduling for experiment or any other information will be on Mattermost. You can access that by pointing a browser at https://chat.atnf.csiro.au/. If you don't have a personal account you can login as hbobserver@gmail.com (usual observer password). Keep an eye on messages here. The other LBA observers at different stations are a good first port of call for problems at Hobart/Ceduna. Chris Phillips is the main LBA coordinator at the ATNF. Jamie Stevens also knows a lot about the systems here. There are also Mattermost apps for smartphones etc if you want to use them.
Check the information email for the session. If in any doubt you can cross-check against the information for the experiment at ATNF LBA wiki

Preparing schedule files

Instructions on preparing LBA schedule files for use at Yaragaddee 12m are discussed here:

First of all, Access yg vnc by vncviewer pcfsyg:1 from ops8. You can also use 'vnc://pcfsyg.phys.utas.edu.au:5901' from an inbuilt mac vnc. you'll need UTAS VPN to connect to the vncs if your are observing outside of UTAS network.
Now find out the experiment name and code which you are going to setup for (eg. v255ah). Find out experiments start time, end time in UTC format. This info can be found either on UTAS observing wiki page ra-wiki page, or offical LBA website page ATNF LBA wiki
Your next job now is to make sure that proc file and snp file the particular experiment are already there or not. From oper@pcfsyg, go to /usr2/proc and /usr2/sched folders to check whether these files are there or not. e.g for exp v255ah, i'll go (from oper@pcfs-2ke):
cd /usr2/proc/
ls v255ahyg.prc
cd /usr2/sched/
ls v255ahyg.snp

Most probably the files will be drudged and checked before the session/exp starts by the main UTAS LBA support person. so, if these files are there in their relevant folders, then it's ok. If they are not there, then you need to download and drudge the exp to make these files.

How to drudge

Most probably the main UTAS LBA support person will make these proc files already before the start of the session and you will find them in proc folder. But in case if they are not there, contact the on-call person.

Everything is done from oper@pcfsyg. Experiment vex file will be in the home area or in /usr/sched/ directory.

Now to drudg, run:

from oper@pcfsyg

drudg /usr2/sched/v255ah.vex
yg (choose antenna)
11 (show/set equipment type)
19 16 1 1 (inputs for equipment type)
3 (make a .snp file)
5 (print a summary file)
12 (make a proc file) 0 (done with drudge)

Now, move the proc file to /us2/proc/ directory at pcfsyg and snp and sum files to /us2/sched/ directory at pcfsyg
scp /usr2/sched/v255ahyg.prc oper@pcsfsyg://usr2/proc/v255ahyg.prc
scp /usr2/sched/v255ahyg.snp oper@pcsfsyg://usr2/sched/v255ahyg.snp
scp /usr2/sched/v255ahyg.sum oper@pcsfsyg://usr2/sched/v255ahyg.sum

After this is done, make sure you have proc and snp file in their relevant directories on pcfsyg (eg, v255ahyg.prc in /usr2/proc/ and v255ahyg.snp & v255ahyg.sum files in /usr2/sched/)

Pre-checkups before setting up for the experiment

Yaragadee12 field system is fs-flexbuff. If you are setup for a new experiment from the start, close the field system which is running on pcfs machine ((e.g on pcfsyg) by typing 'terminate' in Oprin window, and start the right field system by typing 'fs-flexbuff' in any window on oper@pcfsyg. But if there is already an experiment going on and you are doing a change-over setup, then you don't need to kill the fs (because most prob right field system is running)
Now you should check whether jive is running or not. For this from oper@pcfsyg, do ssh observer@flexbuffyg. Now, check jive by doing:
ps -ef | grep -i jive
if it returns jive5ab-3.1.0-64bit-Release, then it's running and good. but if jive is not running, then from observer@flexbuffyg:
StartJ5
Which automatically runs jive5ab-3.1.0-64bit-Release and logs the debug information into a log file under the /home/observer/jive5ab.logs/. That is convenient if we need to review what went wrong on a session. if that doesn't work, you can also start the jive5ab as:
jive5ab-3.1.0-64bit-Release
Make sure that dbbc is running by doing vncviewer dbbcyg from oper@pcfyg. Check the right version of dbbc is open (DBBC2 Control DDC_V_v106.exe). if not, close the one that is currently open and open the right dbbc version. it should be somewhere in the middle of desktop. Yg is uses DBBC2.
If you need to reconfigure or restart the dbbcyf, it is simple. you can use ipswiches website for that. wait for it to boot and then restart the server. it takes around 2-3 minutes. then again open up the vnc of dbbcyg by vncviewer dbbcyg. you'll need Password for that, which you can find either on this ra-wiki page or you need to ask to someone.

Now, start the right dbbc verson once you are inside dbbc3 desktop, and hit 'yes' to reconfigure.

It takes around 5-6 minutes for the system to fully configure. This usually fixes the problems.

Loading the procedure files

Loading the setup is very similar to hoq we do setup for AuScope experiments.

Now we want to load in the procedure files. Do following:

proc=v255ahyg
exper_initi
setup01
iread
bread
clkoff
maserdelay

dbbc=pps_delay mk5=rtime? # to check how much free space you have. if free space is less than 4-5%, let your on-call person know about it.

Now, do a scan test recording by doing following:

scan_name=test,v255ah,yg@disk_record=onmk5=evlbi?mk5=evlbi?mk5=evlbi?@@\\ disk_record=off

check that output number in fs keeps increasing each time when you run mk5=evlbi? and if we see that output number in fs keeps increasing each time, that means that data is being recorded.
Now do,

scan_check # this should show the latest recorded scan in field system.

also, from observer@flexbuffyg, do:
vbs_ls -lrth 'v255ah*' | tail -10

this will show your recorded scan the right name. this check will confirm that data is being recorded at right place.

after all above checks are done, you can start the esp by doing following:
schedule=v255ahyg,#1
onsource # antenna should be tracking or slewing and then tracking to the source.
list # this will show you the next upcoming scan details in fs

OR if late start, do:
schedule=v255ahyg,2021.210.03:00:00 (any time particular time from where you want to start the schedule, hint: time right now)

Monitoring:

Make sure that antenna is tracking and slewing to next source as well, and not struck. this can be done by doing onsource command in Oprin window.
Make sure that data is being recorded by running watch -n60 "vbs_ls -lrth 'v255ah*' | tail -30" on observer@flexbuffyg to check whether all scans are being recorded or not. also, run mk5=evlbi? multiple times in oprin window which scan recording is going on to check whether data is actually being recorded or not (the number in fs should increase every time)
Monitor for wind stows, should be obvious if you are monitoring the antenna moving. make sure that wind is ok from antenna_monitor window.
Eyes out for ERRORS in the field system log. There are a few benign ones that will regularly come up.
Check how much space is left in the disk to record by doing mk5=rtime? , to check how much free space you have. if free space is less than 4-5%, let your on-call person know about it.
Record experiment details to the appropriate ATNF wiki page. (Go here, click the session and then experiment number link, scroll down to "observing comments for each antenna" section and click Hb to update the Katherine 12 log and Cd to update the Ceduna log.) The login username and password can be found here.
These web pages should always be open:
ATNF VLBI monitoring page
ATNF recorder monitoring page
Local Mt Pleasant live page
Local Ceduna live page

Also, it is good to know how to halt and continue the setup during the experiment, in case if you need to do it during your observations. To halt the setup, do following from `Oprin` window:

halt
disk_record=off
and then to continue the setup, do following from oprin window:
setup01
cont
list

few points for observers:

Check every 30 mins during day shift and every 1 hour during night shifts
Always try to use vnc for monitoring
Make sure that you atleast check once every 30 mins that everything is running smooth or not
Always have LBA mattermost chat open when you are on observing duty so other people can interact with you

Trouble-shooting some common errors and probelms:

stuck antenna:

If antenna is stuck, and you see following errors in fs window:
2012.297.03:58:54.01?ERROR st -998 reading SystemClock1
2012.297.03:58:54.01?ERROR st -999 TCP/IP connection was closed by remote peer
2012.297.03:58:54.01?ERROR st -5 Error return from antenna, see Mbus error

OR

2012.297.03:58:54.01?ERROR st -998 antenna stuck

then you can do following things:
first, in Oprin window, type:

antenna=open
antenna=operate
antenna=status

If this doesn't solve the issue, then:

Halt the schedule using step described earlier on this page
Go to timeyg vnc by doing vncviewer timeyg

then, over there:

Go to the antenna control window. Look for red buttons there. turn off the antenna manually by doing from operate to standby. give it sometime to catch up after changing. then switch Drives off.
wait some time, then switch Drives back to on. After turning the drives on, it is recommended to wait a little while before putting the antenna in operate

This step is crucial if the previous steps haven't resolved the issue

This will generally bring back the antenna form stuck. but if this doesn't solve it, do the RESETS at the bottom. Basically at the end, we want green buttons at operate, drives on in antenna control window.

Note: sometimes, when antenna goes to windstow, it does not come back automatically after wind becomes fine. for that, you need to halt the schedule, reboot the timeyg machine using the ipswitch website, then login again to timeyg machine and open all the programs that were open before and make sure that everything is all right. Then terminate the field system, start it again and do a quick re-setup. this will solve the issue.