Tas PGA

The TasPGA Spectrometer Observer's Guide

This page provides instructions for the TasPGA FFT spectrometer system. Currently, version 1.8.6 of the FPGA firmware is in place at Mt. Pleasant. The basic operational specifications of this system are as follows:

Two polarisations: XX,YY or RR,LL
Fixed number of frequency channels, currently 16384
Variable hardware integration time, between 2 and 32 seconds
Two modes of operation, wide-band and narrow-band (with different FPGA firmware)
PC-based control system offering real-time data monitoring
Data recorded to disk as SDFITS files
Remote control possible via a network socket (automated observing)

Data are captured using a MAX105 dual-input A/D converter, which has 6 bit resolution and a maximum bandwidth of 400 MHz. The A/D converter is clocked by a signal generator that determines the sampling rate (and hence the bandwidth) of the spectrometer and is phase-locked to the station maser for coherence. The digitised sampler clock is sent with the data to the FPGA to drive the processing engine. Radio astronomy signals must be mixed to baseband and connected to the A/D inputs by means of under-floor cables. The frequency channel amplitudes are accumulated within the FPGA and can be read out by the on-board CPU at the end of every integration cycle. Integration cycles are synchronised with the station 1-PPS and must be at least 2 seconds long. The output is transmitted to a host PC (currently vortex) via 100 Mb Ethernet over the observatory's internal network.

Presently, observing bandwidth flexibility is limited by the availability of analogue filters which are required to band-limit the signal sent to the A/D converter. The best way to observe a small bandwidth is to route the radio signal into the MkIV rack and use two of the VCs to mix and filter the two polarisations. The PC Field System can be used to automate the setup process (see below).

Hardware Setup

The FPGA board is presently located in the "back room" of the observatory, next to the Origin 3400 supercomputer. It should not often be necessary to interact with the hardware in the FPGA rack. The FPGA "firmware" is automatically loaded when the system is powered and it can safely be left running all the time, provided the signal generator is supplying a clock signal. DO NOT leave the FPGA powered if the signal generator at the top of the rack is disabled or the A/D converter is turned off. It should only be necessary to visit the FPGA rack to change the bandwidth of the spectrometer or power cycle the board if it fails to respond.

Step 1: Decide whether you want to use a bandwidth less than 32 MHz or greater than 32 MHz. You will need to change the FPGA firmware to suit. The wide-band configuration has no provision for under-sampling and therefore cannot go below a bandwidth of 32 MHz (the clock manager in the FPGA has a minimum input period). If you want to observe a bandwidth smaller than 32 MHz, you should use the narrow-band firmware, which internally under-samples the incoming data by a factor of 4. Bandwidths as small as 32/4 = 8 MHz are possible with the narrow-band firmware. In either case, the bandwidth is defined by setting the frequency of the signal generator at the top of the FPGA rack. When using the wide-band firmware, the signal generator should be set to TWICE the required observing bandwidth and when using the narrow-band core, the generator should be set to EIGHT TIMES the required bandwidth.

Step 2: If necessary, load the wide- or narrow-band FPGA firmware into the TasPGA board. This must be done from the PC in the back room. You will first have to plug the multi-coloured parallel cable (usually found hanging on the FPGA rack) into the computer under the desk. Also ensure that the TasPGA configuration header (two rows of pins just under where the parallel cable connects to the board) does NOT have any jumpers across it. This header can be used to select which FPGA appears on the programming chain. The default (no jumpers) is the Virtex 4. Then, log on as the user FPGA and double-click the "iMPACT" icon in the middle of the desktop. Choose to load a pre-existing project. If you click the pull-down selector menu, you should see two project (.ipf) files, one for the wide-band firmware and one for the narrow-band firmware. Select the one you need. Once iMPACT has opened, right-click on the second device icon in the JTAG chain and select "program". Make sure you select the Verify, Erase Before Programming and Load FPGA boxes in the dialog that pops up. This will re-write a flash memory device on the board so that the core you have selected will start up by default, even after a power failure. This process is not very user-friendly, eventually it should be possible to program the core from the spectrometer GUI simply by selecting a mode of operation, but currently there is no physical connection or software that would allow this.

Step 3: Supply a baseband IF signal to the A/D inputs. There are two long cables ending in BNC connectors that exit the 14 m rack in the control room, one with a green sleeve and the other with a blue sleeve. These connect to the A/D inputs in the FPGA rack. You must connect these cables to a baseband signal that is band-limited at the bandwidth you wish to observe. There are a number of ways to mix your observing band down; the method you choose will depend on the total bandwidth required.

Method 1: 8 or 16 MHz bandwidth per polarisation

The best narrow analogue filters we have are in the MkIV VC units. It is possible to use these to observe in parallel with the ATNF correlator by tapping into the IF monitoring ports on the DAS frequency translator. Using this method, the FPGA spectrometer will see exactly the same signal presented to the ATNF correlator and you only need to set the Agilent and SMY01 oscillators as if you were observing with the correlator alone. The IF monitor outputs can be split in the panel immediately above the frequency translator, after which one output can be connected to the 16 MHz filters and back to the square law detectors and the other can be connected to the FPGA frequency converter (located immediately above the DAS). Unfortunately, the IF distributor in the MkIV rack has a band-pass filter that cuts out at 96 MHz (the same as the centre frequency produced by the IF monitoring ports on the DAS frequency translator). It is therefore necessary to mix the output from the frequency translator UP slightly before sending it to the MkIV rack. This can be done by feeding the outputs from the DAS frequency translator into the FPGA frequency converter and giving it a suitable LO with SMX02. At the moment, you should use 54.0 MHz. This will mix the 96 MHz signal from the DAS frequency translator UP to a centre frequency of 150 MHz, well inside the acceptable range for the MarkIV rack. You should connect the outputs from the FPGA frequency converter to the IFD ALT1 and IFD ALT2 inputs above it. Next, run the PCFS (field system) on hobart. Load the spect.prc procedure with the command:

proc=spect

Then, run either setup8 or setup16 in the field system to configure the MarkIV rack (depending on the bandwidth you want). The blue and green cables hanging by the MkIV rack and labeled FPGA EXT1 and FPGA EXT2 should then be connected to the USB outputs of VC03 and VC10 respectively (these are also labeled).

Method 2: wide-band operation

To switch to a broader band, you will have to switch the outputs of the FPGA frequency converter from IFD ALT1 and IFDALT2 to the inputs of the two rotary attenuators sitting directly above the FPGA frequency converter. The inputs to the FPGA frequency converter should be connected to a split copy of the IF coming from the LHC AND RHC ports in rack 3, or from A21 and A22 if using the S/X band receiver. You may have to experiment with levels of amplification in order to make this work. Set the SMX02 LO to whatever frequency puts your band of interest in the middle of the range specified by the low-pass filters that come after the rotary attenuators above the FPGA frequency converter. You will have to take the Agilent and internal 5.2 GHz LOs into account when working out this number.

Once a baseband signal is connected, you should check the levels by observing the sampler statistics, as described in the next section, and adjust the rotary attenuators above the FPGA frequency converter if necessary.

Software Setup

Normally, you will interact with the FPGA spectrometer via its host PC, vortex. This machine is located below the MkV disk recorder. The monitor and keyboard located at eye-level in the same rack service several machines but can be switched to vortex by pressing the appropriate button on the KVM switch above the monitor. Vortex is linked to the FPGA board by a serial (RS232) communications line and it is also the network destination for all Ethernet packets transmitted by the FPGA board. All configuration, monitoring and data capture tasks are administrated via a graphical user interface that can be started with the command "qtspec" from the usual "observer" account. The interface is very self-explanatory and most buttons have help text that can be accessed by pressing the question mark tool-bar button and then clicking on the object of interest. Note that qtspec is usually left running and you will probably be able to find it on one of vortex's virtual desktops.

The "System Reset" button on the GUI should be pressed after any change in the signal power level to the A/D converter. This causes the firmware to re-compute the optimal scaling to perform during its calculations.

Set the hardware integration time by moving the slider to the appropriate position. Be aware that the storage space used for accumulating the amplitude in each channel is only 32 bits wide and can overflow if you integrate for too long. Keep an eye on the amplitudes using the real-time monitoring plot and if they exceed roughly 4E9, use a smaller integration time. You can always average the integrations in software by setting the cycles to average field to something other than 1, so there is no real need to shift the hardware integration time to anything other than 2 seconds (the minimum). The other fields in the SDFITS section do not control the recorder in any way and are simply written to the file header when recording to disk. Note that the sign of the bandwidth is important as it determines the frequency ordering when you load the data into asap.

The optimal IF power level can be determined by observing the cumulative histogram of the A/D converter statistics, which can be activated by clicking the Stats box in the Plot panel. The red and blue histograms plot the number of times a sample has fallen into one of the 64 (2^6) bins of the A/D converter. These bins represent different levels of positive and negative voltage. Bins near the middle of the histogram represent small voltages and bins near the edges represent large positive (right) or negative (left) voltages. Assuming that our astronomical signal is well approximated by Gaussian noise, we should expect the histograms to appear roughly Gaussian in shape, with more samples falling close to the middle than near the edges. To obtain the best dynamic range, it is important to set the power level of the input signal so that the "wings" of the Gaussian are close to the edges of the plot. That way we are using all 6 bits effectively. If the histogram is bunched near the centre of the plot, the power level is too low. If there are many samples in the first and last bins, the power level is too high.

Depending on the shape of the bandpass and the presence (or otherwise) of strong spectral features, you may want to plot the integrated spectrum using a linear or a logarithmic power axis. You can switch modes using the "Log" check-box near the bottom of the Plot control panel. You can zoom in on any region of the spectra by clicking and dragging. To zoom back out, press the "-" key or the middle mouse button.

After you have adjusted the power level to obtain optimal A/D statistics, always be sure to press the "System Reset" button. After an integration or two, the spectra should settle down to a stable state. Once you have a smooth, stable spectrum with good statistics, you are ready to record the data to an SDFITS file on disk. To do this, simply fill in the relevant header parameters in the fields provided and press the "Record" button. A negative bandwidth means that channel zero has the highest frequency and vice versa. If you use the 100 MHz FPGA baseband down-converter above the ATNF DAS, you should enter a bandwidth of -100.0. A file name will be created automatically (from the date and time) and a new file will be opened in /data/raid0/spectra (also pointed to by the environment variable $SPECTRA). Every integration will be inserted into this file as a new row in the SINGLE DISH binary table. When you are ready, you can click the "Stop" button to close the file. If you want to decrease the number of integrations in the file, you can average several together in memory before writing them to disk by setting the Cycles to average parameter in the SDFITS panel to some number greater than one.

It is possible (and desirable) to control qtspec over the network using the same type of schedule that is used to drive the ANTF correlator. Instead of using the survey_ng program, use survey_dual (to control both instruments simultaneously) or survey_fpga with all the same arguments.

Vortex has "fv" (the standard FITS file viewer) and "asap" (the ATNF Spectral line Analysis Package) installed. Please feel free to use it to reduce your data. If you would like to have any other programs installed, contact Aidan. Alternatively, you can copy your files back to Ares for processing and storage. The /data/raid0 partition on vortex is located on a 4-disk RAID0 array and is quite large. It should however be considered temporary storage, because a single disk failure will destroy all the data.

Troubleshooting

Users should note that the TasPGA board currently installed at Mt Pleasant has one broken thermal sensor. The middle temperature dial on the GUI will always read 80 C. Please disregard. The other two dials should be trusted and their full-scale range represents the safe operating parameters for each chip.
If you encounter an error stating that a particular port is unavailable when trying to start qtspec, you can be pretty certain that someone else is running a copy of the software, perhaps from a terminal whose output you cannot see.