VEGAS: VErsatile GBT Astronomical Spectrometer

From Casper
Jump to navigationJump to search

VEGAS is a next generation instrument that will expand the capabilities of the current spectrometer at the Green Bank Telescope, to address the increased bandwidth and spectral resolution needed with the new 7-pixel Ku-band focal plane array receiver. The NRAO website gives a detailed description of the instrument and it's scientific objectives.

VEGAS Testing Notes


Marty Bloss, Patrick Brandt, Hong Chen, Jayanth Chennamangalam, Paul Demorest, Gregory Desvignes, Terry Filiba, John Ford, David Frayer, Bob Garwood, Glenn Jones, Joe Masters, Randy McCullough, Guifré Molera, Karen O'Neil, Aaron Parsons, Jason Ray, Anish Roshi, Simon Scott, Amy Shelton, Mark Wagner, Galen Watts, Dan Werthimer

Block Diagram

Error creating thumbnail: Unable to save thumbnail to destination


ROACH Firmware

  • BORPH - A modified Linux kernel that runs on the PPC, and allows for interaction with shared memory on the FPGA.
  • KATCP - Allows for remote command and control of the firmware, over the network.
  • Roach Monitor - Monitors voltages and temperatures of ROACH boards.

HPC Software

The HPC (which is a cluster of CPUs and GPUs) will be running a modified version of the backend software for GUPPI (guppi_daq). There are two different versions of the HPC software:

  • vegas_hpc_hbw : the HPC software for high-bandwidth modes (the CPU merely accumulates)
  • vegas_hpc_lbw : the HPC software for low-bandwidth modes (the GPU implements a PFB)

Depending on the desired mode, one of the two HPC applications must be run on the HPC. Note that a separate instance of the HPC application must be started on each CPU/GPU node in the cluster.

Starting and Stopping the HPC Software

To start the HPC software, simply run an instance of vegas_hpc_* application on each CPU. The application does not require any command-line parameters. The one optional parameter that can be passed to the process is "--disk", which causes the application to write SDFITS files for debugging purposes.

To stop the HPC software, simply kill the process on each CPU node.

Outputting Data

By default, the HPC software does not write to disk. The HPC software writes integrated spectra to a shared memory buffer. The M&C process (external to the HPC software) then reads the spectra from the shared memory buffer and writes the spectra to disk.

However, if the "--disk" flag is passed to the HPC software, via the command line, the HPC software will create a disk thread which writes the spectra from the output buffer to SDFITS files on disk. This mode is merely for testing purposes, and will not be used during normal operation.

Shared Memory Buffers

Details on the shared memory buffers can be found in the Memo on the Shared Memory Buffers and Memo on the Critical Settings for the HPC, both of which can be found in the section on Technical Documentation on the VEGAS HPC Software Architecture below.

Installing the HPC Software

Installation notes for the VEGAS HPC software can be found here [Installation notes]

Technical Documentation on the VEGAS HPC Software Architecture

Older documents that have been subsumed by the HPC Developer Documentation:

Network Packets

[SPEAD] protocol - The SPEAD protocol allows for a very robust and flexible means for streaming packets. This protocol is used for packets that are sent from the FPGA to the HPC. Detailed information of the packet format at [VEGAS Packet Format]



The firmware is the code that runs on the FPGA. It is available from the VEGAS project [git repository] (deprecated: [gbt_devel git repository]). We are using the 11.5 MSSGE toolflow for all modes.

Monitor and Control of the ROACH Boards

Vegas FPGA Register Definitions

Vegas SSG Modes

All shared memory on the FPGA will be accessed over the 100Mbit port using the corr and katcp python packages that communicate with the tcpborphserver running on the PPC. Monitoring of ROACH voltages and temperatures will be done by the Actel Fusion and can be accessed via the Xport (see ROACH Monitor for details).

| Mark's google doc about VEGAS modes and configurations

Anish's SSG specifications