October/November 2004 ALMA Offline software test 

NAME: Carlos De Breuck

Questionnaire on testing experience

1. Please list briefly your background in the following areas:

   A. Radio Interferometry ((sub)millimeter or centimeter)
      10 year experience with cm interferometry
       3 years experience in mm interferometry (PdBI only)

   B. Experience with VLA and/or PdBI data 
      10 year experience with VLA data, but only P to X band.
       3 years experience with PdBI data; mainly imaging in classic AIPS

   C. Astronomical Data Reduction packages:
      - AIPS: proficient, 10 years experience. 
      - MIRIAD: have reduced ATCA data with it on several occasions
      - MMA: never used
      - Gildas/Clic: basic experience in collaboration with IRAM staff
      - AIPS++: never used

   D. How much experience have you had with the AIPS++ software
      package before this test? 
      Never used, not installed AIPS++.

2. Please identify which dataset you processed during this test:

   B. BIMA CO(1-0) observations of NGC 4928

3. Were you able to complete the fill, editing, and calibration of the
   data?  If not, why?   Please comment on specific steps if desired
   (comments can be positive or negative): 

   A. Filling the data into AIPS++ format
      Quite easy and fast following the cookbook recipe. It is not always easy to know when a task is finished or still computing. This is a minor inconvenience, but most people are used to get a proper 'task finished' message at each step.

   B. Editing and visualizing your data
      1. MAJOR HANGUP: couldn't exit from the msplot GUI after selecting not to save flags (user choice warning popup window). Had to kill AIPS++ from UNIX prompt. Even this doesn't help; the new GUI tool hangs on startup. Had to delete the entire measurement set, and create it again.
      2. The selection of the spectral windows is very cumbersome, with many resulting 'No data selected - selection criteria is too narrow' messages. It would be better to provide the available range of spectral windows and channels for the selected Field ID. This would significantly speed up the flagging.
      3. Another BUG: I tried flagging on the day 2 data set, but didn't do any flagging. I then successfully closed the plot tool, and wanted to start it up again. This gives the following errors, and the GUI crashes:
- plot:=msplot(msfile='allb.ms',edit=T);
<fail>: Problem opening the measurement set. The error was:
F
        File:   note.g, Line 59
        Stack:     throw(), msplot.g line 6077
                       .(), msplot.g line 6115
                  msplot() 
- plot.done();                           
error, <fail>: operand to .done is not a record
        File:   msplot.g, Line 6118 is not a function value
F 
- plot:=msplot(msfile='allb.ms',edit=T);
<fail>: Problem opening the measurement set. The error was:
Cannot create the flag table called[i_am_unset=i_am_unset]:create: Table allb.ms.flags.1 already open; maybe close it first
        File:   note.g, Line 59
        Stack:     throw(), msplot.g line 6077
                       .(), msplot.g line 6115
                  msplot() 


   C. Atmospheric phase corrections (PdBI data only)
      N/A


   D. Gain (phase and amplitude) calibration
      Quite easy and similar to other radio packages. Only the syntax is different (and unfortunately more elaborate than in e.g. classic AIPS).


   E. Absolute flux calibration 
      Same remark as before. Quite easy.


   F. Bandpass calibration
      N/A, I reduced the BIMA data.


4. Were you able to subtract continuum in the uv-plane (if
   applicable)?  If not, why? 
   This BIMA data set does not have a continuum.

5. Were you able to split out the calibrated data (if desired)?  If
   not, why? 
   Yes, without major problems, following the cookbook recipe.


6. Were you able to image the data?  If not, why?  Please identify any
   problems you had during imaging.  
   Yes, no major problems following the cookbook recipe. The actual computation of the deconvolved spectral cube mosaic took <5 minutes of computing time. The syntax to give the commands is very elaborate and verbose, though.

7. Were you able to analyze the images adequately to determine if the
   results you obtained were scientifically reasonable (e.g. display
   the image, calculate RMS and peak, make a moment map or take a
   spectrum)?  If not, why? 
   Yes, no major problems. The default viewing tool is quite good, probably one of the best pieces of the entire AIPS++ system. It is a bit confusion that some tools such as the imageprofilefitter are not part of dv. But in general, this tool is quite straightforward in use, and allows a good examination of the data. Most of the functions are quite well explained in the pop-up help, so I didn't really have to use the cookbook very often (though I did read chapters 6 and 7). I have some minor comments on the production of publication-quality images, listed below.

8. Please summarize the final results of your image(s): 
   - RMS: 9.5 Jy/beam.km/s
   - Peak Flux Density: 105.2 Jy/beam.km/s

9. Did you have adequate support during your test?  If you contacted
   the AIPS++ groups for questions or to fix a bug, please comment on
   the interaction and whether it was helpful.  
   I had no major problems during the test, and didn't really need to contact the AIPS++ group.

10.Was AIPS++ easy to install?  If not, why?  
   Except for a small hick-up due to the wrong placement of the casa_install script, the installation went very smoothly, and quite fast.


11.The Synthesis Reduction Cookbook you used for this test is the
   first version of a comprehensive cookbook for ALMA users.  Please
   evaluate the organization, content, and presentation of the
   cookbook.  It is meant to be the first documentation users will see
   when they want to reduce ALMA data, it provides background on the
   code capabilities, and extensive examples.  The on-line
   documentation provides more details and code descriptions.  With
   this in mind, please answer the questions below.  If you have
   detailed comments, please attach them to the end of this
   questionnaire. 

   - Was the documentation adequate for you to complete your test?
   Yes, I followed the step-by-step guide, which has sufficient detail.

   - Was the cookbook good?
   Yes, I could find all I need in the cookbook.

   - Do you have any suggestions for how to improve the cookbook?
   I would provide the datasets for this or any other test for people to get used to AIPS++ before trying to reduce their own data. I would find it a lot harder to start without having a clear example to follow.

   - Was the on-line documentation good:
      * User Reference Manual?
        I didn't really use this much, as most is in the cookbook.
        However, the search tool on the AIPS++ homepage is really bad (http://aips2.nrao.edu/stable/docs/search/search.html). I tried using it a few times, but never found anything close to what I was looking for (e.g. how to start up the logger, how to get the paper colors in the dv tool, ...).
    
      * Supporting documentation?
        Didn't use much other documentation, except the 'detailed data descriptions'.

12.Roughly how much time did you take to perform the following steps:

   - Installing aips++: 1 hour
   - Fill, editing, & calibration: 5 hours
   - Continuum imaging:	N/A    
   - Spectral line imaging: 1 hour 
   - Analysis: 16 hours    
   - Filling out this questionnaire: 2 hours
   - Evaluating and grading the scientific requirements: 1 hour
   - Total time: 30 hours

13.Please rate your overall testing experience: 

   - good
   - fair
   (the test itself was good, the software itself less so because it is not user-friendly)

14.Was the test well designed and executed by those in the ALMA
   offline subsystem (e.g. the subsystem scientist and the Offline
   subsystem group).  If not, can you provide any suggestions for
   improving the next test?  
   The BIMA data set was quite easy to reduce. This makes a good first introduction to AIPS++. I found the step by step guide a very useful thing to have, and would recommend it for the next tests. Maybe there was even too much detail in it. The only thing I really invested time in is how to use all the functionalities of the default viewer.

15.Do you have any additional comments that may help improve test of
   the offline software in the future?  
   Use data sets with good data, such as day 1 of the BIMA data set, so that people can really test the functionality of the software rather than being distracted by trying to understand low S/N data.


Additional comments and recommendations:

1. As a first time AIPS++ user, I was quite satisfied with the structure of the measurement sets. This is definitely better than e.g. the SN and CL tables in classic AIPS. Also, the default viewer tool is much better, as it is a nice interactive tool, with functionalities lacking in other software packages.

2. My worst experience with AIPS++ is the lack of a clear overview of what options one can give to a tool or function. There is nothing like a list of input parameters as in classic AIPS or miriad. In general, AIPS++ is too much script oriented. This works fine for standard reductions, but this is not what one needs to squeeze the most out of one's data. The syntax is way too verbose for such detailed reductions. It would be very hard to impossible to use AIPS++ without the cookbook or on-line help. The software will need to be made a lot more user-friendly before it can be used by a large pool of astronomers instead of programmers.

3. The ms.summary output is way too long. It's very difficult to find what one really needs in this. There should be an option to produce a short summary, similar to the imheader in classic AIPS.

4. There are a few minor things missing in the default viewer such as (i) an option to plot the beam as a hashed region with a box around it, (ii) the ability to attach different panels together without a white space in between them (the margins in the canvas manager don't really solve this), (iii) the ability to have different labels on each individual panel (e.g. plotting the beam only on a single panel), (iv) when producing a postscript output of a multi-panel plot, the beam moves partly outside of the windows.

5. In imageprofilefitter, one cannot change the name of the output postscript file. 

6. In the spectra produced using positions of the default viewer, there should be an option to plot these spectra as histograms instead of a connected line (histogram should be made the default). It would also be good to be able to export to 'spacebar-frozen' one-pixel spectrum into an ascii output file.

7. When writing out figures in EPS format, they don't print on our printer. This is unlike classic AIPS EPS files, which print without a problem (and have human-readable postscipt).