April 2005 ALMA Offline software test NAME: Rein Warmels Questionnaire on testing experience 1. Please list briefly your background in the following areas: A. Radio Interferometry ((sub)millimeter or centimeter) >> Experience with cm data (WSRT) B. Experience with VLA and/or PdBI data >> none C. Astronomical Data Reduction packages: - AIPS - MIRIAD - MMA - Gildas/Clic - AIPS++ X Gipsy X MIDAS D. How much experience have you had with the AIPS++ software package before this test? >> Experience acquired from previous test 2. Please identify which dataset you processed during this test: >> A. VLA continuum observations of Orion B. BIMA CO(1-0) observations of NGC 4928 3. Were you able to combine the single dish and interferometric data using feather and deconvolution techniques? If not, why? Please comment on specific steps if desired (comments can be positive or negative, you may not have tried all steps): A. Feather images or image cubes provided by Offline subsystem >> I'm not quite sure what is meant here but I was able to feather the MEM Cleaned VLA image and GBT that were provided via the Web and on the DVD. I did not experiment with other than the Orion data. B. Feather single dish image provided and interferometer image that you created from the dataset provided. >> I was able to (re)produce the MEM Clean VLA-only map from the MS that was provided. Subsequently I was able to feather this map and the GBT map. The resulting feathered map contained the total flux that was found in the GBT map. C. Deconvolve the single dish and interferometer data using the single dish image to create an input model. >> I used the direct method: Directly convert the GBT to a model and use this as a starting point. This worked well. I ran several deconvolutions using various parameter settings. The results (with exception of a 10% flux discrepancy) were consistent with the feathering results. Please identify any problems you had during imaging. >> I found that after feathering the peak flux was lower that in the VLA only map. This made me believe that there was sometimes wrong with the GBT data. The details were explained later to me and the flux calibration of the GBT map was corrected. I did not encounter other problems. 4. Were you able to analyze the images adequately to determine if the results you obtained were scientifically reasonable (e.g. display the image, calculate RMS and peak, make a moment map or take a spectrum)? If not, why? >> I was able to the statistics done quickly. Since I only had a continuum map not much analysis was required. 5. Please summarize the final results of your image(s): Total Flux Peak RMS Mean VLA MEM Cleaned 185 0.93 8.7 -2.8 VLA+GBT2 feathered 387 1.07 21.5 18.9 VLA + GBT2 Combined MS 432 1.01 23.5 19.6 The total flux in the feathered map is consistent with the flux measured in the GBT (version 2) of 387 Jy. The flux in the combined deconvolved images is found on the high side. 6. Did you have adequate support during your test? If you contacted the AIPS++ groups for questions or to fix a bug, please comment on the interaction and whether it was helpful. >> I had several contacts with the support group. I obtained quick and very helpful answers. The flux problem I reported was promptly investigated and resolved. 7. Was AIPS++ easy to install? If not, why? Yes, the new installation went rather smooth and without problems. >> The problems that were reported with TST11.1 did not appear. 8. The Synthesis Reduction Cookbook you used for this test is the second version of a comprehensive cookbook for ALMA users. Please evaluate the organization, content, and presentation of the cookbook. It is meant to be the first documentation users will see when they want to reduce ALMA data, it provides background on the code capabilities, and extensive examples. The on-line documentation provides more details and code descriptions. With this in mind, please answer the questions below. If you have detailed comments, please attach them to the end of this questionnaire. - Was the documentation adequate for you to complete your test? - Do you have any suggestions for how to improve the cookbook? - Was the on-line documentation helpful: * User Reference Manual? >> My consultations of the Ref. manual were limited. See my Test 1.1 comments on the Ref. Manual. * Supporting documentation? >> In my previous report (TST1.1) I stated that the Cookbook was already in a good and well developed state. Scanning through the latest version (without again reading it in detail) I noticed a number of further improvements and additions. I do like the organisation as well as the style in which the Cookbook is written. I believe the content is well suited for novice interferometry users. For more inside knowledge of AIPS++ the Getting Started AIPS++ manual are useful. Concerning the additions in the Imaging Chapter. The additional hints for the various cleaning method is very useful. The of the (sub)section 5.3.2 to 5.3.4 I found not so logical: 5.3.2 Deconvolution * Clean - 5.3.2.1 Clean Hints * MEM 5.3.2.2 MEM Hints 5.3.3 Specifying the deconvolution region 5.3.3.1 Interactively define a region 5.3.4 Multi-scale CLEAN 5.3.4.1. General 5.3.5 MS Clean Hints I would suggest to first discuss the all clean methods, including the multiscale method and then finish the section with How to specify a deconvolution region. The text on feathering (section 5.6.2) is slightly more difficult to understand (at least for the novice user). I would recommend to make this description slightly more comprehensive or to rephrase the text. 9. Roughly how much time did you take to perform the following steps: - Installing aips++: >> 0.5 hours - Imaging: >> 5 days, including some learning curve for MEM, feathering and combined deconvolution - Analysis: >> 1 days - Filling out this questionnaire: >> 2 hours - Evaluating and grading the scientific requirements: >> 2 hours - Total time: >> 6.5 days Note: I played a lot with the various imaging options. In the numbers above this time is not accounted for. 10.Please rate your overall testing experience: x excellent - good - fair - poor - horrid 11.Was the test well designed and executed by those in the ALMA offline subsystem (e.g. the subsystem scientist and the Offline subsystem group). If not, can you provide any suggestions for improving the next test? >> I believe so, except for the small confusion regarding the flux scale of the GBT map 12.Do you have any additional comments that may help improve test of the offline software in the future? >> A positive comment (again): I believe that the success of these tests critically depends on the interaction between the tester and the supporting staff. My experience of this aspect is extremely positive and I would like to thanks the ALMA/NRAO for this level of support.