April 2005 ALMA Offline software test NAME: Crystal Brogan Questionnaire on testing experience 1. Please list briefly your background in the following areas: A. Radio Interferometry ((sub)millimeter or centimeter) extensive B. Experience with VLA and/or PdBI data VLA extensive/ PdBI none C. Astronomical Data Reduction packages: - AIPS extensive - MIRIAD some - MMA none - Gildas/Clic none - AIPS++ some D. How much experience have you had with the AIPS++ software package before this test? I was part of the NRAO aips++ testing group in 2001-2003. Much of this testing involved the gui's so I am not an expert in glish. I also took part in the first two external aips++ ALMA tests. 2. Please identify which dataset you processed during this test: A. VLA continuum observations of Orion 3. Were you able to combine the single dish and interferometric data using feather and deconvolution techniques? If not, why? Please comment on specific steps if desired (comments can be positive or negative, you may not have tried all steps): Before trying either of the combining techniques, I tried to establish whether MEM or multiscale (MSC) would give the best VLA only image. Although the final images that I obtained were fairly similar, it took a very long time to determine the best parameters with MEM. I would not use this task in future. B. Feather single dish image provided and interferometer image that you created from the dataset provided. Using my best MSC VLA only image and the rescaled GBT image Feather provided a pretty nice image. C. Deconvolve the single dish and interferometer data using the single dish image to create an input model. I found the NGC4826 specific deconvolve directions pretty difficult to disentangle for the case of Orion. No real guidelines are given in the cookbook for how the parameters were chosen etc or why you would want to use multiscale clean to do the deconvolution. The final image looked good however. Please identify any problems you had during imaging. 4. Were you able to analyze the images adequately to determine if the results you obtained were scientifically reasonable (e.g. display the image, calculate RMS and peak, make a moment map or take a spectrum)? If not, why? Yes, these measurements were the easiest part, although saying which image is the "best" is still pretty subjective through no fault of aips++. 5. Please summarize the final results of your image(s): - RMS: - Peak and Total Flux Density: Feather: Peak: 1.06 Jy/beam Total Flux (whole map): 381 Jy Total Flux (Orion): 292 Jy RMS: 7 mJy/beam mean in rms: 4 mJy/beam Deconvolution: Peak: 0.97 Jy/beam Total Flux (whole map): 231 Jy Total Flux (Orion mask region): 212 Jy RMS: 10 mJy/beam mean in rms: 5 mJy/beam GBT2 (regrided): Peak: 62 Jy/beam Total in whole map: 386 Jy Total flux in mask region: 283 Jy A couple of comments about these results. I got a very strange result for the deconvolution result unless I used the VLA mask in making the clean components (weird artifacts outside of mask region). However, no such mask was used on the GBT image for the feathering technique, so that its emission is much more extended and the total flux is significantly higher. This result is not surprising, and the total fluxes are in good agreement with those for the total region, and just mask region for the feathered image. Since the two methods (at least the way I have implemented them) are not really directly comparable, I would hesitate to say which is "better". 6. Did you have adequate support during your test? If you contacted the AIPS++ groups for questions or to fix a bug, please comment on the interaction and whether it was helpful. All responses were almost immediate and very helpful, including a phone consultation at one point. 7. Was AIPS++ easy to install? If not, why? As with last time, I had trouble installing new aips++ on top of old. Everyone was helpful in solving the problem though. Somehow this needs better pre-release testing. 8. The Synthesis Reduction Cookbook you used for this test is the second version of a comprehensive cookbook for ALMA users. Please evaluate the organization, content, and presentation of the cookbook. It is meant to be the first documentation users will see when they want to reduce ALMA data, it provides background on the code capabilities, and extensive examples. The on-line documentation provides more details and code descriptions. With this in mind, please answer the questions below. If you have detailed comments, please attach them to the end of this questionnaire. - Was the documentation adequate for you to complete your test? More or less. - Do you have any suggestions for how to improve the cookbook? * Interleaving of ME and MSC parameters in the cookbook is very confusing. A list of all possible parameters for each task separately would be much easier to understand. * A list of all available cleaning algorithms should be listed in the cookbook. * Why have "threshold" for MSC and "sigma" for ME? * What does "threshold" mean for MSC? For which scale is this the threshold. * I found the flux scale discussion very confusing. Do you have to put something? What is the best one to use? What is the flux scale if you leave the default? - Was the on-line documentation helpful: * User Reference Manual? Yes (although I didn't look up very many things) * Supporting documentation? The supporting documentation was very helpful, although it isn't clear to me that the recommendations don't preclude an accurate assessment of the cookbook. 9. Roughly how much time did you take to perform the following steps: - Installing aips++: 2 hours (due to trouble installing) - Imaging: 12 hours - Analysis: 1 hour - Filling out this questionnaire: 1 hour - Evaluating and grading the scientific requirements: 1 hour - Total time: 17 hours 10.Please rate your overall testing experience: - good 11.Was the test well designed and executed by those in the ALMA offline subsystem (e.g. the subsystem scientist and the Offline subsystem group). If not, can you provide any suggestions for improving the next test? The test was well designed and implemented. Maybe too well... Although it certainly makes things go faster, I do not believe that so many recommendations (image size, cell size, etc) should be given. This information makes it difficult to assess whether cookbook along with information about the observations would be enough to make informed reasonable choices that actually work. Indeed, when I tried to deviate from the recommendations for the image size in MEM (600" to 800"), based on my imaging experience (which suggested a larger image size might work better) the image complete failed to converge in a way that I still don't understand. 12.Do you have any additional comments that may help improve test of the offline software in the future?