April 2005 ALMA Offline software test NAME: Carlos De Breuck Questionnaire on testing experience 1. Please list briefly your background in the following areas: A. Radio Interferometry ((sub)millimeter or centimeter) 10 years of experience in radio interferometry and 3 years with mm interferometry. B. Experience with VLA and/or PdBI data 10 years of experience with VLA data, and 3 years with PdBI data. C. Astronomical Data Reduction packages: - AIPS 10 years experience. - MIRIAD 3 years moderate experience. - MMA Never used. - Gildas/Clic Assisted to data reduction in Grenoble a few times, but only for simple projects. - AIPS++ Participated in TST1.1. No experience before that. D. How much experience have you had with the AIPS++ software package before this test? Participated to TST1.1. I have since used the dv.gui for some of my own science (making high-resolution overlays). No scripting experience. 2. Please identify which dataset you processed during this test: B. BIMA CO(1-0) observations of NGC 4928 3. Were you able to combine the single dish and interferometric data using feather and deconvolution techniques? If not, why? Please comment on specific steps if desired (comments can be positive or negative, you may not have tried all steps): A. Feather images or image cubes provided by Offline subsystem Combination works very fast and without problems. B. Feather single dish image provided and interferometer image that you created from the dataset provided. N/A. C. Deconvolve the single dish and interferometer data using the single dish image to create an input model. I tried both options described in the cookbook: 1) directly using the SD image as input model for the joint deconvolution. This takes ~1 minute of computing time and provides reasonable results. 2) deconvolving the SD image using multi-scale CLEAN, and using this as a model for the joint deconvolution. This takes much more computing time, roughly 30 minutes to go through all 30 channels. The result is quite similar (or worse?) than the first procedure. But it does work. Please identify any problems you had during imaging. See the 2 bug reports at the end (on using the mask image and total flux determination). Apart from these bugs, the software works for these applications. 4. Were you able to analyze the images adequately to determine if the results you obtained were scientifically reasonable (e.g. display the image, calculate RMS and peak, make a moment map or take a spectrum)? If not, why? There was a bug prohibiting determination of the total flux if the units are Jy/Beam.km/s instead of Jy/Beam. See bug report below. I could display and examine all the images in the default viewer by defining a polygon in the moment 0 image. I also defined a source-free polygon to determine the RMS noise levels. See result in point 5. I found the overlays to be more sensible than the slices are spectra, especially because it does not seem possible to easily overlay spectra of different images extracted at the same sky position. 5. Please summarize the final results of your image(s): (all RMS values measured off-source) - RMS in the SD cube: 0.61 Jy/beam - RMS in the SD moment 0 image: 50 Jy/beam.km/s - RMS in the BIMA cube: 0.092 Jy/beam (extend channels 1-30) - RMS in the BIMA moment 0 image: 8.7 Jy/beam.km/s - RMS: 4.38 Jy/beam.km/s in the feathered image - RMS in directly converted SD + CLEAN cube: 0.10 Jy/beam - RMS in directly converted SD + CLEAN moment 0: 13.6 Jy/beam.km/s - RMS in MS-CLEAN deconvolved SD + CLEAN cube: 0.10 Jy/beam - RMS in MS-CLEAN deconvolved SD + CLEAN moment 0: 12.1 Jy/beam.km/s - Peak in the SD cube: 6 Jy/beam (extend channels 1-256) - Total Flux Density in the SD cube: 322.5 Jy/beam (extend channels 1-256) - Peak in the SD moment 0 image: 994 Jy/beam.km/s - Total Flux Density in the SD moment 0 image: 1690 Jy/beam.km/s - Peak in the BIMA cube: 1.89 Jy/beam (extend channels 1-30) - Total Flux Density in the BIMA cube: 70.3 Jy/beam (extend channels 1-30) - Peak in the BIMA moment 0: 162.7 Jy/beam.km/s - Total Flux Density in the BIMA moment 0: 2061 Jy/beam.km/s - Peak in feathered image: 153.3 Jy/beam.km/s - Total Flux Density in feathered image: 1432 Jy/beam.km/s - Peak in directly converted SD + CLEAN cube: 1.8 Jy/beam - Total Flux Density in directly converted SD + CLEAN cube: 105 Jy/beam - Peak in directly converted SD + CLEAN moment 0: 156.2 Jy/beam.km/s - Total Flux Density in directly converted SD + CLEAN moment 0: 2321 Jy/beam.km/s - Peak in MS-CLEAN deconvolved SD + CLEAN cube: 1.62 Jy/beam - Total Flux Density in in MS-CLEAN deconvolved SD + CLEAN cube: 80.0 Jy/beam - Peak in MS-CLEAN deconvolved SD + CLEAN moment 0: 152.5 Jy/beam.km/s - Total Flux Density in in MS-CLEAN deconvolved SD + CLEAN moment 0: 1978 Jy/beam.km/s Map RMS Peak Total Flux Density -----------------------------------------+--------+-------+------------------- SD cube | 0.61 | 6 | 322.5 Jy/beam SD moment 0 | 50 | 994 | 1690 Jy/beam.km/s BIMA cube (extend channels 1-30) | 0.092 | 1.9 | 70.3 Jy/beam BIMA moment 0 | 8.7 | 162.7 | 2061 Jy/beam.km/s feathered image | 4.38 | 153.3 | 1440 Jy/beam.km/s directly converted SD + CLEAN cube | 0.10 | 1.8 | 106 Jy/beam directly converted SD + CLEAN moment 0 | 13.6 | 157.8 | 2324 Jy/beam.km/s MS-CLEAN deconvolved SD + CLEAN cube | 0.10 | 1.6 | 80 Jy/beam MS-CLEAN deconvolved SD + CLEAN moment 0 | 12.1 | 152.5 | 1978 Jy/beam.km/s =============================================================================== 6. Did you have adequate support during your test? If you contacted the AIPS++ groups for questions or to fix a bug, please comment on the interaction and whether it was helpful. Yes, I had a first reply to my single question within 10 minutes. My question is logged as a bug report. 7. Was AIPS++ easy to install? If not, why? Yes, very easy using the load-casa script. The installation went fully automatic. 8. The Synthesis Reduction Cookbook you used for this test is the second version of a comprehensive cookbook for ALMA users. Please evaluate the organization, content, and presentation of the cookbook. It is meant to be the first documentation users will see when they want to reduce ALMA data, it provides background on the code capabilities, and extensive examples. The on-line documentation provides more details and code descriptions. With this in mind, please answer the questions below. If you have detailed comments, please attach them to the end of this questionnaire. - Was the documentation adequate for you to complete your test? Yes. - Do you have any suggestions for how to improve the cookbook? The examples included in the text (e.g. page 98 to 103) are very well explained. Please keep giving these kinds of exhaustive examples in the future. These are more useful than the large annotated example scripts in section 10. Error on page 146, section 7.9: second line of the script to create a moments map misses a "im." in front of moments. It should be im_mom0:=im.moments(outfile='ngc5921.mom0',...); - Was the on-line documentation helpful: * User Reference Manual? * Supporting documentation? I needed only the cookbook, which contains all the information I needed for the test. 9. Roughly how much time did you take to perform the following steps: - Installing aips++: 1 hour - Imaging: 6 hours - Analysis: 15 hours - Filling out this questionnaire: 2 hours - Evaluating and grading the scientific requirements: 1 hour - Total time: 25 hours 10.Please rate your overall testing experience: - good 11.Was the test well designed and executed by those in the ALMA offline subsystem (e.g. the subsystem scientist and the Offline subsystem group). If not, can you provide any suggestions for improving the next test? My impression is that the single dish data was of too low S/N and spatial resolution wrt the BIMA data. Although it is clearly sufficient to test the procedures, the gain from adding the single dish data is minimal. It may be better to use SD data from a larger telescope with a better matched beam sizes (e.g. MAMBO maps from the IRAM 30m combined with IRAM Plateau de Bure data). For the BIMA and NRAO 12m data, the beams were different by 1 order of magnitude, which leaves a big gap in the uv-plane. 12.Do you have any additional comments that may help improve test of the offline software in the future? See above comment on the datasets used. Figure descriptions: ==================== 1. N4826.12m.BIMA.feather.ps: SD image as raster with BIMA image as green contours and feathered image as black contours. Contour levels are 3.5*(-6,-4.25,-3,3,4.25,6,8.5,12,17,24,34,48) Jy/beam.km/s. Note the the only difference between the feathered and BIMA image is the lower rms noise levels in the area where there is some SD emission. I do not notice any significant difference in the morphologies of these two images. 2. N4826.feather.BIMA.ps: feathered image as raster image and black contours, with the BIMA image overlaid as white contours. Contour levels are 4*(-6,-4.25,-3,3,4.25,6,8.5,12,17,24,34,48) Jy/beam.km/s. Again, no significant difference is seen between the two images, except for a change in flux scale, due to the addition of SD flux. It appears that the SD map does not receive a lot of weight in the feathering procedure, maybe due to the large difference in beamsize, and the relatively low S/N of the SD map compared to the BIMA map. From the morphology viewpoint, the feathered image does not seem to give much new information on the shape of the NW extension (only visible in the low channels). The flux seen in the feathered map seems to trace exactly what has already been seen in the BIMA map. 3. N4826directclean.feather.ps: directly converted SD + combined CLEAN moment 0 map as raster and black contours, with feathered image overlaid as green contours. Contour levels are 4*(-6,-4.25,-3,3,4.25,6,8.5,12,17,24,34,48) Jy/beam.km/s. The jointly deconvolved image clearly contains more diffuse emission than the feathered emission. This can be seen by the black contours (and raster emission) extending further out in all directions, and most clearly in the NW extended emission area. Because the peak flux densities are similar, this clearly suggests the joint deconvolution image picks up more extended emission. 4. N4826MSdeconvolved.feather.ps: MS-CLEAN deconvolved SD + combined CLEAN moment 0 map as raster and black contours, with feathered image overlaid as green contours. Contour levels are 4*(-6,-4.25,-3,3,4.25,6,8.5,12,17,24,34,48) Jy/beam.km/s. Like the directly converted SD technique, the map with a MS-CLEAN deconvolved SD as input model extends further out than the feathered image. Peak flux levels are again similar, so the jointly cleaned image again seems to pick up more extended flux than the feathered image. 5. N4826MSdeconv.directclean.ps: MS-CLEAN deconvolved SD + combined CLEAN moment 0 map as raster and black contours, with directly converted SD + combined CLEAN moment 0 map overlaid as green contours. Contour levels are 5*(-6,-4.25,-3,3,4.25,6,8.5,12,17,24,34,48) Jy/beam.km/s. The general morphologies are quite consistent, but the image using the directly converted SD image as a model seems to pick up slightly more extended emission than the one using a MS-CLEAN deconvolved SD image as a model. The difference can be most clearly seen in the 'bridge' between the 2 bright peaks, and by the outer contours (black contour extends slightly further than green contour). 6. N4826directclean.BIMA.ps: directly converted SD + combined CLEAN cube as raster and black contours, with BIMA image overlaid as green contours. Contour levels are 0.07*(-6,-4.25,-3,3,4.25,6,8.5,12,17,24,34,48) Jy/beam. Like in the moment 0 map, there is little discernible difference between the 2 maps, except for a very minor increase in extended flux when the SD data in included. However, this only marginally shows up in the lower channels (top right), covering the NW region of extended emission. General evaluation of the different methods: ============================================ The peak fluxes in all different moment 0 maps are within 7% from each other. It is a bit strange to find out that the interferometer-only peak flux density is 163 mJy/Beam.km/s, while the feathered and joint deconvolution peak flux densities are lower by ~5%. This is near the uncertainties in the statistical analysis of the images, so maybe not a big worry. The peak flux in the single-dish image is still >6 times higher than in the joint deconvolution images, so a lot of large scale flux still seems to be missing from this viewpoint. However, when looking at the integrated total flux densities, the image directly using the SD as an input model has 13% more flux than the BIMA image. It is surprising that the BIMA image has 22% more total flux than the 12m SD image. Could this be due to low S/N in the SD image? It is not due to a different moment 0 construction, because I also made a moment 0 map from the 12m cube re-gridded to the same geometry as the BIMA image and found an identical result. The feathered image doesn't really bring much of an improvement to the BIMA-only image. Probably the beamsizes are too different. When using joint deconvolution, I see no improvement in using a MS-CLEAN deconvolvolved input model of the single-dish data to the joint deconvolution. Actually, the contrary seems to be true, as the jointly deconvolved image directly using the SD data as an input model seems to recuperate the highest amount of total flux. Given that this procedure is also quite fast in computing time (though not as fast as feathering), it seems to be the best approach for these datasets. However, more testing with a larger large of datasets is clearly needed before we can reach a firm conclusion on which procedure should be used. Bug reports: ============ 1. interactivemask seems to leave a file lock or such, so one cannot use the mask in subsequent operations such as makemodelfromsd: SEVERE ERROR: Table n4826.mask cannot be created because it is in use in another process. -> only exit of aips++ and restart seems to work, if one wants to use the mask. 2. It is impossible to calculate a total flux of an image whose units are not in Jy/Beam. For example, for moment 0 maps, the units are Jy/Beam.km/s. To calculate a total flux in such an image, I had to manually change the units to Jy/Beam using im.sbu('Jy/beam'); This should be fixed in future releases.