After opening that website, in ‘’Target Name’’ box, fill in XXXX then click “Submit Query”, then all the publicly available data related to that source can be downloaded following the instructions on the website.
The data may in “.uvfits” or “.idifits” formats.
Step2 - Converting “.idifits” format data to CASA Measurement Sets
Before handling in CASA, the original “.idifits” format data needs to be converted to the CASA Measurement Sets format. Using the importfitsidi task of CASA:
In CASA, importfitsidi tool is used to import a idifits file
One of the most important aspects of data reduction is to understand your data!. Without knowing what your data looks like and its quality, the resultant calibration can often be sub-optimal. CASA provides lots of tools to assess and understand your data. The first, and one of the most essential of these is provided by the listobs task. This task outlines the structure of your data such as the frequencies observed, the antennas involved and the observing plan.
default(listobs)
vis='XXX.ms'
listfile='XXX.ms.listobs' # This makes a file with the listobs info.
listobs()
The command makes a file in the current working directory called XXX.ms.listobs which contains information about the measurement set. Try opening this file with your favourite text editor such as gedit. Remember a ! lets you run UNIX commands from CASA. So, if you use cat, the command to write in the command line would be !cat XXX.ms.listobs.
• Spectral Windows: 2, 1504 MHz, 1632 MHz, each 256 channels, chanel width 0.5 MHz, total BW 128 MHz. 4 polarisation products (RR RL LR LL)
• Antennas: 9. The offset from the array centre is 0 ß absolute Earth-centred (geocentric) coordinates are given for each antenna.
3.2 Correct antenna tables
3.3 A Priori Calibration
The append_tsys.py(source github) helper script was used to attach the system temperature information for each telescope to the FITS-IDI files. The system temperature is measured every few minutes. This compensates roughly for the different levels of signal from different sources, the effects of elevation and other amplitude fluctuations. Each antenna has a characteristic response in terms of Kelvin of system temperature per Jy of flux, so Tsys is also used to scale the flux density.
In CASA, calibration is conducted using calibration tables which are then used to modify the visibilities. To obtain our fluxscaling we therefore have to generate a system temperature calibration table using the task gencal. This task is used to generate caltables from ancillary information which can be found in the measurement set or in external files, and can also be used to create manual calibration tables from scratch. The gencal task has a special function to generate Tsys tables. To generate these tables we need to input:
# In CASA
default(gencal) # This reads the Tsys information appended to a ms
vis='XXX.ms'
caltable='XXX.tsys'
caltype='tsys'
uniform=False #Set to False to get spw dependent Tsys extraction
gencal()
You will find that there is a new table named XXX.tsys which contains the gain information.
Important: A key factor whenever generating calibration tables is to inspect them to see if there are errant values which simply do not make sense. To inspect tables, we can use either plotcal or, since CASA v5.4, plotms. For this example, we shall use plotcal. If we initialise the task (remember use default), we can see that we have to set the xx and yy axes along with what each plot shall be iterated over. So in this case we want to plot Tsys against frequency for each antenna. Note that the subplot parameter is the matplotlib subplot syntax. We should therefore input the following:
default(plotms)
vis='XXX.tsys'
xaxis='freq'
yaxis='tsys'
gridrows=3
grodcols=3
iteraxis='antenna'
go
We can also see how the Tsys changes over time! Most parameters are already set from the last run of plotms, therefore we can just write (and remembering to check the inputs before running!):
In the next step we will generate the gain curve table. This provides the scaled gain-elevation correction which scales the visibility amplitudes to the sensitivity of the dish, albeit without any allowance for weather or source contributions. Again, we use gencal to do this and the gain information (XXX.gc) that we extracted from the .antab file:
# In CASA
default(gencal)
vis="XXX.ms"
caltable= "XXX.gcal" ## Name of output gain curve cal. table
caltype="gc" ## Special caltype for gaincurves
infile="XXX.gc" ## Gaincurve information derived from antab file
gencal()
One of the most important aspects of radio interferometric data reduction is identifying and removing bad data (This way AOFlagger is so cool). Often these data is due to Radio Frequency Interference (RFI) from satellites and mobile phones 3G，4G，5G的频率和频段是多少? - 3G 约 2 GHz，4G 约 2.5 GHz，5G约 3.5 GHz 和 5 GHz
Sometimes it can be many other factors such as correlation errors or antennas off source. While it sounds tedious, the effects of bad data can be significant if ignored therefore careful extraction and removal of bad data is of paramount importance. In the next, we shall inspect our visibilities and attempt to flag and remove bad data.
To begin, we shall remove the autocorrelations as these are not useful for continuum interferometric observations. To do this we use the manual flagging task called flagdata. Take a look at the inputs. To flag all of the autocorrelations we simply leave all as default (i.e. all data is selected) and set the autocorr parameter to be True
To inspect the quality of the data, we are going to use the task plotms to look for bad data. We start by looking at a bright source where it is easiest to tell good and bad data apart. The plotms task is very complicated (see its inputs) and takes time to understant. We look at the frequency axes first (so we can average the time axes) and look at the baselines to the most sensitive telescope (e.g. the 100m Effelsberg dish if its on the baseline network).
default(plotms)vis ="F.ms"xaxis ="frequency"yaxis ="amp"field='J0132+4325'# The phase-cal/bandpass cal# avgtime is depend on the actual durationavgtime='3600'# Will only average within scans unless additionally told to average scans tooantenna='MK&*'# Plot all baselines to the largest and most sensitive antenna# maybe not MK, I use PT in the GUIcorrelation='RR,LL'# Only plot the parallel hands; the cross hands are fainter (and we won't be using them).coloraxis='antenna2'go
Then use icons on the GUI to check what antennas should be flagged (those who has 0 amplitude with the most sensitive antenna).
amp_freq
Important: Before more flagging, remember to back up the current flags. Do this any time you think you might need to change your mind about flagging. Use a different version name each time and make a note of it (CASA task flagdata can make an automatic backup but it will not have a memorable name). The backups for this data set are stored in XXX.ms.flagversions. We will now back up the flags for this data set now using the task flagmanager. The same task can be used to revert back to old flags too!:
default(flagmanager)vis ="F.ms"mode='save'# This mode saves the current flags, use the mode restore to revert to old flagging versionsversionname='test'# This is the version name, you can see all version names in flagversions by using mode='list'go
If we find/identified that the XX telescope is not working properly, we can flag the telescopes using the flagdata task that we used earlier.
Now we should re-inspect the data for any more corrupted data. Plot the most sensitive baseline to see the end channels more clearly and note which are below about half the maximum amplitude. These cause issues further down the line if not flagged and do not contribute much to the final image sensitivity.
# In CASAdefault(plotms)vis='n14c3.ms'xaxis='channel'yaxis='amp'field='1848+283'# The phase-cal/bandpass calavgtime='3600'# Will only average within scans unless additionally told to average scans tooantenna='EF&JB'# Plot the baseline of largest and most sensitive antennascorrelation='RR,LL'# Only plot the parallel hands; the cross hands are fainter (and we won't be using them).iteraxis='spw'plotms()
Use the arrows at the bottom of plotms to scroll through the spw (which should got through the subplots above). Odd and even spw behave differently but there is no need to spend too much time on this as the suggested channels to flag are given here for time saving purposes:
# In CASAdefault(flagdata)vis='n14c3.ms'mode='manual'spw='0:0~5;29~31,2:0~5;29~31,4:0~5;29~31,6:0~5;29~31,1:0~2;27~31,3:0~2;27~31,5:0~2;27~31,7:0~2;27~31'flagdata()
Next we shall look at amp vs. time instead, now averaging in frequency:
# In CASAdefault(plotms)vis='F.ms'xaxis='time'yaxis='amp'field='B0128+437'spw='0~1'# Average a few central channels where the response is stableavgchannel='8'antenna='PT&MK'correlation='RR,LL'coloraxis='baseline'plotms()
Note that the first one or two integrations of some scans are bad. There is a specific flagdata mode for this called quack. We will safely assume that all sources are affected.
default(flagdata)vis='n14c3.ms'mode='quack'quackinterval =5# 5 seconds, or just over 2 intgrationsgo
Now go through each baseline to identify remaining bad data (enter iteration='baseline' and re-load plotms). Again, this has already been listed so don’t spend too long, just make sure you have some idea of how to identify and record bad data. There is more than one bad scan on HH and as we are plotting the phase-cal it is likely that the target in-between was affected. Plot this.
default(plotms)vis='n14c3.ms'xaxis='time'yaxis='amp'scan='60~70'# a few scans including the bad data; plot all fields in this time rangespw='0~7:13~20'avgchannel='8'antenna='EF&HH'# just the relevant baselinecorrelation='RR,LL'coloraxis='field'# distinguish sourcesgo
Flag the scans when the target (in brown in the plot here) goes right to zero. Or for time-saving purposes we have put the commands in here for you.
Next, lets inspect the next most distant antenna, SH
default(plotms)vis='n14c3.ms'xaxis='time'yaxis='amp'field='1848+283'spw='0~7:13~20'avgchannel='8'antenna='EF&SH'# just the relevant baselinecorrelation='RR,LL'coloraxis='spw'# distinguish spwgo
There are many short, bad periods, some only affecting certain spw. The easiest way to cope with this is to write a flag list. Identify antennas/spw/times, adding 1s to the start and finish of each flagging period. They are all shorter than one scan so the same flags aren’t applied to the target but later, we will look for similar bad data on the target. Take a look at the flag list file: flagSH.flagcmd which is in your current directory and cross-check that these entered values are affected.
If you are happy, then we will use this file to automatically flag those channels specified in the file:
Hooray, the data looks clean! We can finally begin to calibrate these data!
3.5 Fringe fitting
Now that all the data has been cleaned up, we can begin to think about calibrating these data.
As you may remember from the accompanying lectures, the wavefronts arriving at two different telescopes from a single source arrives at different times, i.e. one signal is delayed relative to each other. This is shown by the Dsin (θ) term in the two-element interferometer shown below:
img
3.6 Bandpass calibration
3.7 Split out phase-ref-target pairs
Install casacore, aoflagger and wsclean on Debian
sudo apt update
sudo apt search casacore
sudo apt install casacore-tools casacore-dev libcasa-casa3 python3-casacore casacore-data casacore-data-sources
sudo apt install aoflagger #(should be version 2.14.0, 2019.02.14)
# to reinstall, first sudo apt remove XXX
# and sudo apt autoremove
# then run sudo apt install XXX again
sudo apt install wsclean #(should be version 2.8.0, 2019.09.16)
apt list -a <package name>
apt-cache madison aoflagger
If you can not install the latest version, you can find a specific one on https://packages.debian.org/stable/allpackages
or
https://packages.debian.org/testing/allpackages
or, if you can install a version you want in ubuntu, you can:
Google: ubuntu create deb from installed package
**Or, you should directly download package source file in the website: https://sourceforge.net/p/aoflagger/wiki/Home/**
Because of MacOS Catalina has some issues when install casacore, so aoflagger cannot be installed on MacOS. To use gui tools on linux server, we can add -X option when login into the remote server:
ssh -p2222 -X username@XXX
After logining in, a small logo will show on Docker:
and after you run the rfigui, the window will be opened on Mac:
# python3 --version Python 3.7.3 --> must use Python 3.6
python3 -m venv casa6
source casa6/bin/activate
pip install --index-url https://casa-pip.nrao.edu/repository/pypi-casa-release/simple casatools
pip install --index-url https://casa-pip.nrao.edu/repository/pypi-casa-release/simple casatasks
# Above does not works unless python3.6 is installed, I choose not use this way to install, since not only the python version matters, pip tool also needs to mathch, not convenient.
# I choose download the tar.xz file directly and then unpack the software.
Also, The Ubuntu installation instructions work well for Debian as well. However, if “xvfb” is not installed on your machine, CASA may start with errors like this:
casalogger: cannot connect to X server localhost:10.0
If so, try this:
sudo apt-get install xvfb
Then restart CASA and the problem should be solved.
Besides, do not close the popped window by click by hand, use the quit Manu of the software (or Ctrl+C on linux shell directly), otherwise the XQuartz will not working anymore, you need to relogin.
