Data Reduction Pipeline for Imaging Data †
Overview †
SWIMS Imaging Data Reduction Pipeline, written in Python, processes SWSB*/SWSR* FITS data to produce a final stacked image in a standard manner.
Reduction Flow Chart |
 |
Download †
Core scripts †
File | Contents |
swsred_20220406.tgz | Python core scripts (swsred/*py) and configuration/utility files |
BPM (Bad-Pixel Mask) †
Apr 6, 2022: BPM for S21B/S22A available.
Put files that you need on swsred/calib/bpm/ directory.
Note that BPMs for S22A are just symbolic links to S21B. If you have data on both S21B and S22A, you only need to download either BPMs and create symbolic links to them in swsred/calib/bpm/ directory, like "ln -s bpm_s21b_b1.fits bpm_s22a_b1.fits" to save your disk space.
Imaging Flat †
Apr 6, 2022: Flat for S21B/S22A available.
Put files that you need on swsred/calib/flat/ directory.
Note that Flat data for S22A are just symbolic links to S21B. If you have data on both S21B and S22A, you only need to download either Flats and create symbolic links to them in swsred/calib/flat/ directory, like "ln -s flat_s21b_y_b1.fits flat_s22a_y_b1.fits" to save your disk space.
Mosaic †
Usually, mosaicking is performed (using swarp) based on WCS information.
These files are needed only when your data do not have enough stars for astrometry.
Sample data †
Use these test data (taken in S18B) to check if your swsred works.
- Object: FS12
- Dithering: 4-point
- Filter: J1-band and H1-band
Here is a code to reduce the data. --> sample recipe
The reduced images produced with two iterations (n_iter=2) using swsred version 20210901 and the execution log (swsred.log) should be like these.
Installation †
prerequisites †
SWSRED requires the following python packages and astrOmatic softwares.
Package | Tested version |
Python | 2.7.17 | 3.7.10 |
astropy | 2.0.9 | 4.2 |
numpy | 1.16.6 | 1.19.2 |
scipy | 1.2.1 | 1.6.2 |
matplotlib | 2.2.3 | 3.3.4 |
astroquery | 0.4 | 0.4.1 |
sextractor | 2.19.5 | 2.19.5 |
scamp | 2.7.8 | 2.10.0 |
swarp | 2.38.0 | 2.38.0 |
The astrOmatic softwares can be installed as follows if you have a anaconda environment.
% conda config --append channels conda-forge
% conda install sextractor
% conda install -c conda-forge astromatic-scamp
% conda install -c conda-forge astromatic-swarp
Also, Internet connection is required during reducing data to access Pan-STARRS/2MASS catalogs.
Install SWSRED †
- Download SWSRED files you need.
- Extract all the tar balls on a directory you want.
% cd SWSRED_DIRECTORY
% tar xvf swsred_20210527.tgz
% tar xvf swsred_calib_mosaic_s21a_20210413.tgz
% mkdir -p swsred/calib/bpm swsred/calib/flat
- set PYTHONPATH to SWSRED_DIRECTORY.
% export PYTHONPATH="SWSRED_DIRECTORY:${PYTHONPATH}
Usage †
SWSRED has no documentations yet, but you can use a dirty script 'swsred/reduce_all.py' as a quick tutorial to learn how to use the software.
Be sure to use SWSRED within python interpreter (e.g., in ipython or jupyter, etc.), not via the command line as no command-line interfaces prepared.
Here is a quick introduction to use SWSRED.
- First of all, prepare a list of input FITS files including absolute paths as a python 'array-like' (list, tuple, or numpy array) object, for example as follows.
> import glob
> in_arr1 = glob.glob("DATA/PATH/SWSB*1.fits") # blue chip1 data
> in_arr1.sort() # must be sorted before using.
- Make filenames of intermediate files.
> import swsred.make_file_lst
> file_lst1 = swsred.make_file_lst.make_file_lst(in_arr1)
- 'file_lst1' is a dictionary which defines various filenames such as flat-fielded FITS filename. You can edit any values as you want.
- For example, you get the following result with an input ["data/path/SWSB00000001.fits"] as in_arr1,
Key | Value | Description |
in | data/path/SWSB00000001.fits | input filename |
ff | ffSWSB00000001.fits | filename of flat-fielded data |
ms | msSWSB00000001.fits | filename of self-sky data |
ss | ssSWSB00000001.fits | filename of sky-subtracted data |
wc | wcSWSB00000001.fits | filename of WCS-corrected data |
bpm | /SWSRED_DIRECTORY/calib/bpm/bpm_sXXY_b1.fits | filename of Bad-pixel mask (XXY denotes a semester) |
wht | SWSB00000001_wht.fits | filename of weight map (created by make_weight_map module) |
omsk | SWSB00000001_omsk.fits | filename of object mask |
ldac | ssSWSB00000001.ldac | filename of SExtractor LDAC catalog |
ascii | ssSWSB00000001.ascii | filename of SExtractor ASCII catalog |
head | ssSWSB00000001.head | filename of Scamp HEAD file |
- Then, start your data reduction. For example, to perform flat-fielding,
> import swsred.flat_field
> swsred.flat_field.flat_field(file_lst1["in"], file_lst1["ff"])
- By default, the values 'file_lst1["ff"]' do not have a path which means that 'file_lst1["ff"]' files will be created on the current directory. If you want to specify any directory to put intermediate files, edit and add your path in 'file_lst1'.
Refer to swsred/reduce_all.py for all the procedures.
Questions & Answers †
- Astrometry (performed by scamp) is failed or is not good. How do I work for it?
- Try changing the following parameters,
- detect_thresh_wcs: detection threshold to make a source catalog used in scamp.
- mag_max_wcs: maximum magnitude of Pan-STARRS sources to be used in scamp.
- You may want to set "use_scamp=False" not to use Scamp; in that case (still in beta), the swsred try to find stars in the image around reference catalog positions and calculate WCS parameters.
- How is the semester determined?
- FRAMEID is used to determine when the data has been taken. You can find the definition of which data was taken in which semester in swsred/config/config.ini.
- If swsred would assign the wrong semester, check if you are using the latest swsred (i.e., latest definition).
Known bugs †
Changes †
Version 2021.01.31 †
Contact †
Please send your bug reports or questions to konishi__at__ioa.s.u-tokyo.ac.jp.