public documents.sextractor_doc

[/] [weight.tex] - Diff between revs 19 and 25

Rev 19 Rev 25
Line 1... Line 1...
\chapter{Weighting}
\chapter{Weighting}
\label{chap:weight}
\label{chap:weight}
The noise level in astronomical images is often fairly constant, that
The noise level in astronomical \index{image} images is often fairly constant, that
is, constant values for the gain, the background noise and the
is, constant values for the \index{gain} gain, the background noise and the
detection thresholds can be used over the whole frame. Unfortunately,
detection \index{threshold} thresholds can be used over the whole frame. Unfortunately,
in some cases, like strongly vignetted or composited images, this
in some cases, like strongly vignetted or composited \index{image} images, this
approximation is no longer good enough. This leads to detecting
approximation is no longer good enough. This leads to detecting
clusters of detected noise peaks in the noisiest parts of the image,
clusters of detected noise peaks in the noisiest parts of the \index{image} image,
or missing obvious objects in the most sensitive ones. {\sc
or missing obvious objects in the most sensitive ones. {\sc
SExtractor} is able to handle images with variable noise. It does so
SExtractor} is able to handle \index{image} images with variable noise. It does so
through {\em weight maps}, which are frames having the same size as
through {\em \index{weight map} \index{weight maps} weight maps}, which are frames having the same size as
the images where objects are detected or measured, and which describe
the \index{image} images where objects are detected or measured, and which describe
the noise intensity at each pixel. These maps are internally stored in
the noise intensity at each pixel. These maps are internally stored in
units of {\em absolute variance} (in ADU$^2$). We employ the generic
units of {\em absolute variance} (in ADU$^2$). We employ the generic
term weight map'' because these maps can also be interpreted as
term weight map'' because these maps can also be interpreted as
quality index maps: infinite variance ($\ge 10^{30}$ by definition in
quality index maps: infinite variance ($\ge 10^{30}$ by definition in
{\sc SExtractor}) means that the related pixel in the science frame is
{\sc SExtractor}) \index{mean} means that the related pixel in the science frame is
totally unreliable and should be ignored. The variance format was
totally unreliable and should be ignored. The variance format was
adopted as it linearizes most of the operations done over weight maps
adopted as it linearizes most of the operations done over \index{weight map} \index{weight maps} weight maps
(see below).
(see below).
 
 
This means that the noise covariances between pixels are ignored.
This \index{mean} means that the noise \index{covariance} \index{covariances} covariances between pixels are ignored.
Although raw CCD images have essentially white noise, this is not the
Although raw \index{CCD} CCD \index{image} images have essentially white noise, this is not the
case for warped images, for which resampling may induce a strong
case for warped \index{image} images, for which resampling may induce a strong
correlation between neighbouring pixels. In theory, all non-zero
correlation between \index{neighbour} neighbouring pixels. In theory, all non-zero
covariances within the geometrical limits of the analysed patterns
\index{covariance} \index{covariances} covariances within the geometrical limits of the analysed patterns
should be taken into account to derive thresholds or error estimates.
should be taken into account to derive \index{threshold} thresholds or error estimates.
Fortunately, the correlation length of the noise is often smaller than
Fortunately, the correlation length of the noise is often smaller than
the patterns to be detected or measured, and constant over the image.
the patterns to be detected or measured, and constant over the \index{image} image.
In that case one can apply a simple fudge factor'' to the estimated
In that case one can apply a simple fudge factor'' to the estimated
variance to account for correlations on small scales. This proves to
variance to account for correlations on small scales. This proves to
be a good approximation in general, although it certainly leads to
be a good approximation in general, although it certainly leads to
underestimations for the smallest patterns.
underestimations for the smallest patterns.
 
 
\section{Weight map formats}
\section{Weight map formats}
\label{chap:weighttype}
\label{chap:weighttype}
{\sc SExtractor} accepts in input, and converts to its internal
{\sc SExtractor} accepts in input, and converts to its internal
variance format, several types of weight maps. This is controlled
variance format, several types of \index{weight map} \index{weight maps} weight maps. This is controlled
through the {\tt WEIGHT\_TYPE} configuration keyword. These
through the {\tt WEIGHT\_TYPE} configuration keyword. These
weight maps can either be read from a FITS file, whose name is
\index{weight map} \index{weight maps} weight maps can either be read from a FITS file, whose name is
specified by the {\tt WEIGHT\_IMAGE} keyword, or computed internally.
specified by the {\tt WEIGHT\_IMAGE} keyword, or computed internally.
Valid {\tt WEIGHT\_TYPE}s are:
Valid {\tt WEIGHT\_TYPE}s are:
\begin{itemize}
\begin{itemize}
\item{\tt NONE}: No weighting is applied. The related {\tt
\item{\tt NONE}: No weighting is applied. The related {\tt
WEIGHT\_IMAGE} and {\tt WEIGHT\_THRESH} (see below) parameters are
WEIGHT\_IMAGE} and {\tt WEIGHT\_THRESH} (see below) parameters are
ignored.
ignored.
\item {\tt BACKGROUND}: the science image itself is used to compute
\item {\tt BACKGROUND}: the science \index{image} image itself is used to compute
internally a variance map (the related {\tt WEIGHT\_IMAGE} parameter
internally a \index{variance map} variance map (the related {\tt WEIGHT\_IMAGE} parameter
is ignored). Robust ($3\sigma$-clipped) variance estimates are first
is ignored). Robust ($3\sigma$-clipped) variance estimates are first
computed within the same background meshes as those described in
computed within the same background meshes as those described in
\S\ref{chap:backest}\footnote{The mesh-filtering procedures act on the
\S\ref{chap:backest}\footnote{The mesh-filtering procedures act on the
variance map, too.}. The resulting low-resolution variance map is then
\index{variance map} variance map, too.}. The resulting low-resolution \index{variance map} variance map is then
bicubic-spline-interpolated on the fly to produce the actual full-size
bicubic-spline-interpolated on the fly to produce the actual full-size
variance map. A check-image with {\tt CHECKIMAGE\_TYPE} {\tt
\index{variance map} variance map. A \index{check-image} check-image with {\tt CHECKIMAGE\_TYPE} {\tt
MINIBACK\_RMS} can be requested to examine the low-resolution variance
MINIBACK\_RMS} can be requested to examine the low-resolution variance
map.
map.
\item {\tt MAP\_RMS}: the FITS image specified by the {\tt
\item {\tt MAP\_RMS}: the FITS \index{image} image specified by the {\tt
WEIGHT\_IMAGE} file name must contain a weight map in units of
WEIGHT\_IMAGE} file name must contain a \index{weight map} weight map in units of
absolute standard deviations (in ADUs per pixel).
absolute \index{standard deviation} standard deviations (in ADUs per pixel).
\item {\tt MAP\_VAR}: the FITS image specified by the {\tt
\item {\tt MAP\_VAR}: the FITS \index{image} image specified by the {\tt
WEIGHT\_IMAGE} file name must contain a weight map in units of
WEIGHT\_IMAGE} file name must contain a \index{weight map} weight map in units of
relative variance. A robust scaling to the appropriate absolute level
relative variance. A robust scaling to the appropriate absolute level
is then performed by comparing this variance map to an internal,
is then performed by comparing this \index{variance map} variance map to an internal,
low-resolution, absolute variance map built from the science image
low-resolution, absolute \index{variance map} variance map built from the science \index{image} image
itself.
itself.
\item {\tt MAP\_WEIGHT}: the FITS image specified by the {\tt
\item {\tt MAP\_WEIGHT}: the FITS \index{image} image specified by the {\tt
WEIGHT\_IMAGE} file name must contain a weight map in units of
WEIGHT\_IMAGE} file name must contain a \index{weight map} weight map in units of
relative weights. The data are converted to variance units (by
relative weights. The data are converted to variance units (by
definition ${\rm variance} \propto 1/{\rm weight}$), and scaled as for
definition ${\rm variance} \propto 1/{\rm weight}$), and scaled as for
{\tt MAP\_VAR}. {\tt MAP\_WEIGHT} is the most commonly used type of
{\tt MAP\_VAR}. {\tt MAP\_WEIGHT} is the most commonly used type of
weight map: a flat-field, for example, is generally a good
\index{weight map} weight map: a \index{flat-field} flat-field, for example, is generally a good
approximation to a perfect weight map.
approximation to a perfect \index{weight map} weight map.
\end{itemize}
\end{itemize}
 
 
\section{Weight threshold}
\section{Weight \index{threshold} threshold}
It may happen, that some weights are too low (or variances too high) to be of any interest: it is then more appropriate
It may happen, that some weights are too low (or variances too high) to be of any interest: it is then more appropriate
to discard such pixels than to include them in unweighted measurements such as {\tt FLUX\_APER}. To allow discarding these
to discard such pixels than to include them in unweighted measurements such as {\tt FLUX\_APER}. To allow discarding these
very bad pixels, a threshold can be set with the {\tt WEIGHT\_THRESH} parameter. The unit in which this threshold
very \index{bad pixel} \index{bad pixels} bad pixels, a \index{threshold} threshold can be set with the {\tt WEIGHT\_THRESH} parameter. The unit in which this threshold
should be expressed is that of input data: ADUs for
should be expressed is that of input data: ADUs for
BACKGROUND and MAP\_RMS maps, uncalibrated ADUs$^2$ for MAP\_VAR, and uncalibrated weight-values for MAP\_WEIGHT maps.
BACKGROUND and MAP\_RMS maps, uncalibrated ADUs$^2$ for MAP\_VAR, and uncalibrated weight-values for MAP\_WEIGHT maps.
Depending on the weight map type, the threshold will set a lower or a higher limit for bad pixel'' values: higher for
Depending on the \index{weight map} weight map type, the \index{threshold} threshold will set a lower or a higher limit for bad pixel'' values: higher for
weights, and lower for variances and standard deviations. The default value is 0 for weights, and $10^{30}$ for variance and
weights, and lower for variances and \index{standard deviation} standard deviations. The default value is 0 for weights, and $10^{30}$ for variance and
standard deviation maps.
\index{standard deviation} standard deviation maps.
 
 
\section{Effects of weighting}
\section{Effects of weighting}
Weight maps modify the working of {\sc SExtractor} in the following respects:
Weight maps modify the working of {\sc SExtractor} in the following respects:
\begin{enumerate}
\begin{enumerate}
\item Bad pixels are discarded from the background statistics. If more than 50\% of the pixels in a background mesh are bad, the local background
\item Bad pixels are discarded from the background statistics. If more than 50\% of the pixels in a background mesh are bad, the \index{local background} local background
value and its standard deviation are replaced by interpolation of the
value and its \index{standard deviation} standard deviation are replaced by \index{interpolation} interpolation of the
nearest valid meshes.
nearest valid meshes.
\item The detection threshold $t$ above the local sky background is
\item The detection \index{threshold} threshold $t$ above the local sky background is
adjusted for each pixel $i$ with variance $\sigma^2_i$: $t_i = {\tt adjusted for each pixel$i$with variance$\sigma^2_i$:$t_i = {\tt
DETECT\_THRESH}\times\sqrt{\sigma^2_i}$, where {\tt DETECT\_THRESH} is DETECT\_THRESH}\times\sqrt{\sigma^2_i}$, where {\tt DETECT\_THRESH} is
expressed in units of standard deviations of the background noise.
expressed in units of \index{standard deviation} standard deviations of the background noise.
Pixels with variance above the threshold set with the {\tt
Pixels with variance above the \index{threshold} threshold set with the {\tt
WEIGHT\_THRESH} parameter are therefore simply not detected. This may
WEIGHT\_THRESH} parameter are therefore simply not detected. This may
result in splitting objects crossed by a group of bad pixels.
result in splitting objects crossed by a group of \index{bad pixel} \index{bad pixels} bad pixels.
Interpolation (see \S\ref{chap:interp}) should be used to avoid this
Interpolation (see \S\ref{chap:interp}) should be used to avoid this
problem. If convolution filtering (\S\ref{filter-conv}) is applied for detection, the
problem. If \index{convolution} convolution filtering (\S\ref{filter-conv}) is applied for detection, the
variance map is convolved too. This yields optimum scaling of the
\index{variance map} variance map is convolved too. This yields optimum scaling of the
detection threshold in the case where noise is uncorrelated from pixel
detection \index{threshold} threshold in the case where noise is uncorrelated from pixel
to pixel. Non-linear filtering operations (like those offered by
to pixel. Non-linear filtering operations (like those offered by
artificial retinae, \S\ref{filter-non-linear}) are not affected.
artificial retinae, \S\ref{filter-non-linear}) are not affected.
\item The {\tt CLEAN}ing process (\S\ref{chap:clean}) takes into
\item The {\tt CLEAN}ing process (\S\ref{chap:clean}) takes into
account the exact individual thresholds assigned to each pixel for
account the exact individual \index{threshold} thresholds assigned to each pixel for
deciding about the fate of faint detections.
deciding about the fate of faint detections.
\item Error estimates like {\tt FLUXISO\_ERR}, {\tt ERRA\_IMAGE}, ...
\item Error estimates like {\tt FLUXISO\_ERR}, {\tt ERRA\_IMAGE}, ...
make use of individual variances too. The local background noise standard
make use of individual variances too. The \index{local background} local background noise standard
deviation is simply set to $\sqrt{\sigma^2_i}$. In addition, if the
deviation is simply set to $\sqrt{\sigma^2_i}$. In addition, if the
{\tt WEIGHT\_GAIN} parameter is set to {\tt Y} --- which is the
{\tt WEIGHT\_GAIN} parameter is set to {\tt Y} --- which is the
default ---, it is assumed that the local pixel gain (i.e., the
default ---, it is assumed that the local pixel \index{gain} gain (i.e., the
conversion factor from photo-electrons to ADUs) is inversely
conversion factor from photo-electrons to ADUs) is inversely
proportional to $\sigma^2_i$, its median value over the image being
proportional to $\sigma^2_i$, its \index{median} median value over the \index{image} image being
set by the {\tt GAIN} configuration parameter. In other words,
set by the {\tt GAIN} configuration parameter. In other words,
the changes in noise intensities seen over the
the changes in noise intensities seen over the
images are assumed to be caused by spatial variations in the gain. This is the most common case:
\index{image} images are assumed to be caused by spatial variations in the \index{gain} gain. This is the most common case:
correction for vignetting, or coverage depth. When this is not the
correction for \index{vignetting} vignetting, or coverage depth. When this is not the
case, for instance when changes are purely dominated by those of the
case, for instance when changes are purely dominated by those of the
read-out noise, {\tt WEIGHT\_GAIN} shall be set to {\tt N}.
read-out noise, {\tt WEIGHT\_GAIN} shall be set to {\tt N}.
\item Finally, pixels with weights beyond {\tt WEIGHT\_THRESH} are
\item Finally, pixels with weights beyond {\tt WEIGHT\_THRESH} are
treated just like pixels discarded by the {\tt MASK}ing process
treated just like pixels discarded by the {\tt MASK}ing process
(\S\ref{chap:mask}). \end{enumerate}
(\S\ref{chap:mask}). \end{enumerate}
 
 
\section{Combining weight maps}
\section{Combining \index{weight map} \index{weight maps} weight maps}
All the weighting options listed in \S\ref{chap:weighttype} can be
All the weighting options listed in \S\ref{chap:weighttype} can be
applied separately to detection and measurement images
applied separately to detection and measurement \index{image} images
(\S\ref{chap:using}), --- even if some combinations may not always
(\S\ref{chap:using}), --- even if some combinations may not always
make sense. For instance, the following set of configuration lines:
make sense. For instance, the following set of configuration lines:
\begin{verbatim}
\begin{verbatim}
WEIGHT_IMAGE    rms.fits,weight.fits
WEIGHT_IMAGE    rms.fits,weight.fits
WEIGHT_TYPE     MAP_RMS,MAP_WEIGHT
WEIGHT_TYPE     MAP_RMS,MAP_WEIGHT
\end{verbatim}
\end{verbatim}
will load the FITS file {\tt rms.fits} and use it as an RMS map for
will load the FITS file {\tt rms.fits} and use it as an RMS map for
adjusting the detection threshold and CLEANing, while the {\tt
adjusting the detection \index{threshold} threshold and CLEANing, while the {\tt
weight.fits} weight map will only be used for scaling the error
weight.fits} \index{weight map} weight map will only be used for scaling the error
estimates on measurements. This can be done in single- as well as in
estimates on measurements. This can be done in single- as well as in
dual-image mode (\S\ref{chap:using}). {\tt WEIGHT\_IMAGE}s can be
dual-image \index{mode} mode (\S\ref{chap:using}). {\tt WEIGHT\_IMAGE}s can be
ignored for {\tt BACKGROUND} {\tt WEIGHT\_TYPE}s. It is of course
ignored for {\tt BACKGROUND} {\tt WEIGHT\_TYPE}s. It is of course
possible to use weight maps for detection or for measurement only. The
possible to use \index{weight map} \index{weight maps} weight maps for detection or for measurement only. The
following configuration:
following configuration:
\begin{verbatim}
\begin{verbatim}
WEIGHT_IMAGE    weight.fits
WEIGHT_IMAGE    weight.fits
WEIGHT_TYPE     NONE,MAP_WEIGHT
WEIGHT_TYPE     NONE,MAP_WEIGHT
\end{verbatim}
\end{verbatim}