Skip to content

RedshiftCandidates datamodel update #124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

RedshiftCandidates datamodel update #124

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a5a6b3b
Add warning flags and error codes to redshiftCandidates DM
AliAllaouiLAM Oct 21, 2022
a44c92f
Add error messages
AliAllaouiLAM Apr 10, 2024
da4adb9
[redshiftCandidates] Add init warning and error
AliAllaouiLAM Jun 19, 2024
8b0cdf7
Fix PDF columns definitions
AliAllaouiLAM Jan 7, 2025
b39a306
[redshiftCandidates] Add warning flag description
AliAllaouiLAM Feb 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 138 additions & 17 deletions datamodel.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1074,26 +1074,47 @@ The path would be
The file will have several HDUs:

HDU#0 PHDU
tract INT Tract identifier
patch KEYWORD Patch identifier
catId INT Catalog identifier
objId INT Object identifier
nvisit INT Number of visit
vHash KEYWORD 63-bit SHA-1 list of visits
CRPIX1 FLOAT Pixel coordinate of reference point
CRVAL1 FLOAT [m] Coordinate value at reference point
CDELT1 FLOAT [m] Coordinate increment at reference point
D1D_VER KEYWORD Version of the DRP_1D library
D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline
DAMD_VER KEYWORD Version of the data model
U_PARAM KEYWORD User parameters contents, json
ZWARNING INT64 Quality flag
tract INT Tract identifier
patch KEYWORD Patch identifier
catId INT Catalog identifier
objId INT Object identifier
nvisit INT Number of visit
vHash KEYWORD 63-bit SHA-1 list of visits
CRPIX1 FLOAT Pixel coordinate of reference point
CRVAL1 FLOAT [m] Coordinate value at reference point
CDELT1 FLOAT [m] Coordinate increment at reference point
D1D_VER KEYWORD Version of the DRP_1D library
D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline
DAMD_VER KEYWORD Version of the data model
U_PARAM KEYWORD User parameters contents, json
INIT_WARNING INT64 Quality flags for spectrum initialization
GALAXY_ZWARNING INT64 Quality flags for galaxy solver
QSO_ZWARNING INT64 Quality flags for QSO solver
STAR_ZWARNING INT64 Quality flags for star solver
GALAXY_LWARNING INT64 Quality flags for galaxy line measurement solver
QSO_LWARNING INT64 Quality flags for QSO line measurement solver
CLASSIFICATION_WARNING INT64 Quality flags for classification solver
GALAXY_ZERROR INT64 Error code for galaxy solver
QSO_ZERROR INT64 Error code for QSO solver
STAR_ZERROR INT64 Error code for star solver
GALAXY_LERROR INT64 Error code for galaxy line measurement solver
QSO_LERROR INT64 Error code for QSO line measurement solver
INIT_ERROR INT46 Error code for spectrum initialization
G_ZERR KEYWORD Error message for galaxy solver
Q_ZERR KEYWORD Error message for QSO solver
S_ZERR KEYWORD Error message for star solver
G_LERR KEYWORD Error message for galaxy line measurement solver
Q_LERR KEYWORD Error message for QSO line measurement solver
I_ERR KEYWORD Error message for spectrum initialization
CLASSIFICATION_ERROR INT64 Error code for classification solver


HDU#1 CLASSIFICATION
CLASS KEYWORD Spectro classification: GALAXY, QSO, STAR
P_GALAXY KEYWORD Probability to be a galaxy
P_QSO KEYWORD Probability to be a QSO
P_STAR KEYWORD Probability to be a star


HDU#2 GALAXY_CANDIDATES Binary table [FITS BINARY TABLE] NGALAXY_CANDIDATES
Colums for :
Expand All @@ -1109,7 +1130,7 @@ HDU#2 GALAXY_CANDIDATES Binary table [FITS BINARY TABLE]
HDU#3 GALAXY_PDF Binary table [FITS BINARY TABLE] NPIX
Colums for :
REDSHIFT FLOAT[NPIX] Redshift grid
PDF FLOAT[NPIX] PDF marginalised over all models
ln PDF FLOAT[NPIX] ln(Probability) marginalised over all models

HDU#4 GALAXY_LINES Binary table [FITS BINARY TABLE] NLINE
Columns for :
Expand Down Expand Up @@ -1140,7 +1161,7 @@ HDU#5 QSO_CANDIDATES Binary table [FITS BINARY TABLE]
HDU#6 QSO_PDF Binary table [FITS BINARY TABLE] NPIX
Colums for :
REDSHIFT FLOAT[NPIX] Redshift grid
PDF FLOAT[NPIX] PDF marginalised over all models
ln PDF FLOAT[NPIX] ln(Probability) marginalised over all models

HDU#7 QSO_LINES Binary table [FITS BINARY TABLE] NLINE
Columns for :
Expand Down Expand Up @@ -1172,7 +1193,107 @@ HDU#8 STAR_CANDIDATES Binary table [FITS BINARY TABLE]
HDU#9 STAR_PDF Binary table [FITS BINARY TABLE] NPIX
Colums for :
VELOCITY FLOAT[NPIX] Radial velocity grid
PDF FLOAT[NPIX] PDF related to best model
ln PDF FLOAT[NPIX] ln(Probability) related to best model

Warning flags :

**AIR_VACUUM_CONVERSION_IGNORED**
An air to vacuum conversion has been defined in the parameters.
However, it is specified in the input spectrum that the data are in vacuum.
The air to vacuum conversion is therefore ignored.

**PDF_PEAK_NOT_FOUND**
For a given redshift candidate, in the second pass window, no pdf peak has
been found.

**ESTIMATED_STD_FAR_FROM_INPUT**
Residual std (obtained from rms) (spectrum - model) very different
(ratio > 1.5 & 1/ 15 <-> 50 + or 50%-) from input spectrum std

**LINEMATCHING_REACHED_ENDLOOP**
In linematching, when trying to match lines on the spectrum, the maximum
number of iterations has been reached without converging.

**FORCED_IGNORELINESUPPORT_TO_FALSE**
In line model, for continuum fitting, in the parameters fft processing is
active and ignore line support too (lineModel.continuumFit.ignoreLineSupport).
However masking lines to fit continuum is in this case impossible.
To fix this, ignorelinesupport is forced to False i.e.full spectrum with lines
is used for template fitting.

**FORCED_CONTINUUM_COMPONENT_TO_FROMSPECTRUM**
In line model, while fitting the continuum, a negative continuum amplitude
has been found.
To fix this, continuumComponent is forced to fromSpectrum.

**AIR_VACUUM_REACHED_MAX_ITERATIONS**
Wavelengths convertion from air to vacuum is made by iterating an inverse
translation, until a minimal precision is reached or until the max number
of iterations is reached.
The marning means that max precision was not reached at the end ot the iterations.

**ASYMFIT_NAN_PARAMS**
At least one asymetric fitting param is NaN. All parameters of asymetric line
profile are therefore set to NaN.

**DELTAZ_COMPUTATION_FAILED**
Some error appeared when trying to calculate Deltaz on a redshift candidate
probability.

**INVALID_FOLDER_PATH**
Absolute path to piors calibration file does not exist. No priors used

**FORCED_CONTINUUM_TO_NOCONTINUUM**
The calculated continuum amplitude is weaker than the min continuum amplitude
defined in parameters (`continuumFit.nullThreshold`).
Therefore, continuum is set to noContinuum.

**FORCED_CONTINUUM_REESTIMATION_TO_NO**
Continuum reestimation was set to "onlyextrema" but it is not possible since
second pass is skipped. It is automatically set to no continuum reestimation.

**LESS_OBSERVED_SAMPLES_THAN_AMPLITUDES_TO_FIT**
Deficient rank for lines fitting: when trying to fit lines with line model,
the number of parameters to find is greater than the number of observed samples.

**LBFGSPP_ERROR**
There was an error while fitting whith lbfgspp. We then fix line position and
width and use the initial svd guess for the amplitude.

**PDF_INTEGRATION_WINDOW_TOO_SMALL**
PDF integration range goes beyond redshift window size. Integration range is
therefore cropped.
If this warning appears many times, consider increasing in the parameter
second pass `halfWindowSize` value.

**FORCED_POWERLAW_TO_ZERO**
For QSO power law fitting: many samples have fluxes too low compared to SNR to be taken into account.
Power law coefficients are set to zero.

**UNUSED_PARAMETER**
A parameter has been defined in the parameters file but it is not used.
NB: Some parameters are usefull only under some conditions.

**SPECTRUM_WAVELENGTH_TIGHTER_THAN_PARAM**
Parameters lambda range goes beyond spectrum range. This means that a part of the lambda range specified in the parameters will not be studied.

**MULTI_OBS_ARBITRARY_LSF**
Happens in the case of multi obs, for which only one lsf is support.
Only the lsf of the first observation is taken into account and applied
to all observations.

**NULL_LINES_PROFILE**
One line profile is null (zero flux) on all its samples: the corresponding line is discarded

**STD_ESTIMATION_FAILED**
The number of samples available to estimate the RMS of the residual is too low.

**VELOCITY_FIT_RANGE**
The lineMeas velocity fit range does not include the input velocity
(comming from either the previous redhiftSolver stage, input lineMeas catalog).
The range is thus extended to contain this velocity.


-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-


Expand Down