Overview of the data flow in SBART#
Goals of this notebook:
Very brief explanation/overview of the basic user interface to load data from disk:
Load data from .fits files
Configure the “instrument”
Reject observations based on different conditions (e.g. HEADER values)
Reject wavelength regions
Loading data from disk#
In this Section we look at how we can load spectral data from disk
[1]:
from SBART.data_objects import DataClassManager, DataClass
from SBART.outside_tools.create_logger import setup_SBART_logger
/home/amiguel/.virtualenvs/seminar-Pt5BMgSq-py3.8/lib/python3.8/site-packages/pysynphot/locations.py:46: UserWarning: PYSYN_CDBS is undefined; functionality will be SEVERELY crippled.
warnings.warn("PYSYN_CDBS is undefined; functionality will be SEVERELY "
/home/amiguel/.virtualenvs/seminar-Pt5BMgSq-py3.8/lib/python3.8/site-packages/pysynphot/locations.py:345: UserWarning: Extinction files not found in extinction
warnings.warn('Extinction files not found in %s' % (extdir, ))
How to “point” towards the .fits files#
[2]:
# Option a) -> path to a disk file that contains (in each line) the full path to the .fits files (either S1D or S2D)
data_in_path = '/home/amiguel/seminar/test_new_version/espresso_data_in.txt'
# Option b) List with entries that are full paths to the .fits files
data_in_list = ["full_path_1.fits"]
Selection and configuration of the Instrument#
[3]:
from SBART.Instruments import ESPRESSO, CARMENES
# Select one of the available instruments in the SBART.Instruments (currently HARPS/CARMENES/ESPRESSO - HARPS temporarily disabled)
instrument = ESPRESSO
# Each instrument can be (optionally) configured. The available configurations can only be seen inside the code (no 'external docs') in the following places:
# i) Global configurations to ALL instruments: SBART/Base_Models/Frame.py
# ii) Instrument-specific configurations: SBART/Instruments/<inst_name>.py
#
# The available configurations and their default values are placed inside the "controllable_options" dictionary, as a class attribute.
# To change them, we define a dictionary with Keys being the configuration name and values the new value
inst_options = {"minimum_order_SNR" : 10,
"spectra_format" : "S2D" # S1D if we want to use those files; For the S2D mode we can use either S2D_A or S2D_BLAZE_A files
}
Configure the logs of the library#
If this is not used, there will be no information printed on the terminal. Can be called multiple times, to “point” towards different folders
[4]:
log_storage_path = "" # Root folder for the logger (path to that folder)
rv_method = "" # Name of a folder created inside the root, e.g. log_storage_path/rv_method
write_to_file = False # if False, only prints to the terminal
setup_SBART_logger(log_storage_path, rv_method, instrument=instrument, write_to_file=False)
2021-12-02T22:46:15 - SBART.outside_tools.create_logger - WARNING -Not storing logs to disk
Loading the data from disk#
There are two ways of loading the data from disk (that work in the same fashion): - A) Load the data as an independent process - must be used when we want to compute RVs (to work with parallel code) - B) Load the data in the main python process
Note: Generally better to just go with option A) all the time
The entire SBART pipeline is built in such a way that it can process data from multiple “time divisions” of the same instrument individually. For example, it can recognize data from ESPRESSO18, 19 and 21, create individual stellar templates and compute ‘individual’ RVs
[5]:
load_independent_process = False
if load_independent_process: # Option A)
manager = DataClassManager()
manager.start()
data = manager.DataClass(data_in_path,
instrument=ESPRESSO,
instrument_options=inst_options,
storage_path=""
)
else: # Option B)
data = DataClass(data_in_path,
instrument=instrument,
instrument_options=inst_options,
storage_path=""
)
2021-12-02T22:46:15 - SBART.data_objects.DataClass - INFO -DataClass loading data from /home/amiguel/seminar/test_new_version/espresso_data_in.txt
2021-12-02T22:46:15 - SBART.Base_Models.Frame - INFO -Updating class properties from default values of {'bypass_QualCheck': False, 'reject_order_percentage': 0.25, 'minimum_order_SNR': 20, 'spectra_format': 'S2D', 'apply_FluxCorr': False} to: {'minimum_order_SNR': 10}
2021-12-02T22:46:15 - SBART.Base_Models.Frame - INFO -Creating frame from: /home/amiguel/seminar/test_new_version/dataset/r.ESPRE.2019-02-10T08:25:37.895_S2D_BLAZE_A.fits
2021-12-02T22:46:15 - SBART.Base_Models.Frame - DEBUG -Validating header KeyWords
2021-12-02T22:46:15 - SBART.Instruments.ESPRESSO - INFO -Found 0 warning flags in the header KWs
2021-12-02T22:46:15 - SBART.Base_Models.Frame - INFO -Frame determined to be from ESPRESSO18
2021-12-02T22:46:15 - SBART.Base_Models.Frame - INFO -Creating frame from: /home/amiguel/seminar/test_new_version/dataset/r.ESPRE.2019-02-18T04:41:22.303_S2D_BLAZE_A.fits
2021-12-02T22:46:15 - SBART.Base_Models.Frame - DEBUG -Validating header KeyWords
2021-12-02T22:46:15 - SBART.Instruments.ESPRESSO - INFO -Found 0 warning flags in the header KWs
2021-12-02T22:46:15 - SBART.Base_Models.Frame - INFO -Frame determined to be from ESPRESSO18
2021-12-02T22:46:15 - SBART.Base_Models.Frame - INFO -Creating frame from: /home/amiguel/seminar/test_new_version/dataset/r.ESPRE.2019-02-19T07:49:19.028_S2D_BLAZE_A.fits
2021-12-02T22:46:15 - SBART.Base_Models.Frame - DEBUG -Validating header KeyWords
2021-12-02T22:46:15 - SBART.Instruments.ESPRESSO - INFO -Found 0 warning flags in the header KWs
2021-12-02T22:46:15 - SBART.Base_Models.Frame - INFO -Frame determined to be from ESPRESSO18
2021-12-02T22:46:15 - SBART.data_objects.DataClass - INFO -Collecting MetaData from the observations
2021-12-02T22:46:15 - SBART.data_objects.Target - INFO -Found KOBE dictionary; Loading keys
2021-12-02T22:46:15 - SBART.data_objects.Target - DEBUG -Parsing through loaded OBJECTs
2021-12-02T22:46:15 - SBART.data_objects.Target - INFO -Validated target to be V V645 Cen
2021-12-02T22:46:15 - SBART.data_objects.DataClass - INFO -Loaded data from DRS version {'espdr/2.2.8'}
2021-12-02T22:46:15 - SBART.data_objects.DataClass - INFO -Loaded data from DRS version {'M5'}
2021-12-02T22:46:15 - SBART.data_objects.DataClass - INFO -
+---------------+-----------+----------------------+-------------+
| subInstrument | Total OBS | Valid OBS [warnings] | INVALID OBS |
+---------------+-----------+----------------------+-------------+
| ESPRESSO18 | 3 | 3 [0] | 0 |
+---------------+-----------+----------------------+-------------+
| ESPRESSO19 | 0 | 0 [0] | 0 |
+---------------+-----------+----------------------+-------------+
| ESPRESSO21 | 0 | 0 [0] | 0 |
+---------------+-----------+----------------------+-------------+
| Total | 3 | 3 [0] | 0 |
+---------------+-----------+----------------------+-------------+
Removing activity indicators (Optional)#
By default, we remove lines that are typically used as activity indicators
This interface can also be used to manuall remove other wavelength regions
[12]:
from SBART.Quality_Control.activity_indicators import Indicators
inds = Indicators()
Removing extra regions#
We must define a unique name (i.e. no repetitions, even among the default “features”
We must define a region that will be removed from all observations that have been loaded from disk
BY default we assume that the region is defined in air. Change to vacuum by passing vacuum_wavelength=True
[13]:
inds.add_feature(name="feature_1",
region=[5000, 5500],
vacuum_wavelengths=True
)
Applying the selected region#
[14]:
data.remove_activity_lines(inds)
2021-12-02T22:46:15 - SBART.data_objects.DataClass - INFO -Computing activity windows for each RV measurements