pycrtools.crdatabase

BaseParameter(parent)	Base parameter functionality for the cr database.
CRDatabase([filename, datapath, ...])	Functionality to let the VHECR pipeline communicate with an SQL database.
Datafile([db, id])	CR datafile information.
DatafileParameter([parent])	Container class for datafile parameters.
Event([db, id])	CR event information.
EventParameter([parent])	Container class for event parameters.
Polarization([db, id])	CR polarization information.
PolarizationParameter([parent])	Container class for polarization parameters.
Settings([db])	Global settings of the CR database.
Station([db, id])	CR station information.
StationParameter([parent])	Container class for station parameters.
pickle_parameter(value)	Return the parameter as a database friendly pickle representation.
unpickle_parameter(value)	Return the parameter value from parsing a database friendly representation of the parameter.
unpickle_parameter_old(value)	Return the parameter value from parsing a database friendly representation of the parameter.

Interface class to communicate with the Cosmic Ray Pipeline Database.

Author:Martin van den Akker <m.vandenakker@astro.ru.nl>

class pycrtools.crdatabase.BaseParameter(parent)

Base parameter functionality for the cr database.

keys()

Return a list of valid parameter keys.

read()

Read parameters from the database.

summary(parent_class_name='BaseParameter')

Summary of the parameter object.

update()

Write updated parameters to the database.

write()

Write parameters to the database.

class pycrtools.crdatabase.CRDatabase(filename=':memory:', datapath='', resultspath='', lorapath='', host=None, user=None, password=None, dbname=None, basepath=None)

Functionality to let the VHECR pipeline communicate with an SQL database.

deleteDatafile(id=0)

Delete a datafile and its underlying information objects from the database.

Properties

Parameter	Description
id	id of the requested datafile entry in the database.

deleteEvent(id=0)

Delete an event and its underlying information objects from the database.

Properties

Parameter	Description
id	id of the requested event entry in the database.

deletePolarization(id=0)

Delete a polarization and its underlying information objects from the database.

Properties

Parameter	Description
id	id of the requested polarization entry in the database.

deleteStation(id=0)

Delete a station and its underlying information objects from the database.

Properties

Parameter	Description
id	id of the requested station entry in the database.

getDatafile(id=0)

Get the Datafile with a specific id.

Properties

Parameter	Description
id	id of the requested datafile.

getDatafileIDs(eventID=None, filename=None, status=None, order='')

Return a list of datafileIDs satisfying the values of this functions arguments.

Properties

Parameter	Description
eventID	id of the event.
filename	name of the datafile.
status	status of the datafile.
order	database fields that are used to sort the returned records.

If no arguments are given all datafileIDs are selected. When multiple arguments are provided, all returned datafileIDs satisfy all argument values.

getEvent(id=0)

Get the event with a specific id.

Properties

Parameter	Description
id	id of the requested event entry in the database.

getEventIDs(timestamp=None, timestamp_start=None, timestamp_end=None, status=None, datafile_name=None, antennaset=None, order='e.timestamp')

Return a list of eventIDs satifying the values of this functions arguments.

Properties

Parameter	Description
timestamp	timestamp of the event is equal to this value.
timestamp_start	timestamp of the event is larger than this value.
timestamp_end	timestamp of the event is smaller than this value.
status	status of the event.
datafile_name	name of a datafile associated to an event.
antennaset	type of the antennaset.
order	database fields that are used to sort the returned records.

The timestamp, timestamp_start and timestamp_end parameters are integers representing the unix-time of the timestamp. If timestamp is provided, than timestamp_start and timestamp_end are ignored. If no arguments are given all eventIDs are selected.

When multiple arguments are provided, all returned eventIDs satisfy all argument values.

getPolarization(id=0)

Get the Polarization with a specific id.

Properties

Parameter	Description
id	id of the requested polarization.

getPolarizationIDs(eventID=None, datafileID=None, stationID=None, antennaset=None, direction=None, status=None, order='')

Return a list of polarizationIDs that satisfy the values of the provided arguments of this method.

Properties

Parameter	Description
eventID	id of the event.
datafileID	id of the datafile.
stationID	id of the station.
antennaset	type of antennaset.
direction	direction of the polarization.
status	status of the polarization.
order	database fields that are used to sort the returned records.

If no arguments are given all polarizationIDs are selected. When multiple arguments are provided, the returned polarizationIDs satisfy all argument values that are provided.

getStation(id=0)

Get the Station with a specific id.

Properties

Parameter	Description
id	id of the requested station.

getStationIDs(eventID=None, datafileID=None, stationname=None, status=None, order='s.stationname')

Return a list of stationIDs that satisfy the values of the provided arguments of this method.

Properties

Parameter	Description
eventID	id of the event.
datafileID	id of the datafile.
stationname	name of the station.
status	status of the station.
order	database fields that are used to sort the returned records.

If no arguments are given all stationIDs are selected. When multiple arguments are provided, the returned stationIDs satisfy all argument values that are provided.

isLocked()

Check if the database is locked.

summary()

Summary of the CRDatabase object.

unlock()

Unlock the database to be able to modify it. An unlocked database cannot be locked again!

class pycrtools.crdatabase.Datafile(db=None, id=0)

CR datafile information.

This object contains the following information of a datafile:

• filename: the filename of the datafile.
• status: the status of the datafile.
• stations: a list of station information objects (Station) that are stored in the datafile.
• parameter: a dictionary of optional parameters with additional information for this specific datafile.

addStation(station=None)

Add a station object to this event.

This adds a station object to the list of stations of the current datafile object and updates the database.

Properties

Parameter	Description
station	station object to be added to the list of stations of this datafile object.

Returns True if adding the station object went successfully, False otherwise.

id

Return the ID of the object as it is identified in the database.

inDatabase()

Check if the datafile information is available in the database.

This method checks if the datafile information of this object, uniquely identified by the id and the filename, is in the database.

If the datafile information is in the database True is returned, False otherwise.

read()

Read datafile information from the database.

removeStation(station=None, stationID=0)

Remove station object with id= stationID from this datafile.

This removes the station information object from this datafile object and updates the database. Note that the station info is not deleted from the database: only the link between the datafile and the station objects is deleted.

Properties

Parameter	Description
station	station that needs to be removed from this datafile.
stationID	id of the station that needs to be removed from this datafile.

If station is not provided, stationID will be used to remove it from the list of stations.

Returns True if removing the station object went successfully, False otherwise.

summary(showParameters=False)

Summary of the Datafile object.

Properties

Parameter	Description
showParameters	Flag to enable/disable (default disabled) the display of all parameters associated with this information object.

update(recursive=True, parameters=True)

Update datafile information to the database in a buffered (=faster) way.

Properties

Parameter	Description
recursive	if True write all underlying datastructures (stations, etc.)
parameters	if True write all parameters

write(recursive=True, parameters=True)

Write datafile information to the database.

Properties

Parameter	Description
recursive	if True write all underlying datastructures (stations, etc.)
parameters	if True write all parameters

class pycrtools.crdatabase.DatafileParameter(parent=None)

Container class for datafile parameters.

summary()

Summary of the DatafileParameter object.

class pycrtools.crdatabase.Event(db=None, id=0)

CR event information.

This object contains the following information of an event:

• timestamp: the timestamp of the event in UTC stored as the number of seconds after 1 January 1970.

• status: the status of the event.

• datafiles: a list of datafile information objects (Datafile) associated with this event.

• parameter: a dictionary of optional parameters with additional information for this specific event. The database supports the following parameter keywords:

  crp_average_direction, lora_10_nsec, lora_azimuth, lora_core,sss lora_core_x, lora_core_y, lora_coree_x, lora_coree_y, lora_coreuncertainties, lora_datafile, lora_detectorid, lora_direction, lora_elevation, lora_energy, lora_energy_ev, lora_ldf, lora_moliere, lora_moliere_rad_m, lora_nsecs, lora_particle_density__m2, lora_posx, lora_posy, lora_posz, lora_time, lora_utc_time_secs, plotfiles.

addDatafile(datafile=None)

Add a datafile object to this event.

This adds a datafile object to the current event object and updates the database.

Properties

Parameter	Description
datafile	datafile object to be linked to the current event.

Returns True if adding the datafile object went successfully, False otherwise.

alt_status

Get event alt_status.

alt_statusmessage

Get event alt_statusmessage.

antennaset

Get the antennaset.

flagged

Get event flagged

flagged_reason

Get event flagged_reason

id

Return the ID of the object as it is identified in the database.

inDatabase()

Check if event information is available in the database.

This method checks if the event information of this object, uniquely identified by the id and the timestamp, is already in the database.

If the event information is in the database True is returned, False otherwise.

is_cr_found(alt_status=False)

Check if a polarization with a CR detection is found within this event.

Returns True if a polarization with a CR detection (polarization.status==’GOOD’) is found, False otherwise.

read()

Read event information from the database.

removeDatafile(datafile=None, datafileID=0)

Remove datafile object with id= datafileID from this event.

This removes the datafile information object from this event object and updates the database. Note that the datafile info is not deleted from the database: only the link between the event and the datafile objects is deleted.

Properties

Parameter	Description
datafile	datafile that needs to be removed from this event.
datafileID	id of the datafile that needs to be removed from this event.

If datafile is not provided, datafileID will be used to remove it from the list of datafiles.

Returns True if removing the datafile object went successfully, False otherwise.

simulation_status

Get event simulation_status.

simulation_statusmessage

Get event simulation_statusmessage.

status

Get event status

statusmessage

Get event statusmessage

summary(showParameters=False)

Summary of the Event object.

Properties

Parameter	Description
showParameters	Flag to enable/disable (default disabled) the display of all parameters associated with this information object.

update(recursive=True, parameters=True)

Update event information to the database in a buffered (=faster) way.

Properties

Parameter	Description
recursive	if True write all underlying datastructures (datafiles, etc.)
parameters	if True write all parameters

write(recursive=True, parameters=True)

Write event information to the database.

Properties

Parameter	Description
recursive	if True write all underlying datastructures (datafiles, etc.)
parameters	if True write all parameters

class pycrtools.crdatabase.EventParameter(parent=None)

Container class for event parameters.

summary()

Summary of the EventParameter object.

class pycrtools.crdatabase.Polarization(db=None, id=0)

CR polarization information.

This object contains the following information of a polarization:

• antennaset: the name of the antennaset (e.g. LBA_OUTER, LBA_INNER, etc.).

• direction: the direction of the polarization component.

• status: the status of the polarization.

• resultsfile: the name of the resultsfile, where all the results from the pipeline are stored.

• parameter: a dictionary of optional parameters. This contains also resulting information from the pipeline. The database supports the following parameter keywords:

  antenna_positions_array_xyz_m, antenna_positions_rf, antenna_positions_station_xyz_m, antenna_set, antennas, antennas_final_cable_delays, antennas_flagged_delays, antennas_residual_cable_delays, antennas_spectral_power, antennas_timeseries_npeaks, antennas_timeseries_rms, antennas_with_peaks, antennas_with_strong_pulses, bad_antennas, block, blocksize, crp_itrf_antenna_positions, crp_polarization_angle, crp_pulse_delays, crp_pulse_peak_amplitude, crp_rms, crp_stokes, data_length, data_length_ms, delay_quality_error, dipole_calibration_delay_applied, dirty_channels, filedir, filename, frequency_range, nantennas_with_strong_pulses, ndipoles, nof_dipole_datasets, npeaks_found, nyquist_zone, pipeline_version, plotfiles, polarization, pulse_core_lora, pulse_coreuncertainties_lora, pulse_direction, pulse_direction_delta_delays_final, pulse_direction_delta_delays_start, pulse_direction_lora, pulse_direction_planewave, pulse_end_sample, pulse_energy_lora, pulse_height, pulse_height_incoherent, pulse_height_rms, pulse_location, pulse_moliere_lora, pulse_normalized_height, pulse_start_sample, pulse_time_ms, pulses_absolute_arrivaltime, pulses_maxima_x, pulses_maxima_y, pulses_power_snr, pulses_refant, pulses_sigma, pulses_strength, pulses_timelags_ns, sample_frequency, sample_interval, sample_number, station_antennas_homogeneity_factor, station_spectral_power, station_timeseries_npeaks, station_timeseries_rms, status, svn_revision, telescope, time, timeseries_power_mean, timeseries_power_rms, timeseries_raw_rms, timeseries_rms.

alt_status

Get polarization alt_status

alt_statusmessage

Get polarization alt_statusmessage

flagged

Get polarization flagged

flagged_reason

Get polarization flagged_reason

id

Return the ID of the object as it is identified in the database.

inDatabase()

Check if the polarization information is available in the database.

This method checks if the polarization information of this object, uniquely identified by id, is in the database.

If the polarization information is in the database True is returned, False otherwise.

read()

Read polarization information from the database.

status

Get polarization status

statusmessage

Get polarization statusmessage

summary(showParameters=False)

Summary of the Polarization object.

Properties

Parameter	Description
showParameters	Flag to enable/disable (default disabled) the display of all parameters associated with this information object.

update(parameters=True)

Update polarization information to the database in a buffered (=faster) way.

Properties

Parameter	Description
parameters	if True write all parameters

write(recursive=True, parameters=True)

Write polarization information to the database.

Properties

Parameter	Description
recursive	if True write all underlying data structures.
parameters	if True write all parameters.

class pycrtools.crdatabase.PolarizationParameter(parent=None)

Container class for polarization parameters.

summary()

Summary of the PolarizationParameter object.

class pycrtools.crdatabase.Settings(db=None)

Global settings of the CR database.

This container class contains the following settings from the database:

• datapath: basepath from which the datafiles can be found.
• resultspath: basepath from which the results can be found.

datapath

Get the value of the datapath variable as set in the database.

db_version

Get the value of the db_version variable as set in the database.

lorapath

Get the value of the lorapath variable as set in the database.

resultspath

Get the value of the resultspath variable as set in the database.

summary()

Summary of the Settings object.

class pycrtools.crdatabase.Station(db=None, id=0)

CR station information.

This object contains the following information of a station:

• stationname: the name of the station, e.g. CS001, RS203.

• status: the status of the station.

• polarizations: a list of polarization information objects (Polarization) stored in this station.

• parameter: a dictionary of optional parameters with additional information for this specific station. The database supports the following parameter keyword:

  crp_pulse_direction, plotfiles.

addPolarization(polarization=None)

Add a polarization object to this station.

This adds a polarization object to the current station object and updates the database.

Properties

Parameter	Description
polarization	polarization object to be linked to the current station.

Returns True if adding the polarization object went successfully, False otherwise.

alt_status

Get station alt_status

alt_statusmessage

Get station alt_statusmessage

flagged

Get station flagged

flagged_reason

Get station flagged_reason

id

Return the ID of the object as it is identified in the database.

inDatabase()

Check if the station information is available in the database.

This method checks if the station information of this object, uniquely identified by id, is in the database.

If the station information is in the database True is returned, False otherwise.

read()

Read station information from the database.

removePolarization(polarization=None, polarizationID=0, key=None)

Remove polarization object with id= polarizationID from this station.

This removes the polarization information object from this station object and updates the database. Note that the polarization info is not deleted from the database: only the link between the station and the polarization objects is deleted.

Properties

Parameter	Description
polarization	polarization that needs to be removed from this station.
polarizationID	id of the polarization that needs to be removed from this station.
key	key name of the polarization, this is equal to the polarization direction.

Only one of the parameters is used to identify the polarization that needs to be removed. The order of evaluating the parameters to identify the polarization is polarization, polarizationID, key

Returns True if removing the polarization object went successfully, False otherwise.

status

Get station status

statuscategory

Get station statuscategory

statusmessage

Get station statusmessage

summary(showParameters=False)

Summary of the Station object.

Properties

Parameter	Description
showParameters	Flag to enable/disable (default disabled) the display of all parameters associated with this information object.

update(recursive=True, parameters=True)

Update station information to the database in a buffered (=faster) way.

Properties

Parameter	Description
recursive	if True write all underlying datastructures (polarizations, etc.)
parameters	if True write all parameters

write(recursive=True, parameters=True)

Write station information to the database.

Properties

Parameter	Description
recursive	if True write all underlying datastructures (polarizations)
parameters	if True write all parameters

class pycrtools.crdatabase.StationParameter(parent=None)

Container class for station parameters.

summary()

Summary of the StationParameter object.

pycrtools.crdatabase.pickle_parameter(value)

Return the parameter as a database friendly pickle representation.

Properties

Parameter	Description
value	Python representation of the value of the parameter.

pycrtools.crdatabase.unpickle_parameter(value)

Return the parameter value from parsing a database friendly representation of the parameter.

Properties

Parameter	Description
value	Database representation of the parameter value.

pycrtools.crdatabase.unpickle_parameter_old(value)

Return the parameter value from parsing a database friendly representation of the parameter.

Properties

Parameter	Description
value	Database representation of the parameter value.