Difference between revisions of "STIX data center"

From stix
Jump to: navigation, search
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page is still under construction.
+
The STIX data center is an automated data processing, management and visualization platform developed by a team lead by [mailto:hualin.xiao@fhnw.ch Hualin Xiao], in order to process and, archive STIX telemetry data, support the operations of the instrument, and science activities with STIX data.  It receives STIX telemetry data from ESA, and then processes it into different data products automatically.  Analysis reports, instrument health status, and analysis results are displayed using responsive web GUIs.  The whole system has been deployed on a cloud server at FHNW.  
  
STIX data center is a virtual server hosted at FHNW.  
+
Thanks to its modular design and compatibility with ESA’s mission control system  SCOS2000,  the system can be easily adapted for other space missions.   After some minor modifications, it can also be applied to other use cases, such as automated data processing platforms for large scientific facilities and the Internet of Things (IoT).
== STIX Data Center tasks ==
 
Its tasks include
 
* Process STIX raw data & produce high level data products
 
* Monitor Data quality and instrument health status
 
* Provide data products and data browsing tools for  the Solar Physics Community
 
* Support STIX operations
 
* Data archiving
 
== STIX data center architecture ==
 
The data flow at STIX data center is shown in the picture below:
 
  
[[File:Stix-data-center-flow.png|500px|none|STIX data center]]
+
STIX data center main features:
Data processing SW  mainly written in python.
 
NoSQL DB (MongoDB) used to stored telemetry packets, file metadata and configurations
 
  
== Key Algorithms ==
+
- Automated data processing platform for STIX
=== Offline flare location solution ===
 
  
 +
- Cloud-based
  
===  Flare  identification ===
+
- Interactive web-GUIs and Python APIs
4-10 keV quick-look light curves are used for flare identification.
 
Identifications of flares consists of several steps as follows:
 
* Light curve smoothing. 4-sec time bins  are  rebinned to 1 minute
 
time bins in order to filter spikes from the electronics and fluctuation  due to statistics and the on-board integer compression;
 
* Background estimation.  The light curves acquired during sun quiet periods  are selected for background estimation.
 
*
 
  
The smoothed light curves have time bins of 1 minute.
+
- Automated monitoring of data quality and instrument health
Those during the quiet period of the sun.  to calculate the background
 
Quick-look light curves are
 
used for flare identification.
 
  
Then the smoothed light curves are used
+
- Support ESA's SCOS2000 standards
to search for peaks.
 
A flare is considered being detected if a
 
peak satisfies several conditions as follows:
 
* The peak counts is above 2 sigma of the background.
 
  
== Source code ==
+
- Can be adapted to other space missions and user cases easily
The source code can be found at:
 
* Data processing pipeline:  https://github.com/i4Ds/STIX_python_PUB023
 
* STIX data center web applications:  https://github.com/i4Ds/STIX-data-center
 
== Developers ==
 
* Data processing software & data processing pipeline developers
 
** [mailto:hualin.xiao@fhnw.ch Hualin Xiao]
 
** [mailto:shane.maloney@tcd.ie Shane Maloney]
 
  
* Data center  
+
STIX data center contact person: [mailto:hualin.xiao@fhnw.ch Hualin Xiao] <br>
** [mailto:hualin.xiao@fhnw.ch Hualin Xiao]
+
[https://datacenter.stix.i4ds.net/ STIX data center Portal]

Latest revision as of 14:46, 29 August 2022

The STIX data center is an automated data processing, management and visualization platform developed by a team lead by Hualin Xiao, in order to process and, archive STIX telemetry data, support the operations of the instrument, and science activities with STIX data. It receives STIX telemetry data from ESA, and then processes it into different data products automatically. Analysis reports, instrument health status, and analysis results are displayed using responsive web GUIs. The whole system has been deployed on a cloud server at FHNW.

Thanks to its modular design and compatibility with ESA’s mission control system SCOS2000, the system can be easily adapted for other space missions. After some minor modifications, it can also be applied to other use cases, such as automated data processing platforms for large scientific facilities and the Internet of Things (IoT).

STIX data center main features:

- Automated data processing platform for STIX

- Cloud-based

- Interactive web-GUIs and Python APIs

- Automated monitoring of data quality and instrument health

- Support ESA's SCOS2000 standards

- Can be adapted to other space missions and user cases easily

STIX data center contact person: Hualin Xiao
STIX data center Portal