Difference between revisions of "STIX data center"

From stix
Jump to: navigation, search
(Flare identification)
 
(13 intermediate revisions by the same user not shown)
Line 1: Line 1:
This page is still under construction.
+
The STIX data center is an automated data processing, management and visualization platform developed by a team lead by [mailto:hualin.xiao@fhnw.ch Hualin Xiao], in order to process and, archive STIX telemetry data, support the operations of the instrument, and science activities with STIX data.  It receives STIX telemetry data from ESA, and then processes it into different data products automatically.  Analysis reports, instrument health status, and analysis results are displayed using responsive web GUIs.  The whole system has been deployed on a cloud server at FHNW.  
  
STIX data center is a virtual server hosted at FHNW.  
+
Thanks to its modular design and compatibility with ESA’s mission control system  SCOS2000,  the system can be easily adapted for other space missions.   After some minor modifications, it can also be applied to other use cases, such as automated data processing platforms for large scientific facilities and the Internet of Things (IoT).
== STIX Data Center tasks ==
 
Its tasks include
 
* Process STIX raw data & produce high level data products
 
* Monitor Data quality and instrument health status
 
* Provide data products and data browsing tools for  the Solar Physics Community
 
* Support STIX operations
 
* Data archiving
 
== STIX data center architecture ==
 
The data flow at STIX data center is shown in the picture below:
 
  
[[File:Stix-data-center-flow.png|500px|none|STIX data center]]
+
STIX data center main features:
Data processing SW  mainly written in python.
 
NoSQL DB (MongoDB) used to stored telemetry packets, file metadata and configurations
 
  
== Key Algorithms ==
+
- Automated data processing platform for STIX
=== Offline flare location solution ===
 
  
 +
- Cloud-based
  
===  Flare  identification ===
+
- Interactive web-GUIs and Python APIs
  
Quick-look light curves are used for flare identification.  Identifications of flares consists of several steps as follows:
+
- Automated monitoring of data quality and instrument health
Light curves are smoothed by using average filtering in order to filter spikes from the electronics and fluctuations due to the on-board integer compression.
 
Then, light curves acquired during the quiet sun periods are selected for background estimation. 
 
We take  median values and standard deviations of a certain number of data points (typically 1500 points) as the background level.
 
Then the smoothed light curves are used to search for peaks.
 
A flare is considered being detected if a
 
peak satisfies several conditions as follows:
 
* The peak counts is above 2 sigma of the background.
 
  
== Source code ==
+
- Support ESA's SCOS2000 standards
The source code can be found at:
 
* Data processing pipeline:  https://github.com/i4Ds/STIX_python_PUB023
 
* STIX data center web applications:  https://github.com/i4Ds/STIX-data-center
 
== Developers ==
 
* Data processing software & data processing pipeline developers
 
** [mailto:hualin.xiao@fhnw.ch Hualin Xiao]
 
** [mailto:shane.maloney@tcd.ie Shane Maloney]
 
  
* Data center  
+
- Can be adapted to other space missions and user cases easily
** [mailto:hualin.xiao@fhnw.ch Hualin Xiao]
+
 
 +
STIX data center contact person: [mailto:hualin.xiao@fhnw.ch Hualin Xiao] <br>
 +
[https://datacenter.stix.i4ds.net/ STIX data center Portal]

Latest revision as of 14:46, 29 August 2022

The STIX data center is an automated data processing, management and visualization platform developed by a team lead by Hualin Xiao, in order to process and, archive STIX telemetry data, support the operations of the instrument, and science activities with STIX data. It receives STIX telemetry data from ESA, and then processes it into different data products automatically. Analysis reports, instrument health status, and analysis results are displayed using responsive web GUIs. The whole system has been deployed on a cloud server at FHNW.

Thanks to its modular design and compatibility with ESA’s mission control system SCOS2000, the system can be easily adapted for other space missions. After some minor modifications, it can also be applied to other use cases, such as automated data processing platforms for large scientific facilities and the Internet of Things (IoT).

STIX data center main features:

- Automated data processing platform for STIX

- Cloud-based

- Interactive web-GUIs and Python APIs

- Automated monitoring of data quality and instrument health

- Support ESA's SCOS2000 standards

- Can be adapted to other space missions and user cases easily

STIX data center contact person: Hualin Xiao
STIX data center Portal