SpectraVue - An Interactive Web Application Enabling Rapid Data Visualization and Analysis for Wearable Spectroscopy Research

The application was developed using Plotly Dash [6]. Dash is especially useful for this application for several reasons:

1. Allows for the use of existing Python numerical and scientific libraries that are typically used in prototypical research workflows.
   
2. Allows for the rapid creation of graphs and dashboards, while still maintaining customizability relative to other tools such as Tableau and PowerBI.
   
3. Abstracts away the need for intensive application set-up relative to other web application development tools such as JavaScript.
   

Figure 2 displays the landing page when the application is initially launched. The application consists of user input and an adjacent generated graph or video, depending on the mode of operation. The application can operate under two modes: spectrometer mode and biomarker mode. The spectrometer mode is intended for those who simply have Lumos spectrometer data that they would like to visualize. The biomarker mode is intended for those who have spectrometer data, as well as a clinical biomarker for comparison. The app currently only supports a single biomarker for comparison.

The basic flow of the application is as follows, with some minor differences depending on the mode that is chosen: data upload, user input parameters, and graph generation. Each specific step within the flow is expanded upon below. A comprehensive tutorial page is provided that delivers in-depth instructions on the input data required and the data processing and visualization process that the application uses to compute the graphs. More information about the app can be found in the open source repository [3].

The web application accepts two different inputs as file uploads: text files and csv files. The text file data format that is accepted is in the form of JSON [1] objects for each LED and photodiode (PD) response; this format is intended to mimic data that is acquired from the device directly. This format is especially applicable when designing new prototypes for the Lumos device, or when there is not a preexisting data collection server. The csv file data format that is accepted contains columns for each PD spectral response, with each row corresponding to a single LED; this format is intended to mimic data that is acquired from a server that houses Lumos device data. For our use, we utilized Raproto [9] for server set-up, associated communication protocols, and data storage/retrieval. Raproto allows for the rapid adoption and utilization of wearable medical devices such as Lumos by minimizing data collection, transmission, and storage set-up typically associated with data-intensive novel wearable devices.

The user input parameters differ based on the mode; Figure 3 demonstrates these inputs for each mode. Spectrometer mode requires a spectrometer file upload and a selection of a static or animated graph. Biomarker mode simply requires a file uploaded with merged spectrometer and biomarker data. For the merged spectrometer and biomarker data, the following conditions must be satisfied: a) the sampling frequencies must be equal between the two signals; if the sampling frequencies are uneven, the data must be upsampled or downsampled, and b) the biomarker to be analyzed must occupy the last column of the input csv file.

All static plots are generated using Plotly's [5] interactive plotting tool, allowing for full user interactivity and multiple export options. The animated plots are saved as MP4 files in the assets folder of the local directory during processing, which are then displayed using Dash Player [4], a native HTML5 video renderer. Plotly was not used for this as the render time for large time-series inputs caused the application to frequently crash. To compensate, components are provided that allow interactivity with the video player, such as video play-speed.

For spectrometer mode, a static or animated graph can be generated. A static graph will compute the mean counts for each LED-PD pair and plot them as a single snapshot in time, as shown in Figure 4.

An animated graph will plot the input data as a time-series, where each point in time is a single, static, three-dimensional graph representing the counts for each respective LED-PD pair, as demonstrated in Figure 5. The graph is served as an HTML5 video in the application.

For biomarker mode, an animated graph is generated similar to that in spectrometer mode. However, a plot of the desired biomarker is also plotted adjacent to this graph, with a marker on the biomarker graph denoting where in the time-series the animation is currently displaying, as shown in Figure 6.

All graphs provide resultant surface plots by using cubic spline interpolation [7], resulting in a smooth, distributed plane for visualization. This is preferred over a point-by-point analysis as many clinical biomarkers can be detected within a windowed spectral range. Thus, cubic interpolation allows for better comparison by providing a greater spatial spectral resolution in the resultant graph.

The following is a list of planned features for the near future; more may be added depending on user feedback.

1. Additional parsers - adding parsers for different file formats such as npz or other compressed data file types; adding parsers for different data structures/formats.
   
2. Data processing interactivity - a page that allows interactive data processing such as row, column, and data-point addition/deletion, and the addition of various statistical signal processing methodologies such as filtering and anomaly detection; support for PDs/LEDs that are not used in Lumos.
   
3. Dataset download options - data download options for users who simply want their data processed, and will visualize it independently.
   
4. Biomarker mode improvements - ability to visualize multiple biomarkers independently, or at the same time based on user input; ability to use methods such as cross-correlation and Pearson correlation to quantify comparisons in-app.
   
5. Data processing improvements - currently, the application takes up to a minute for larger timeseries data-sets (> 5000 rows). We are considering ways to speed up this computation, including tools such as parallelization and caching.
   
6. Mobile device adaptation - we are currently exploring ways to incorporate this analysis into the Raproto [9] workflow to facilitate real-time viewing and analysis of collected data.
   

Our primary objective is to collaborate closely with interested research groups to establish the best pathway for integrating SpectraVue into their respective research workflows. Currently, one solution we are considering is hosting the application on a dedicated physical server. This approach could offer several benefits over local execution. Firstly, it would potentially provide better performance and more storage, especially for large-scale computations or for research groups with extensive data sets. Secondly, it would promote more efficient collaboration, as data and computations could be easily accessed and shared. Lastly, it could pave the way for a standardized spectroscopic database that can be used for joint analysis of specific biomarkers across research groups.