A novel mobile application to examine impaired vigilance through digital means

In this study, we aimed to study the differences in median and mean reaction times between the 10-minute-based standard PVT_std and the 3-minute-based mobile application-based PVT_SR. We observed that the median and mean RTs were always 50–70 ms longer on average in the PVT_SR compared to the PVT_std. The novel PVT_SR method, despite showing significantly longer median and mean RTs compared to PVT_std, demonstrated a statistically significant correlation with PVT_std.

The systematically higher RTs in PVT_SR than PVT_std could be due to multiple reasons. First, participants needed to tap the spacebar on the keyboard in the PVT_std. In contrast, participants were required to tap on the touchscreen in the mobile application (PVT_SR), and tapping on a mobile touchscreen might take longer than tapping on a keyboard¹⁹. Second, the PVT_std test was conducted in a calm, silent laboratory setting, whereas the PVT_SR was performed in a more comfortable and relaxed home environment, which, being rich in stimuli, may have been more distracting and contributed to longer reaction times²⁰. In addition, there was no control for what finger particiapnts use; previous study also showed that using the index finger to tap resulted in the fastest task completion time compared to all other fingers¹⁹. Third, another key difference between the standard PVT and the SR app is that the PVT_std provides live feedback (e.g., a timer display), which the PVT_SR app lacks; this absence may affect engagement and performance, potentially contributing to longer reaction times. Therefore, other factors, such as device screen size and thumb or index finger use, feedback differences could account for longer response time in PVT_SR.

Our results align with an earlier study, which reported that the response times are longer in a 3-minute PVT than in a 10-minute PVT¹². However, when they corrected the systematic error by a scaling factor of 0.75, the response times became similar to the standard 10-minute PVT test¹². Although we did not include scaling factors in our results, we also investigated with adjusted scaling factors to observe whether that makes any difference in PVT_SR response time. We adjusted our results with a scaling factor of 0.70, which makes the difference between the response time of PVT_std and PVT_SR small. For instance, the mean reaction time for all participants in PVT_std was 350.2 ms, and after adding a scaling factor of 0.70, the mean reaction time for all participants in PVTSR decreased to 349.1 ms. The response time for PVT_SR for the OSA severity groups became ± 10 ms less than in PVT_std. Thus, systematic delays or device errors in sleep revolution applications can also be a reason for longer PVT_SR response times in our analysis. We also observed that the agreement between the PVTs deteriorated along worsening OSA severity. The difference between the two PVTs in median RTs was 57.4 ms in healthy participants, while the difference increased to 70.2 ms in mild and further to 108.6 ms in participants with moderate to severe OSA (Table 2). The Bland-Altman plot (Fig. 5) shows that the discrepancy between the two methods increasesfor participants with higher reaction times. This pattern suggests that the methods diverge more as reaction times get longer, which could be important when interpreting results or choosing a measurement method for participants who tend to have lower vigilance. However, despite these increases in differences in RTs the correlation between the two PVTs for all groups was high and the pattern of increased RTs with increased OSA severity was captured well with both PVT tests.

In general, the present results indicate that our PVT_SR method is adequate for measuring vigilance and capturing the pattern of increased response time due to the progression of OSA severity similar to the standard PVT. Moreover, the test is available through any smartphone, which makes its usage easy and cost-effective, allowing, e.g., measurements among larger patient groups, estimation of changes in vigilance over time, and following how OSA treatment affects vigilance. Importantly, the PVT_SR is not intended to replace the standard in-lab PVT but rather to expand its applicability and enable long-term follow-up monitoring of vigilance. It can also guide individuals to keep track of their habitual sleep behaviors on their path to optimal sleep health².

Our study had a few limitations to acknowledge. First, the PVT_std was performed at the lab, while PVT_SR was performed in a home environment. For instance, while the in-lab PVT was more controlled, the PVT_SR test was not restricted to any specific time, space, or environment²⁰. Participants could have taken the test anytime during the week and multiple times during the study period. Secondly, the PVT methods differed; PVT_std involved keyboard tapping, while PVT_SR used mobile screen tapping, and only PVT_std provided live feedback to the participants. These factors can impact the reaction times determined by using the two different methods for PVTs. In addition, we had limited information on whether there was a delay due to the test administration hardware and software platforms in the Sleep Revolution application, and thus, we did not assess any system delay in the PVT_SR. System delays or device errors can affect reaction times¹², and this fact might have also impacted the present analysis. Lastly, the errors and systematic discrepancies between the two methods warrant further investigation. Consequently, it is premature to confidently assert that the proposed method can fully replace the structured laboratory test. Nevertheless, our objective was not to replace the standard method, but the proposed approach appears promising and enables conducting the PVT multiple time over longer period which would be infeasible with the laboratory test. However, the test warrants further exploration, ideally with a larger sample size and more comprehensive statistical analyses beyond mean and median evaluations. Moreover, despite the differences, our objective was to study the participants’ performance in both PVT_std and PVT_SR and investigate if the novel mobile application-based method can detect deterioration of vigilance.