Development of a rating instrument to identify catatonia by virtual viewing of motor assessments

Abstract

We initially sought to assess the feasibility of virtual motor assessments of a boy with slow movements utilizing three rating tools suitable for children with catatonia [Diagnostic and Statistical Manual of Mental Disorders Fifth Edition (DSM-5)® (American Psychiatric Association, 2013) (Figure 1); Pediatric Catatonia Rating Scale (PCRS) (Benarous, et al., 2016) (Figure 2); Bush-Francis Catatonia Rating Scale (BFCRS) (Bush, et al., 1996) (Figure 3)]. On their own monitors in different locations, three investigators viewed an edited video of clinical assessments of a 16-year-old boy displayed by the coordinator. The edited video was shown to all raters together once before each rater independently completed rating of each tool. If an item could not be scored based on the clinical video, raters scored the item with an “X.” First, they rated the BFCRS (Bush, et al., 1996) (Figure 3), then the PCRS (Benarous, et al., 2016) (Figure 2), and finally, the DSM-5 (American Psychiatric Association, 2013) (Figure 1). After each scale was completed, the coordinator obtained the independent scores of all raters and conducted a consensus conference to attain agreement on the scores for each item. If the raters did not agree on a score, then the consensus score was “.” (period) to indicate lack of agreement (Kadubandi, et al., 2024, 2025a,b). We then sought to conduct a comprehensive review of all rating scales for catatonia to construct an optimal scale including the crucial items from extant scales. A recent systematic review (Hirjak, et al., 2024) identified two clinical criteria (American Psychiatric Association, 2013; ICD-11 for Mortality and Morbidity Statistics, 2024) and two scales (Bush, et al., 1996; Northoff, et al., 2001) as the most widely utilized instruments so those instruments were used to construct our optimal scale. Since we aimed to rate videos of a child, we also utilized the PCRS (Benarous, et al., 2016). We then constructed a spreadsheet with the key criteria for each scale and each of the scales identified with a column as follows: ICD11 (ICD-11 for Mortality and Morbidity Statistics, 2024), DSM5 (American Psychiatric Association, 2013), Bush-Francis Catatonia Rating Scale (BFCRS) (Bush, et al., 1996), Northoff Catatonia Rating Scale (NCRS) (Northoff, et al., 2001), and Pediatric Catatonia Rating Scale (PCRS) (Benarous, et al., 2016) (Table 1). Because many scales included many specific items, Table 1 was expanded to include all the detailed items of the five target rating procedures (American Psychiatric Association, 2013; Benarous, et al., 2016; Bush, et al., 1996; ICD-11 for Mortality and Morbidity Statistics, 2024; Northoff, et al., 2001) (Table 2). We then sought to construct a checklist of behaviors that could be identified on remote viewing of a video of a prior motor assessment of a patient. Since the patient could not be interviewed to determine subjective experiences, items were limited to objective movements and utterances. Since clinical motor assessment examinations were viewed on videotapes, specific durations for assessments and repetitious actions and utterances were omitted. The goal was to construct a checklist for items to identify as present with a 1 or absent with a 0 on viewing the videotape (Table 3). We then sought to organize the rating form in a manner that could be readily scored by raters viewing virtual motor assessments once (Table 4). We further revised the rating form by removing (A) items that could not be readily assessed by viewing a video of a motor assessment and (B) phrases that did not convey the crucial concepts that could be recognized on recorded motor assessments in standard English (Table 5). Further efforts to evaluate a proposed instrument to identify catatonia were the remote testing of the novel tool by a team of trained raters utilizing a process for remote scoring of videos shown to the team of raters once by a coordinator followed by independent ratings by each rater and then a consensus conference to attain agreement by discussion by all raters (Elshourbagy, et al., 2023a,b). We utilized the dictum to rate as present only items that are unequivocably present (Oldham and Francis, 2022). We strived to use terminology commonly used in the literature (Oldham and Francis, 2022). We sought to develop a protocol without a structured interview and examination of the participant suitable for general motor assessments (Table 6). We recruited a team of trained raters to independently score videos of people with and without evidence of the increased, decreased, and abnormal movements characteristic of catatonia utilizing the procedure under development. We welcomed domestic and international colleagues to participate in an organizational meeting to discuss the planned rating of videos of a male. To establish skill as a trained rater, all raters completed a one-hour online course intended to teach psychiatrists and psychiatry trainees how to improve their diagnosis of catatonia using the Bush-Francis Catatonia Rating Scale (https://redcap.urmc.rochester.edu/redcap/surveys/?s=LWNL3EC78Y) and registered for the 3-month follow up. All raters independently scored the pre-module and post-module tests without checking the answers until their scores were recorded to send to the organizing committee. All raters independently scored on blank BFCRS forms the videos for hypokinetic catatonia, parakinetic catatonia, and hyperkinetic catatonia, and Patients A and B, before viewing the answers and sent their original independent score sheets to the organizing committee before viewing the answers. The scores will be evaluated for presentation and publication. All raters participated in an online organizational training session. This session was recorded for subsequent viewing by all raters including those who were unable to attend the online session. Raters were trained to use a battery of instruments that have been used to assess people with catatonia and related conditions: Abnormal Involuntary Movement Scales (AIMS) (National Institute of Mental Health, 1988), BFCRS (Bush, et al., 1996), Catatonia Quick Screen (CQS) (Luccarelli, et al., 2024), Children’s Global Assessment Scale (CGAS) (Shaffer, et al., 1983, 1985) (Figure 4), Clinical Global Impressions (CGI) (Guy, 1976), DSM-5 (American Psychiatric Association, 2013), Hillside Akathisia Scale (HAS) (Fleischhacker, et al., 1989), ICD-11 (ICD-11 for Mortality and Morbidity Statistics, 2024), Timed Stereotypies Rating Scale (TSRS) (Brasic, 2003); and the new tool under development (Table 7). The PCRS (Benarous, et al., 2016) (Table 1) was not included because the key items were already included in the battery of instruments. A goal of the current project is to generate a tool including only objective items. A flaw of other tools, including NCRS (Northoff, et al., 2001) and the BFCRS (Bush, et al., 1996), is the need of the examiner to identify the motivation of the patient by observation of behavior. A further goal is to organize items in related categories. Therefore, “a persisting lack of eye contact (e.g., as though dejected or may be actively avoiding eye contact),” (Oldham and Francis, 2022, page 8), a concept that requires the examiner to know the motivation of the patient, is now categorized as staring fixed gaze in the new tool (Table 7), not withdrawal (Bush, et al., 1996). This step also the removes the association of two unrelated occurrences, the failure to consume food and water and the avoidance of eye contact (Bush, et al., 1996). The subjective thoughts of the patient cannot be determined by an examiner by observing a motor assessment. However, to score items included in catatonia scales, raters imagined what patients were experiencing by viewing the videos of the patients. Another goal of the current project is to eliminate items that cannot be objectively identified by observation of a motor assessment. Since temperature, blood pressure, pulse, respiratory rate, and diaphoresis (Bush, et al., 1996) cannot be determined by observation of a recorded motor assessment, those items are not included (Table 7). To include the key features of extant scales, the new scale was revised (Table 7). The team of trained raters met to score videos of a male. At the beginning of each session the coordinator instructed raters to begin scoring forms (American Psychiatric Association, 2013; Bush, et al., 1996; Guy, 1976; ICD-11 for Mortality and Morbidity Statistics, 2024; Luccarelli, et al., 2024; Shaffer, et al., 1983, 1985) including the new tool under development during the video and to check their scores after the video was displayed. The coordinator advised the team that the Abnormal Involuntary Movement Scale (AIMS) (National Institute of Mental Health, 1988), the Hillside Akathisia Scale (HAS) (Fleischhacker, et al., 1987), and the Timed Stereotypies Rating Scale (TSRS) (Brasic, 2003) would be administered in the middle of the video. The coordinator advised that he would warn the team a minute or so in advance to put the blank TSRS on their instrument to begin scoring. Videos of sequential sessions of a male were displayed by the coordinator to be viewed independently by each trained rater. Raters were asked to complete their scoring independently without consultation with others. After the raters completed and checked their independent score sheets, they sent them to the organizing committee for tabulation. The coordinator then conducted a consensus meeting with all raters to attain agreement on the score for each item if possible. The raters were told only the age and sex of the first patient for each session. Sessions were conducted twice weekly for three weeks. The patient was identified as a 17-year-10-month-19-day-old male for sess