Preliminary cross-sectional reliability and validity of the Skull Base Inventory (SBI) quality of life questionnaire
Journal of Otolaryngology - Head & Neck Surgery volume 45, Article number: 45 (2016)
The Skull Base Inventory (SBI) was developed to assess the quality of life of patients undergoing endoscopic or open approaches for anterior and central skull base pathologies. In this study, we sought to establish the discriminative and evaluative properties for this instrument.
The SBI was administered in a cross-sectional fashion to patients who previously had skull base surgery after treatment and then again 2 weeks after completing the instrument. Internal consistency, test-retest reliability, and construct validity were determined. Four constructs were evaluated with the following a priori hypotheses: lower scores will be seen in patients with 1.malignant versus benign histology, 2.a history of radiation versus none, and those with 3.recurrences versus no recurrence, and 4.items deemed relevant versus irrelevant by respondents.
Fifty-two patients completed the questionnaire; 32 had endoscopic and 20 open surgeries. Internal consistency was good (>0.7 and <0.95) for all domains except one. Test-retest reliability was good (>0.70) for 38 of 41 items. Four constructs were evaluated and three were consistent with a priori hypotheses (p < 0.05). The instrument failed to confirm the hypothesis that malignant tumours are associated with poorer scores than benign.
The SBI demonstrated preliminary reliability and validity for discriminative use.
Skull base tumours pose a significant challenge in surgical oncology due to their proximity to vital anatomical structures . In many instances, the primary management of these tumours is surgical resection. Given the challenging anatomical locations of such tumours, patients may experience significant morbidity secondary to the disease itself and/or the treatment [2–4]. With recent improvements in endoscopic and open surgical approaches, there has been an increasing interest in the impact of tumours and surgical resection on patients’ health-related quality of life (QOL) . QOL studies can help inform health care providers of the functional limitations that patients face, the obstacles to returning back to their pre-illness health state, and the magnitude of quality of life impairments by treatment approach.
To date, a number of QOL instruments for head and neck cancers have been developed [2, 6–8]. Very few have been designed to measure the QOL of patients with anterior and central skull base pathologies. The existing instruments are also limited in capturing the unique morbidities that result from different tumour pathologies, locations, and surgical approaches. The Skull Base Inventory (SBI) was designed to overcome the limitations of existing instruments .
The SBI is a multidimensional, disease-specific instrument designed to measure QOL of patients who undergo surgical treatment, regardless of surgical approach, for anterior or central skull base pathologies . The instrument has been designed using both expert and patient input to be applicable to those who undergo open or endoscopic surgical approaches. The SBI was designed for both discriminative and evaluative purposes, to capture the temporal change in QOL, and to differentiate between the QOL of different patient populations, which is of particular importance for assessing various surgical modalities. It consists of 41 questions covering 11 disease-specific domains including social, emotional, physical, cognitive, family, financial, spiritual, endocrine, nasal, neurologic, and visual. The comprehensive measurement properties of the SBI however, have not yet been evaluated. The purpose of the present study is to evaluate the psychometric properties of the SBI in a cross-sectional study. In specific, we sought to determine the instrument’s reliability and validity in capturing disease-specific domain and overall QOL of patients who have undergone skull base surgery.
This study was granted approval by the University Health Network’s Research Ethics Board. Informed consent was obtained from all included patients.
All patients who were surgically treated for anterior or central skull base pathologies between 1998 and 2008 at the University Health Network were screened as potential participants for this study. Patients were included if they had a minimum of 6 months from time of surgery, were English speaking, currently alive, not lost to follow-up, and were willing to consent to study participation. Patients were contacted by telephone, interviewed over the phone, and asked to complete the questionnaire by interview. This study was executed in a cross-sectional fashion with patients recruited between January and June 2009.
Instrument administration and chart review
Included patients were asked to complete the 41 item questionnaire by telephone and were subsequently asked to complete the questionnaire again 14 days after their initial completion in order to assess test-retest reliability. Although there is no evidence for the optimum time interval between test administration in test retest reliability of health status instruments, we chose 14 days to be long enough to minimize subject recall and carryover effect, while having the time period short enough to reduce the chance for change in health and/or mood status . In addition, patients were asked to rate the relevance of each item on a seven point Likert scale to their overall quality of life. The instrument scoring has previously been described . In brief, each domain is scored out of 100 with lower scores associated with poorer quality of life. Items within each domain for which a response was recorded were summed and averaged out of 100. A composite score assuming equivalent weighting of domains in overall quality of life was created by averaging domain scores. A retrospective chart review was conducted for patients who consented to participate in this study, and the following data were collected: patient factors (age, gender), disease factors (histopathology, tumour location, stage) and type of adjuvant/neoadjuvant therapies (radiotherapy, chemotherapy).
The consensus-based standards for the selection of health measurement instruments (COSMIN) guidelines for assessing the quality of studies on measurement properties of health measurement instruments was reviewed, and the following statistical analyses were performed to establish the discriminative and evaluative properties of the SBI . Internal consistency within domains was assessed using Cronbach’s alpha. Internal consistency was rated from unacceptable to acceptable as previously described . Item test-retest reliability was performed using intra-class correlation (ICC). Correlation coefficients of 0.7 or higher were considered good test-retest reliability. Validity testing was performed using construct validity. Four constructs with the following a priori hypotheses were tested: (1) patients with malignant pathologies have lower scores than those with benign pathologies (2) patients with a history of radiation have lower scores than those with no history of radiation, (3) patients with tumour recurrence will have lower scores than those with no recurrence, and (4) items deemed relevant by respondents will be associated with lower scores than those deemed irrelevant. All 41 questions were examined for floor and ceiling effects; floor and ceiling effects were considered when more than 20 % of participants chose the lowest or highest possible score, respectively. The minimally important clinical difference (MCID) was determined using distribution method (MCID = 0.5 standard deviation) . Probability (p) values ≤ 0.05 were considered statistically significant. All statistical analyses were performed using IBM SPSS 20.0.
Fifty-two patients completed the 41-question SBI. Patient, disease, and treatment factors are described in Table 1. The group of tumours consisted of both anterior and central skull base locations. Tumour histopathologies varied considerably, and included adenocarcinoma (5.8 %), adenoid cystic carcinoma (1.9 %), anterior skull base/olfactory groove meningioma (3.8 %), planum or tuberculum meningioma (5.8 %), cavernous hemangioma (3.8 %), hemangioma (1.9 %), hemangiopericytoma (1.9 %), chondrosarcoma (1.9 %), chordoma (17.3 %), craniopharyngioma (7.7 %), esthesioneuroblastoma (13.5 %), corticotropic pituitary adenoma (13.5 %), somatotropic pituitary adenoma (7.7 %), gonatotropic pituitary adenoma (1.9 %), lactotropic pituitary adenoma (3.8 %), juvenile nasopharyngeal angiofibroma (1.9 %), leiomyosarcoma (1.9 %), osteosarcoma (1.9 %), and squamous cell carcinoma (1.9 %).
Internal consistency of all domains of the SBI are presented in Table 2. The ‘other’ domain contains items generally felt to be unrelated and comprised a type of miscellaneous domain thus explaining the unacceptable rating for internal consistency. The spiritual domain also had an unacceptable rating for internal consistency.
Test-retest reliability for 38 out of 41 questions was good (≥0.7). Three items had a test-retest reliability less than 0.7, two of which were in the visual subdomain (ICC = 0.66, 0.66) and one of which was in the social domain (ICC = 0.42).
Using the SBI, four constructs were evaluated (Table 3). Three of the four constructs tested matched the a priori hypotheses. Patients with a history of radiotherapy had a worse overall QOL than those who did not receive radiotherapy (73.3 vs 82.0, p = 0.04) with statistically poorer QOL in family (79.0 vs 92.3, p = 0.03), financial (75.3 vs 91.1, p = 0.03) social (76.5 vs 87.2, p = 0.05), and nasal domains (54.5 vs 68.1, p = 0.05) (Fig. 1a). Patients who had recurrent tumours also had worse overall QOL than those who did not (70.0 vs 80.7, p = 0.03); with lower domains scores in the emotional (63.2 vs 79.1, p = 0.05), family (73.9 vs 90.2, p = 0.02), social (73.2 vs 85.2, p = 0.05), and neurologic (68.2 vs 80.4, p = 0.05) domains (Fig. 1b). The fourth construct examined demonstrated that items on the questionnaire that were deemed irrelevant by patients were associated with better scores than those that were deemed relevant (p < 0.001) (Fig. 1c). Contrary to the a priori hypothesis, patients with malignant tumours did not have a statistically significant difference in overall QOL compared to those with benign tumours (75.7 vs 80.3, p = 0.28) (Fig. 1d). The average time after surgery for completion of the SBI for those with benign disease was 47.7 months versus 112.7 months for those with malignant disease (p < 0.0001).
Floor and ceiling effects
No floor effects were observed for any of the items on the SBI. Ceilings effects (>20 %) were observed in the following 6 domains: emotional, family, financial, spiritual, endocrine, and visual (Table 4). The financial and family domains had ceiling responses in 56 % and 52 % of respondents, respectively. The MCID was determined to be 0.82.
Measuring QOL in patients with skull base neoplasms is challenging because of the variety of changes experienced as a result of the pathology as well as the treatment. The World Health Organization defines quality of life as ‘individuals’ perception of their position in life in the context of their culture and value systems in which they live in and in relation to their goals, expectations, standards, and concerns . It is a broad ranging concept affected in a complex way by the person’s physical health, psychological state, level of independence, social relationships, personal beliefs, and their relationship to salient features of their environment” . In addition to affecting ones physical functioning, treatment for skull base neoplasms affects several other domains of life . We designed an instrument which purports to measure changes in QOL for patients with skull base neoplasms. Herein we present the first psychometric assessment of the SBI and evaluation of the measurement properties of this instrument. The study incorporated responses from participants who had undergone skull base surgery at different time points with a wide range of histopathologies, anatomic locations (anterior and central skull base) and surgical approaches (endoscopic and open). Our results demonstrate that the SBI has sound reliability and validity in capturing QOL in disease specific domains and overall QOL in patients who undergo skull base surgery, regardless of when the instrument was completed with respect to the operation. The only domain that had an unacceptable rating for internal consistency was the spiritual domain. In the SBI, there are two questions within the spiritual domain: (1) In the past 2 weeks, has your condition made you question your religious beliefs? and (2) How would you rate your ability to appreciate the small things in life. These two questions may or may not be of the same measure and this highly depends on one’s personal belief system, thoughts, opinions, and perceptions of the world around them; therefore explaining the lack of internal consistency observed within this domain. In the present study, we did not have sufficient sample size to perform factor analysis. In future studies we will perform factor analysis to determine if items load well onto each domain to confirm the items belong in the proposed domains.
We tested construct validity through four a priori hypotheses. Three of these hypotheses were confirmed: (1) patients with a history of radiation have a worse QOL, (2) patients with primary cancer recurrence have a worse QOL, and (3) questions on the SBI that have little relevance to patients disease state are consistently scored high. However, results obtained rejected our hypothesis of patients with malignant tumours having worse QOL than those with tumours of benign pathology. Failure to verify this construct may be due either to the fact that there indeed is no difference in QOL between patients with malignant or benign tumours in our cohort, there is a difference but the sample size was inadequate to detect a difference, or there is a difference but the instrument was not sensitive enough to detect this difference. The timing after surgery at which the questionnaire was completed may also explain the failure to detect this difference. Previous studies have shown a difference in QOL between malignant and benign skull base tumours at 6 months after surgery . However, the QOL impairment shortly after surgery, seems to improve over time. In our cohort, patients completed questionnaires generally much later after their surgery. As such it may be that with time there is a normalization of QOL that makes patients with benign and malignant pathologies have similar QOL.
In the last two decades, anterior skull base surgery has evolved rather rapidly owing to improved surgical techniques [16, 17]. Advancements have led to reduced mortality and morbidity. With the advent of endoscopic approaches, comparisons have been made with open approaches in terms of the extent of surgical resection, survival, and now, QOL [15–17]. In general, most studies focusing on QOL have found that endoscopic approaches allow for earlier improvements in QOL following surgery [18, 19]. Nevertheless, open approaches continue to be necessary for tumours that are not amenable to endoscopic resection and depending on surgeon preference.
There are a number of challenges in measuring the QOL of patients with skull base pathologies. Skull base tumours are situated within the vicinity of vital neurovascular structures that play significant roles in different sensory organs and cognition. Endoscopic and open surgical approaches also result in different post-operative morbidities. We have shown that the symptom profiles of different tumours vary considerably owing to variation in size, morphology, location, pathology, and the surgical technique used for resection . Although it is ideal to have different QOL instruments for different tumour pathologies and locations, this is simply not feasible due to the vast number of possible combination of tumours types and locations. It is therefore important to have a site-specific instrument that can accurately and holistically capture sensory, physical, and cognitive symptoms following skull base surgery, and be applicable to both endoscopic and open surgical approaches for different tumour pathologies.
The Anterior Skull Base questionnaire (ASB) is an existing instrument for measuring QOL after skull base surgery. It is a validated and reliable tool that has six domains [2, 21]. The ASB has been shown to be able to predict postoperative QOL after skull base surgery . However, the ASB was originally developed for open skull base surgical patients. It consists of only seven disease-specific items that may limit its discriminative ability for different populations. In contrast, the SBI consists of 26 disease specific items along with five physical subdomains and one cognitive domain that may increase its discriminative ability; this must be tested in future comparative studies.
Based on the COSMIN guidelines for assessing the methodological quality of studies on measurements properties of health status instrument, a comprehensive instrument must consist of three domains: reliability, validity, and responsiveness . In this cross-sectional study, we were limited in only assessing reliability and validity since our study population comprised of patients who had already undergone surgery and were measured in a cross-sectional fashion. Additional multi-institutional prospective studies are required to assess the responsiveness of the instrument by capturing the QOL of patients before and after surgery, and evaluating the change in QOL over time. Furthermore, a number of aspects of validity were not evaluated, such as cross-cultural validity, which is critical in QOL studies as perception of varying health status’ can differ considerably for patients of different socio-cultural backgrounds. Validating such properties of the instrument require additional studies with different patient populations and at different institutions.
Our preliminary data suggests that the SBI has sound reliability and validity in assessing disease-specific and overall QOL of patients who undergo skull base surgery for anterior and central pathologies with both endoscopic and open surgical approaches. Additional studies are needed to further validate the psychometric and evaluative properties of the instrument in different patient populations, and of different institutions.
Ganly I, Patel SG, Singh B, Kraus DH, Bridger PG, Cantu G, Cheesman A, De Sa G, Donald P, Fliss D, Gullane P, Janecka I, Kamata SE, Kowalski LP, Levine P, Medina LR, Pradhan S, Schramm V, Snyderman C, Wei WI, Shah JP. Complications of craniofacial resection for malignant tumors of the skull base: report of an International Collaborative Study. Head Neck. 2005;27(6):445–51.
Gil Z, Abergel A, Spektor S, Cohen JT, Khafif A, Shabtai E, Fliss DM. Quality of life following surgery for anterior skull base tumors. Arch Otolaryngol Head Neck Surg. 2003;129(12):1303–9.
Woertgen C, Rothoerl RD, Hosemann W, Strutz J. Quality of life following surgery for malignancies of the anterior skull base. Skull Base. 2007;17(2):119–23.
Gil Z, Fliss DM. Quality of life in patients with skull base tumors: current status and future challenges. Skull Base. 2010;20(1):11–8.
Gil Z, Patel SG, Bilsky M, Shah JP, Kraus DH. Complications after craniofacial resection for malignant tumors: are complication trends changing? Otolaryngol Head Neck Surg. 2009;140(2):218–23.
Gil Z, Abergel A, Spektor S, Shabtai E, Khafif A, Fliss DM. Development of a cancer-specific anterior skull base quality-of-life questionnaire. J Neurosurg. 2004;100(5):813–9.
Weymuller EA, Alsarraf R, Yueh B, Deleyiannis FW, Coltera MD. Analysis of the performance characteristics of the UW QOL and modification of the instrument. Arch Otolaryngol. 2001;127:489–93.
Bjordal K, Ahlner-Elmqvist M, Tollesson E, Jensen AB, Razavi D, Maher EJ, Kaasa S. Development of a European Organization for Research and Treatment of Cancer questionnaire module to be used in quality of life assessments in head and neck cancer patients. Acta Oncol. 1994;33:879–85.
de Almeida JR, Vescan AD, Gullane PJ, Gentili F, Lee JM, Lohfeld L, Ringash J, Thoma A, Witterick IJ. Development of a disease-specific quality-of-life questionnaire for anterior and central skull base pathology - the skull base inventory. Laryngoscope. 2012;122(9):1933–42.
Marx RG, Menezes A, Horovitz L, Jones EC, Warren RF. A comparison of two time intervals for test-retest reliability of health status instruments. J Clin Epidemiol. 2003;58(8):730–5.
Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HC. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49.
George D, Mallery P. SPSS for Windows step by step: A simple guide and reference. 11.0 update. 4. Boston: Allyn & Bacon; 2003.
Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care. 2003;41(5):582–92.
World Health Organization Quality of Life Working Group. Rehabilitation after cardiovascular diseases, with special emphasis on developing countries. Report of a WHO Expert Committee. World Health Organ Tech Rep Ser 1993;831:1–122.
Abergel A, Fliss DM, Margalit N, Gil Z. A prospective evaluation of short-term health-related quality of life in patients undergoing anterior skull base surgery. Skull Base. 2010;20(1):27–33.
Fletcher AM, Marentette L. Anterior skull-base surgery: current opinion. Curr Opin Otolaryngol Head Neck Surg. 2014;22(4):322–5.
Komotar RJ, Starke RM, Raper DM, Anand VK, Schwartz TH. Endoscopic endonasal compared with anterior craniofacial and combined cranionasal resection of esthesioneuroblastomas. World Neurosurg. 2013;80(1):148–59.
Pant H, Bhatki AM, Snyderman CH, Vescan AD, Carrau RL, Gardner P, Prevedello D, Kassam AB. Quality of life following endonasal skull base surgery. Skull Base. 2010;30:35–40.
Kirkman MA, Borg A, Al-Mousa A, Haliasos N, Choi D. Quality-of-life after anterior skull base surgery: a systematic review. J Neurol Surg B Skull Base. 2014;75(2):73–89.
de Almeida JR, Witterick IJ, Gullane PJ, Gentili F, Lohfeld L, Ringash J, Thoma A, Vescan AD. Physical morbidity by surgical approach and tumor location in skull base surgery. Head Neck. 2013;35(4):493–9.
Abergel A, Cavel O, Margalit N, Fliss DM, Gil Z. Comparison of quality of life after transnasal endoscopic vs open skull base tumor resection. Arch Otolaryngol Head Neck Surg. 2012;138(2):142–7.
Fliss DM, Abergel A, Cavel O, Margalit N, Gil Z. Combined subcranial approaches for excision of complex anterior skull base tumors. Arch Otolaryngol Head Neck Surg. 2007;133(9):888–96.
This work was funded by a grant from the Physicians Services Incorporated (PSI) and in part by an educational grant from the Latner Family in conjunction with Samuel Lunenfeld Research Institute at Mount Sinai Hospital, Toronto, Canada. The authors have no other funding or financial relationships to disclose.
Availability of data and materials
The authors confirm that all data underlying the findings are fully available without restriction. The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
Conception and design: JA, AV, IW. Acquisition of data: JA. Analysis and interpretation of data: SL, EM, DG, JA. Drafting the article: SL, EM, DG, JA. Critically revising the article: SL, EM, IW, DG, JA, GZ, FG, IW. Reviewed submitted version of manuscript: all authors. Statistical analysis: SL, EM, DG, JA. Study supervision: IW, AV, JA. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
All data has been analyzed as a whole and there are no individual participant’s data being published.
Ethics approval and consent to participate
This study has obtained appropriate ethics approval from the University Health Network Research Ethics Board to conduct this study. All participants have signed an informed consent form to agree to participate in this study.
About this article
Cite this article
Larjani, S., Monteiro, E., Witterick, I. et al. Preliminary cross-sectional reliability and validity of the Skull Base Inventory (SBI) quality of life questionnaire. J of Otolaryngol - Head & Neck Surg 45, 45 (2016). https://doi.org/10.1186/s40463-016-0158-y