Original Article

Evaluation of the Accuracy of Risk of Malignancy Index in Distinguishing Benign and Malignant Ovarian Tumours in a Nigerian Population: A Research Conducted at Federal Teaching Hospital Katsina Nigeria

 ABDUL Asma’u Eleojo¹, ADAMOU Natalia², YAKASAI Ibrahim Adamu², NURA Abdulkarim¹

1FTH KATSINA 

2AKTH KANO

Corresponding Author:

Abdul Asma’u Eleojo

FTH KATSINA 

ojoabdul1@gmail.com

08065567547

ABSTRACT

Background: Ovarian cancer is one of the deadliest gynaecological cancers. Preoperative differentiation between benign and malignant ovarian masses in women can be problematic with no test or algorithm being superior in terms of accuracy. The risk of malignancy index (RMI) combines three pre-surgical features: serum carcinoma antigen 125, menopausal status (M) and ultrasound score (U). The RMI is a product of the ultrasound score, the menopausal status and the serum CA125 level IU/ml. Objective: To determine if RMI can accurately differentiate benign from malignant ovarian masses. Research Methods: The study was a descriptive cross-sectional study carried out at Federal Teaching Hospital Katsina between April and November 2018. Sixty women from the gynaecological unit scheduled for exploratory laparotomy who met the inclusion criteria were recruited after obtaining their consent. RMI was calculated and the results obtained were compared with the final histopathologic diagnosis. Statistical analysis was done using SPSS 20 and the level of significance was <0.05%. Results: Risk of malignancy index 1,2,3,4 were able to accurately distinguish between benign and malignant ovarian tumours in the women, the sensitivity was 80.00%, specificity 100.00%, positive predictive value 100.00%, negative predictive value 87.50% and accuracy 91.70%. All four RMIs performed equally however RMI 4 had the greatest area under the curve. Conclusion: The study validated RMI as being able to accurately distinguish benign and malignant ovarian tumours in the population studied. RMI should be used in the preoperative evaluation of women with adnexal masses in Katsina, 

Keywords: CA 125, Perimenopause, Risk of Malignancy Index.

INTRODUCTION

Ovarian cancer is the leading cause of mortality among gynaecological malignancies1. Survival from ovarian cancer depends on the stage, histological type, and optimal cytoreductive surgery.¹ The most important factor for survival is the stage at diagnosis, therefore, attempts have been made to develop a screening method, which by detecting ovarian cancer at an early stage has the potential to decrease deaths. ¹

Ovarian cancer has few, nonspecific or no symptoms in its early stage when it is most curable; hence screening for ovarian cancer is of high clinical interest. ^2,3 The purpose of screening is to differentiate benign from malignant masses as well as diagnose ovarian cancer at an early curable stage.^2,3 Several diagnostic methods for pelvic masses have been reported such as pelvic examination, CA 125 tumour marker level, and ultrasonography.¹ However, none of these methods used individually has shown significantly better performance in detecting malignant ovarian tumours. ¹ In gynaecological malignancies; tumour markers have a crucial role in screening, monitoring treatment, follow up and also predicting the recurrence of the disease. ^4,5

Various methods of evaluating ovarian cancer risk have been proposed.⁴ Ultrasound or cancer antigen 125 assay when used alone has many limitations while combined methods give more sensitive and accurate results. [4] The development of a mathematical formula using a logistic model, incorporating menopausal status, the serum level of a glycoprotein called CA-125 and the ultrasound score has been described in the form of different malignancy indexes. These indices were calculated using a simplified regression equation obtained from the product of the ultrasound finding score, the menopausal status score and the absolute serum level of CA- 125.⁵

In 1990, Jacobs et al.⁶ initially developed the risk of malignancy index as a validated clinical tool that can be used for risk stratification in ovarian lesions and to guide further management. The risk of malignancy index (RMI 1) combines three pre-surgical features: serum CA-125, menopausal status (M) and ultrasound score (U). The RMI is a product of the ultrasound score, the menopausal status and the serum CA125 level IU/ml.⁶

RMI= (U) x (M) x (CA125)

The ultrasound feature is scored 1 point for each of the following characteristics: multilocular cyst, solid area, metastasis, ascites and bilateral lesion. U =0 (for an ultrasound score of 0), U =1 (for an ultrasound score of 1), and U =3 (for an ultrasound score of 2-5).

The menopausal status is scored as 1 = pre-menopausal and 3 = post-menopausal. The classification of "postmenopausal" is a woman who has had no menstrual period for more than 1 year or a woman over 50 years who has had a hysterectomy. 

Perimenopause is the period around the onset of menopause. Serum CA125 is measured in IU/ml and can range from (0-35 IU/ml) in normal women.

Using a cut-off value of 200 for RMI 1, an 85% sensitivity and 97% specificity were obtained.⁶ Certain modifications of the risk of malignancy index model have been proposed which are specified as the risk of malignancy index 2 and risk of malignancy index 3. ³ The difference between the three indexes lies in the different scoring of ultrasound finding and menopausal status. The main advantage of all four RMI is that they are simple scoring systems that can be applied directly into clinical practice without the introduction of expensive or complicated methods. ²

RMI 2 was developed in 1996 and then RMI 3 in 1999. Because the average size of a malignant tumour was significantly greater than that of a benign tumour, tumour size (S) was added in 2008 to the risk of malignancy index, thus improving accuracy and creating RMI 4 ³. Cut off value of 200 for RMI 1 2 and 3 and 450 for RMI 4 showed the best discrimination between benign and malignant pelvic masses with a high level of sensitivity and specificity. ³

All risks of malignancy index versions are comparable in clinical use. Moreover, the risk of malignancy index 1 is considered a reference test for any new diagnostic model that is developed. ³.

RMI 2= U x M x CA 125, where a total ultrasound index 0/1 denotes U=1, ultrasound index > or= 2 denotes U= 4, premenopausal status denotes M =1 and postmenopausal M =4.

RMI 3= U x M x CA-125, where a total ultrasound index 0/1 made U=1, ultrasound index > or =2 denotes U=3, premenopausal status denotes M=1 and postmenopausal status M =3.

The fourth version, RMI 4, includes the tumour size S. RMI 4 = U x M x S x CA-125, where a total ultrasound index 0/1 denotes U=1, ultrasound index > or = 2 denotes U=4, premenopausal status M =1 and postmenopausal status M =4. A tumour diameter < 7cm denotes S=1, and > or= 7cm denotes S =2. ^3,7

Clinical/Public Health Context

A woman's lifetime risk of developing invasive ovarian cancer is 1 in 75 and the risk of dying from invasive ovarian cancer is 1 in 100. ⁸ In Kano state, Nigeria, ovarian cancer accounted for 33.33% of gynaecological cancers observed between April 2014 and April 2015. ^9,10

Justification

Mean survival time for women with ovarian malignancy is significantly improved when managed within a specialized gynaecological oncology unit.¹¹ Unnecessary surgery or overly radical intervention with the abrupt loss of childbearing potential in young women are all significant risks to patients with cysts that are inappropriately characterized, whilst the consequence of failing to recognize a cyst as malignant significantly impacts prognosis.¹¹ Although RMI has been widely evaluated and employed in developed countries, its utility in risk prediction in developing countries is currently unknown it is apt to evaluate the accuracy of RMI in predicting the risk of malignant ovarian tumours in women preoperatively.

Aim, Objectives and Research Question

Aim

To evaluate the utility of the risk of malignancy index in differentiating between benign and malignant ovarian masses in a Nigerian population.

Objectives

1) To validate RMI in distinguishing benign and malignant ovarian tumours in a Nigerian population

2) To compare the performance of the four different RMIs in distinguishing benign and malignant ovarian masses in a Nigerian population.

3) To determine the predictive value of RMI in distinguishing between benign and malignant ovarian tumours in perimenopausal women. 

4) Based on objectives 1, 2, and 3 above to make recommendations for routine calculation of RMI in women with ovarian masses.

Research Question: 

Can RMI accurately differentiate benign from malignant ovarian tumours in a population of Nigerian women?

Hypothesis

Null hypothesis (Ho): The risk of malignancy index RMI cannot distinguish benign from malignant ovarian tumours in a population of Nigerian women

The alternate hypothesis (H1): The risk of malignancy index can distinguish benign from malignant ovarian tumours in Nigerian women

RESEARCH METHODS

Study Design: It was a descriptive cross-sectional study

Sampling Approach: The study population comprised women with an ovarian cyst or adnexal mass being worked up for surgery. 

An adnexal mass was defined as a simple, complex, cystic or solid mass of the ovary or surrounding connective tissues as determined by pelvic examination or ultrasound scan. Perimenopausal women were defined as women between the ages of forty and fifty years.

Inclusion Criteria: All consenting women with an ovarian mass based on ultrasound finding of ovarian cyst or adnexal mass and being worked up for exploratory laparotomy. Patients with clinically palpable adnexal masses

Exclusion Criteria: Women diagnosed with any other intra-abdominal mass or disease associated with elevation of CA125 such as liver cirrhosis, endometriosis, adenomyosis, pelvic inflammatory disease, pancreatitis and advanced intra-abdominal non-ovarian malignancy.

• Women with a history of ovarian cancer
• Women who have had bilateral oophorectomy
• Women who are currently known to be pregnant

Sample Size Determination: The sample size was determined using Taylor's formula¹² and calculated to be 60 patients

Sample Recruitment Method: The purpose of the study was explained to the women. Written consent was obtained from those willing to participate in the study. A pretested structured, interviewer-administered questionnaire was used to obtain information about age, parity, menopausal status, educational status, past medical history, past obstetric and gynaecological history as well as family history. The questionnaires were administered by the researcher aided by trained research assistants. A systemic examination was done by the researcher. 

A questionnaire administration and systemic examination were performed in the gynaecology clinic and in the gynaecology ward for those women that were recruited from the gynecologic emergency unit 

Sampling Technique: Convenience and purposive sampling were used. Blinding was done to reduce bias. The chemical pathologist and histopathologist were blinded. The serum CA125 samples were coded before they were taken for analysis and the specimens sent for histology were also coded. 

Ethical Consideration: Ethical approval for the study was obtained from the research and ethical committee of the hospital and it has been attached. Informed written consent was sought from the patients and their confidentiality was respected. All participants were informed about their right to refuse to participate in the study and were free to opt-out of the study at any point if they so wished. The provisions of the Helsinki Declaration of 1975 on the investigation of human subjects were adhered to. 

Calculation of Risk of Malignancy Index

Ultrasound Score: Included women had a trans-abdominal ultrasound scan using a Mindray DC7 ultrasound device with a 5 MHz convex abdominal probe. The characteristic appearance of masses (bilaterality, multilocularity, solid area, ascites, intra-abdominal metastasis) were recorded, and the presence of any of the above features was allotted 1 while the absence was allotted 0. 

Measurement Of Ca 125 Level: Five ml of venous blood was collected from each of the participants in the study for serum CA125 estimation within thirty days before their surgical procedures. The biochemical analysis for serum CA125 level was done using ARCHITECT CA125 II assay. 

Determination Of Menopausal Score: Menopause was defined as one or more years of amenorrhea or women who were over 50 years that had undergone a hysterectomy. The menopausal score was assigned as M=1 and if postmenopausal, M=3 for RMI 1 and RMI 3 or M =4 for RMI 2 and RMI 4.

The risk of malignancy index 1,2,3,4 was calculated: RMI 1 = U x M x CA125

The ultrasound feature is scored 1 point for each of the following characteristics: (multilocular cysts, solid areas, metastasis, ascites and bilateral lesions). U =0 (for an ultrasound score of 0), U =1 (for an ultrasound score of 1), and U =3 (for an ultrasound score of 2-5).

• The menopausal status is scored as 1 = pre-menopausal and 3 = post-menopausal
• RMI 2= U x M x CA 125, where a total ultrasound score of 0 or 1 denotes U=1, ultrasound score > or= 2 denotes U= 4, premenopausal status denotes M =1 and postmenopausal M =4.
• RMI 3= U x M x CA-125, where a total ultrasound score of 0 or 1 made U=1, ultrasound score > or =2 denotes U=3, premenopausal status denotes M=1 and postmenopausal status M =3. 
• RMI 4 = U x M x S x CA-125, where a total ultrasound index 0/1 denotes U=1, ultrasound index > or = 2 denotes U=4, premenopausal status M =1 and postmenopausal status M =4. A tumour diameter < 7cm denotes S=1, and > or= 7 cm denotes S =2. [3,7]

Post-operatively, each specimen was sent for histopathologic review to determine if it was a benign or malignant ovarian tumour.

The researcher then compared the RMI calculated for each woman with the histopathologic result. The primary endpoint of the study was to determine if the risk of malignancy index can accurately predict ovarian malignancy

RESULTS 

A total of sixty women who met the inclusion criteria were recruited for the study between April and November 2018. Thirty-five women (58.33%) had benign ovarian tumours while twenty-five women (41.67%) had malignant ovarian tumours, 51.43% of the benign ovarian tumours were follicular cysts, mature teratomas and serous cyst adenomas constituted 40.00 and 8.57 % respectively. 

Table 1: Socio-Demographic Characteristic of Patients

  * = p value significant 

The predominant malignant tumour was papillary cyst adenocarcinoma 36.00%, and the least seen was endometrioid adenocarcinoma 8.00%. Other malignant tumours included mucinous cyst adenocarcinoma 20.00% and yolk sac tumour 12.00%. Most of the tumours in the premenopausal group (31.00%) were benign unlike in the postmenopausal group where most of the tumours (15.00%) were malignant. 

Table 2:  Gynaecological Characteristics and components of RMI of Patients 

 *= p value significant

Table 3: Validity of RMI in detecting Ovarian Malignancy in all women

Se = Sensitivity; Sp = Specificity; Ac = Accuracy

A significant association was found between menopausal status and ovarian malignancy; the level of significance was found to be 0.023%. there was no significant association between parity and ovarian malignancy, the level of significance was 0.433%. The sensitivity. specificity, positive predictive value, negative predictive value and accuracy of risk of malignancy indexes were found to be 80.00%, 100.00%, 87.50% and 100.00% respectively. The accuracy of the test was 91.70%. 

Fig. 1 Receiver operator characteristic curve for RMI 1, Area under the curve is 0.929

Figure 2: Receiver operator characteristic curve for RMI 2, AUC 0.910

Figure 3: Receiver operator characteristic curve for RMI 3, Area under the curve is 0.911

The four risks of malignancy indexes performed equally in distinguishing benign from malignant ovarian tumours. All of the RMI did so at a level of significance of 0.000%. The four risks of malignancy indexes were compared using a Cochran test which showed that all the risk of malignancy indexes can distinguish benign from malignant ovarian tumours with a significant difference in their performance. Using receiver operator characteristic curves generated, the four RMIs had different areas under the curve. RMI 1 had an AUC of 0.929, RMI 2 had an AUC of 0.910, RMI 3 had an AUC of 0.911, and RMI 4 had the greatest area under the curve of 0.931, hence it had the best performance. 

Figure 4: Receiver operator characteristic curve for RMI 4, Area under the curve is 0.931

Figure 5: Receiver operator characteristic curve for RMI 4 in perimenopausal women, Area under the curve is 0.904

Table 4: Performance of the various RMI

*=p value significant; Test used = Cochran's test

Table 5: Validity of RMI in detecting Ovarian Malignancy in Perimenopausal group

Se = Sensitivity; Sp = Specificity; Ac = Accuracy

In premenopausal women, the risk of malignancy index was able to differentiate benign from malignant ovarian tumours. The test showed a sensitivity of 90.00%, specificity of 80.00%, negative and positive predictive values of 75.00% and 92.00% respectively and an accuracy of 84.00%. RMI 4 had an area under the curve of 0.904 hence it performed better.

DISCUSSION.

In this study, the risk of malignancy index was able to distinguish benign and malignant ovarian tumours in the Nigerian population that was studied as demonstrated by a significant association. Many other documented studies all had this outcome of RMI being able to differentiate benign from malignant ovarian tumours.^1,4,14,16,18 The sensitivity, specificity, predictive value of positive, the predictive value of negative and diagnostic accuracy (the per cent of accurately diagnosed patients divided by all of the patients) were determined.¹³ In this study the sensitivity of all four risks of malignancy indexes was found to be 80.00%, specificity was 100.00%, negative predictive value 87.50% and positive predictive value 100.00% respectively. The accuracy of the test was 91.70% this can be compared with other studies which had sensitivities ranging from 80-91% and specificities ranging from 85-93%. ^1,16,18,25

This study demonstrates the ability of RMI to correctly identify benign and malignant adnexal masses. Several other documented studies have validated the risk of malignancy index as a predictor of malignancy.^1,4,14-19 although the cutoff was varied in some of the studies to get the best performance of RMI. ^1,14 In a prospective study to evaluate the risk malignancy index and its diagnostic implication in patients with suspected ovarian mass, the risk of malignancy index compared with individual parameters of Ultrasound score, CA-125 or menopausal score at a cut-off point of 236 shows very high sensitivity (72.50%), specificity (98.20%), positive predictive value (98.10%), negative predictive value (74.70%) and diagnostic accuracy (84.13%) for discriminating malignant and benign pelvic masses. [20] The high specificity recorded in this study is similar to our study which was 100.00% however in this study the cutoff RMI used for differentiating benign and malignant masses was 236 unlike in our study in which the cutoff was 200

In this study, the four risks of malignancy indexes had equal performance in distinguishing benign from malignant ovarian tumours. They had the same sensitivity, specificity, positive and negative predictive value in all the women. In one retrospective study ²¹, two hundred and forty-seven women with adnexal masses were included, they had a mean age of 58.09 years (range 19–95 years). RMI 1 had the greatest area under the ROC curve of 0.89. The area under the ROC curve for RMI 2 was 0.844 and for RMI 3 was 0.842. Using a threshold of 200, RMI 1 had a sensitivity of 66% and a specificity of 91%; RMI 2 had a sensitivity of 74% and a specificity of 79%; and RMI 3 had a sensitivity of 68% and a specificity of 85%. Using an RMI cutoff level of 250, the sensitivity of RMI 1 dropped to 60% while the specificity increased to 94%. The sensitivity of RMI 2 also dropped to 71% and the specificity was 84%. At a threshold of 250, RMI 3 also recorded a drop in sensitivity to 61% and the specificity increased to 90%.²¹ A lower threshold of 120 for RMI 1, however, revealed a sensitivity of 74% and specificity of 84%. RMI 2 is a better malignancy predictor than RMI 3. ²¹ In another study three RMIs were checked without considerable difference in calculated parameters and in all RMIs the best cutoff point was at 200. ²²

To further test the performance of the four RMIs in our study, receiver operator characteristic curves were generated at the cutoff of 200 for RMI 1,2,3 and cutoff of 450 for RMI 4, using sensitivity and specificity. The receiver operator characteristic curves were generated for the various RMIs and all had an AUC of >0.90 in the women which are interpreted as an excellent performance of the models.²³ The receiver operator characteristic curves drawn showed the greatest area under the curve 0.931 was observed for RMI 4, based on this it can be said that in this particular study despite having the same sensitivity, specificity, PPV, NPV and accuracy, the RMI 4 having the greatest AUC of 0.931 performed better at distinguishing benign and malignant ovarian tumour. 

In one study, RMI 2 showed the best performance in predicting malignancy, compared with the other three indexes. At the cutoff point 90 above which the probability of malignancy of masses was high, RMI 2 had the most area under the curve 0.08, showing the greatest concordance with pathologic results.¹¹ Several other studies have proved RMI 2 to be a better predictor of malignancy. ^24-28

In this study, the various RMI was also compared in perimenopausal women, with a sensitivity of 90.00%, specificity of 80.00%, negative and positive predictive values of 75.00% and 92.00% respectively and an accuracy of 84.00% was found. In this study, the mean age of occurrence of the malignant ovarian tumour was 49.60 years, which is in the perimenopausal years. It was also observed that the ratio of benign to malignant masses was almost equal in this group of women after which the incidence of malignant masses rose and with further increase in age (61-70 years) no benign masses were seen. This implies that the perimenopausal group of women were important in the age-related distribution of malignant ovarian masses. 

CONCLUSION

The risk of malignancy index has been validated as a reliable method for distinguishing benign and malignant ovarian tumours preoperatively in women including perimenopausal women. 

The four risks of malignancy indexes were all able to distinguish benign and malignant ovarian tumours in the population of Nigerian women that were studied, they performed equally however RMI 4 had the greatest area under the curve of 0.931 in women generally and 0.904 in perimenopausal women. 

Recommendations

1. This is the first study in the geographical area to evaluate the accuracy of RMI in distinguishing benign and malignant ovarian tumours. RMI is a simple and cheap scoring system that should be adopted across secondary health centres in referring patients to tertiary or specialist
   centres for timely intervention.

1. A future research with a larger sample size is recommended targeting only perimenopausal women and based on the results, a screening test using RMI could be proposed for all women in this age group.

Strengths And Limitations:

Strengths of the Study: Blinding was utilized during the process of data collection and analysis, the histopathologist that reported the specimens postoperatively and the chemical pathologist that analyzed the serum CA125 were blinded and had no contact whatsoever. 

Limitations of the Study: CA125 assay is usually performed after pooling about ten samples in each batch, this caused a delay in the dissemination of the result to the participant and her managing team 

REFERENCES

1. 1.Vasudevan J, Nair V, Sukumaran S. Evaluation of risk of malignancy index in the preoperative assessment of ovarian tumors: Study from a tertiary care center. Saudi J Heal Sci. Available from: http://www.saudijhealthsci.org/text.asp?2016/5/2/67/193000
2. Kumari N, Gupta V, Kumari R, Makhija A. Evaluation of risk of malignancy index as a diagnostic tool in cases with adnexal mass. Int J Reprod Contracept Obs Gynecol. 2016; 5(6):1857–61. 
3. Stukan. Diagnostic Indices for Discriminating Benign from Malignant Ovarian Masses. J    Ultrasound Med. 2015; 34:207–17
4. 4.Javdekar R, Maitra N. Risk of Malignancy Index (RMI) in Evaluation of Adnexal Mass. J Obs Gynecol India. 2015; 65(2):117–21. 
5. Arun-Muthuvel V, Jaya V. Pre-operative evaluation of ovarian tumors by risk of malignancy index, CA125 and ultrasound. Asian Pac J Cancer Prev [Internet]. 2014; 15(6):2929–32. Available from: httsp://www.ncbi.nlm.nih.gov/pubmed/24761927
6. Jacobs I, Oram D Farbanks J, Turner J, Frost C, 
   Grudznskas JG. A risk of malignancy index incorporating 
   CA-125, ultrasound and menopausal status for the 
   accurate preoperative diagnosis of ovarian cancer. Br J 
   obstet Gynaeco 1990;97:922-9
7. Hemeda HM, Ali KK, Onsil KA. The role of risk of malignancy index in the preoperative assessment of patients with adnexal masses. Life Sci J. 2014; 11(1):241–6. 
8. Surveillance Epidemiology and End Results (SEER) Program. Research Data 1973-2014. National Cancer Institute, April 2017
9. Adamou N, Umar UA. Delayed Presentation of Patients with Gynecological Malignancies in Kano, North-Western Nigeria. Open J Obstet Gynecol [Internet]. 2015; 05(06):333–40. Available from: http://www.scirp.org/journal/doi.aspx?DOI=10.4236/ojog.2015.56048
10. 10.Yakasai IA, Ugwa EA, Otubu J. Gynecological malignancies in Aminu Kano Teaching Hospital Kano: A 3 year review. Niger J Clin Pr. 2013; 16(1):63–6.
11. 11.Karimi-Zarchi M, Paymani Mojaver S, Rouhi M, Hekmatimoghaddam SH, Nafisi Moghaddam R, Yazdian-Anari P, et al. Diagnostic Value of the Risk of Malignancy Index (RMI) for Detection of Pelvic Malignancies Compared with Pathology. Electron physician [Internet]. 2015; 7(7):1505–10.http://www.ephysician.ir/index.php/browse-issues/2015/7/253-1505
12. 12.Singha P, Biostatistics, Third edition by Ghadza prints Page 216
13. 13.Zweig MH CG. Receiver-operating characteristic (ROC) plots: a funda- mental evaluation tool in clinical medicine. Clin Chem. 1993; 39 (4):561–77.
14. 14.Ulusoy S, Akbayir O, Numanoglu C, Ulusoy N, Odabas E, Gulkilik A. The risk of malignancy index in discrimination of adnexal masses. Int J Gynecol Obstet. 2007; 96(3):186–91.
15. Yamamoto Y, Yamada R, Oguri H, Maeda N FT. Comparison of four malignancy risk indices in the preoperative evaluation of patients with pelvic masses. . Eur J Obstet Gynecol Reprod Biol. 2009; 144(2):163–7.
16. Van den Akker PAJ, Aalders AL, Snijders MPLM, Kluivers KB, Samlal RAK, Vollebergh JHA, et al. Evaluation of the risk of malignancy index in daily clinical management of adnexal masses. Gynecol Oncol. 2010; 116 (3):384–8.
17. Yelikar KA,  Deshpande SS, Nanaware SS,  Pagare SB Evaluation of the validity of risk malignancy index in clinically diagnosed ovarian masses and to compare it with the validity of individual constituent parameter of risk malignancy index Int J Reprod Contracept Obstet Gynecol. 2016 Feb;5(2):460-464)
18. Obeidat BR, Amarin A LJE al. Risk of malignancy index in the perioperative evaluation of pelvic masses. Int J Gynecol. 2004; 85:255–8.
19. Anton C, Carvalho F, Oliveira E, Maciel G, Baracat E, Carvalho J. A comparison of CA125, HE4, risk ovarian malignancy algorithm (ROMA), and risk malignancy index (RMI) for the classification of ovarian masses. Clinics [Internet]. 2012; 67(5):437–41. Available from: http://clinics.org.br/article.php?id=751
20. Santosh Kumar Dora, Atal Bihari Dandapat, Benudhar Pande JPH. A prospective study to evaluate the risk malignancy index and its diagnostic implication in patients with suspected ovarian mass. J Ovarian Res. 2017; 10(1):1.
21. Ganiy OA, Liam M KL. A. Taiwan J Obs gynecol. 2014; 53(3):376–81.
22. Manjunath A, Sujatha K VR. Comparison of three risk of malignancy indices in evaluation of pelvic masses. Gynecol Oncol. 2001; 81(2):225–9.
23. Akobeng AK. Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr. 2007; 96(5):644–7.
24. Moolthiya W, Yuenyao P. The risk of malignancy index (RMI) in diagnosis of ovarian malignancy. Asian Pacific J Cancer Prev [Internet]. 2009; 10(5):865–8. Available from: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=emed9&NEWS=N&AN=2016285
25. Giuseppe M, Antonio L, Antonio D VD. Comparison of two malignancy risk indices based on serum CA125, ultrasound score and menopausal status in the diagnosis of ovarian masses. Br J Obs Gynaecol. 1999; 106:524–521. 
26. Asadi JN, Al-Maliki SK, Al-Dahhhan F, Al-Naama L SF. The accuracy of risk malignancy index in prediction of malignancy in women with adnexal mass in Basrah, Iraq. Niger J ClinPract. 2018; 21:1254–9. 
27. Yamamoto Y, Tsuchida A, Ushiwaka T, Nagai R, Matsumoto M, Komatsu J, et al. Comparison of 4 Risk-of-Malignancy Indexes in the Preoperative Evaluation of Patients with Pelvic Masses: A Prospective Study. Clin Ovarian Other Gynecol Cancer [Internet]. Elsevier; 2014https://www.sciencedirect.com/science/article/pii/S2212955314000623#cebib0010