SCIENCE CHINA Information Sciences, Volume 62, Issue 11: 212205(2019) https://doi.org/10.1007/s11432-019-9866-3

Data-driven group decision making for diagnosis of thyroid nodule

More info
  • ReceivedJan 2, 2019
  • AcceptedMar 29, 2019
  • PublishedSep 20, 2019


Emerging information technologies' integration into various fields has enhanced the development of these fields. Large volumes of data have been accumulated in this process. The accumulated data offer opportunities and challenges for people facing practical problems. On the one hand, it is essential to depend on a group's capabilities rather than an individual's capabilities to handle practical problems because the individual may lack sufficient expertise and experience to use data. In this situation, the practical problems can be considered as group decision making (GDM) problems. On the other hand, the accumulated data can help generate quality solutions to GDM problems. To obtain such solutions under the assumption that the accumulated data regarding a specific decision problem are available, this paper proposes a data-driven GDM method. In the method, decision makers' weights are learned from historical overall assessments and the corresponding gold standards, while criterion weights are learned from historical overall assessments and the corresponding decision matrices. The learned expert weights and criterion weights are used to produce the aggregated assessments, from which alternatives are compared or the overall conclusion is made. In a tertiary hospital located in Hefei, Anhui Province, China, the proposed method is applied to aid radiologists in diagnosing thyroid nodules.


This work was supported by National Natural Science Foundation of China (Grant Nos. 71622003, 71571060, 71690235, 71690230, 71521001).




[1] Galo N R, Calache L D D R, Carpinetti L C R. A group decision approach for supplier categorization based on hesitant fuzzy and ELECTRE TRI. Int J Production Economics, 2018, 202: 182-196 CrossRef Google Scholar

[2] Qin G D, Liu X W, Pedrycz W. An extended TODIM multi-criteria group decision making method for green supplier selection in interval type-2 fuzzy environment. Eur J Operational Res, 2017, 258: 626-638 CrossRef Google Scholar

[3] Cheng S H. Autocratic multiattribute group decision making for hotel location selection based on interval-valued intuitionistic fuzzy sets. Inf Sci, 2018, 427: 77-87 CrossRef Google Scholar

[4] Chen L H, Ko W C, Tseng C Y. Fuzzy Approaches for Constructing House of Quality in QFD and Its Applications: A Group Decision-Making Method. IEEE Trans Eng Manage, 2013, 60: 77-87 CrossRef Google Scholar

[5] Ren J Z, Liang H W. Measuring the sustainability of marine fuels: A fuzzy group multi-criteria decision making approach. Transpation Res Part D-Transp Environ, 2017, 54: 12-29 CrossRef Google Scholar

[6] Wu B, Yan X P, Wang Y. Selection of maritime safety control options for NUC ships using a hybrid group decision-making approach. Saf Sci, 2016, 88: 108-122 CrossRef Google Scholar

[7] Lu J, Ma J, Zhang G Q. Theme-Based Comprehensive Evaluation in New Product Development Using Fuzzy Hierarchical Criteria Group Decision-Making Method. IEEE Trans Ind Electron, 2011, 58: 2236-2246 CrossRef Google Scholar

[8] Li G X, Kou G, Peng Y. A Group Decision Making Model for Integrating Heterogeneous Information. IEEE Trans Syst Man Cybern Syst, 2018, 48: 982-992 CrossRef Google Scholar

[9] Liu P D, Chen S M. Group Decision Making Based on Heronian Aggregation Operators of Intuitionistic Fuzzy Numbers.. IEEE Trans Cybern, 2017, 47: 2514-2530 CrossRef PubMed Google Scholar

[10] Wu Q, Wu P, Zhou LG. Some new Hamacher aggregation operators under single-valued neutrosophic 2-tuple linguistic environment and their applications to multi-attribute group decision making. Comput Industrial Eng, 2018, 116: 144-162 CrossRef Google Scholar

[11] Jana C, Senapati T, Pal M. Picture fuzzy Dombi aggregation operators: Application to MADM process. Appl Soft Computing, 2019, 74: 99-109 CrossRef Google Scholar

[12] Fu C T, Yang S L. An evidential reasoning based consensus model for multiple attribute group decision analysis problems with interval-valued group consensus requirements. Eur J Operational Res, 2012, 223: 167-176 CrossRef Google Scholar

[13] Yeh C T. Existence of interval, triangular, and trapezoidal approximations of fuzzy numbers under a general condition. Fuzzy Sets Syst, 2017, 310: 1-13 CrossRef Google Scholar

[14] Lima A S, de Souza J N, Moura A B. A consensus-based multicriteria group decision model for information technology management committees. IEEE Trans Eng Manage, 2018, 65: 276-292 CrossRef Google Scholar

[15] Yan H B, Ma T J, Huynh V N. On qualitative multi-attribute group decision making and its consensus measure: A probability based perspective. Omega, 2017, 70: 94-117 CrossRef Google Scholar

[16] Yang Y, Wang X X, Xu Z S. The multiplicative consistency threshold of intuitionistic fuzzy preference relation. Inf Sci, 2019, 477: 349-368 CrossRef Google Scholar

[17] Li C C, Rodriguez R M, Martinez L. Consensus Building With Individual Consistency Control in Group Decision Making. IEEE Trans Fuzzy Syst, 2019, 27: 319-332 CrossRef Google Scholar

[18] Meng F Y, An Q X, Tan C Q. An Approach for Group Decision Making With Interval Fuzzy Preference Relations Based on Additive Consistency and Consensus Analysis. IEEE Trans Syst Man Cybern Syst, 2017, 47: 2069-2082 CrossRef Google Scholar

[19] Wan S P, Wang F, Dong J Y. A Three-Phase Method for Group Decision Making With Interval-Valued Intuitionistic Fuzzy Preference Relations. IEEE Trans Fuzzy Syst, 2018, 26: 998-1010 CrossRef Google Scholar

[20] Kou G, Ergu D J, Lin C S. PAIRWISE COMPARISON MATRIX IN MULTIPLE CRITERIA DECISION MAKING. Tech Economic Dev Economy, 2017, 22: 738-765 CrossRef Google Scholar

[21] Kou G, Ergu D J, Shang J. Enhancing data consistency in decision matrix: Adapting Hadamard model to mitigate judgment contradiction. Eur J Operational Res, 2014, 236: 261-271 CrossRef Google Scholar

[22] Liu B S, Shen Y H, Zhang W. An interval-valued intuitionistic fuzzy principal component analysis model-based method for complex multi-attribute large-group decision-making. Eur J Operational Res, 2015, 245: 209-225 CrossRef Google Scholar

[23] Wu T, Liu X W, Liu F. An interval type-2 fuzzy TOPSIS model for large scale group decision making problems with social network information. Inf Sci, 2018, 432: 392-410 CrossRef Google Scholar

[24] Wu T, Liu X W. An interval type-2 fuzzy clustering solution for large-scale multiple-criteria group decision-making problems. Knowledge-Based Syst, 2016, 114: 118-127 CrossRef Google Scholar

[25] Chen X, Zhang H J, Dong Y C. The fusion process with heterogeneous preference structures in group decision making: A survey. Inf Fusion, 2015, 24: 72-83 CrossRef Google Scholar

[26] Tang J, Chen S M, Meng F Y. Heterogeneous group decision making in the setting of incomplete preference relations. Inf Sci, 2019, 483: 396-418 CrossRef Google Scholar

[27] Wan S P, Xu J, Dong J Y. Aggregating decision information into interval-valued intuitionistic fuzzy numbers for heterogeneous multi-attribute group decision making. Knowledge-Based Syst, 2016, 113: 155-170 CrossRef Google Scholar

[28] Haag F, Lienert J, Schuwirth N. Identifying non-additive multi-attribute value functions based on uncertain indifference statements. Omega, 2019, 85: 49-67 CrossRef Google Scholar

[29] Qin G D, Liu X W. Multi-attribute group decision making using combined ranking value under interval type-2 fuzzy environment. Inf Sci, 2015, 297: 293-315 CrossRef Google Scholar

[30] Yue C. A geometric approach for ranking interval-valued intuitionistic fuzzy numbers with an application to group decision-making. Comput Industrial Eng, 2016, 102: 233-245 CrossRef Google Scholar

[31] Entani T, Inuiguchi M. Pairwise comparison based interval analysis for group decision aiding with multiple criteria. Fuzzy Sets Syst, 2015, 274: 79-96 CrossRef Google Scholar

[32] Fu C, Xu D L. Determining attribute weights to improve solution reliability and its application to selecting leading industries. Ann Oper Res, 2016, 245: 401-426 CrossRef Google Scholar

[33] Kim J H, Ahn B S. Extended VIKOR method using incomplete criteria weights. Expert Syst Appl, 2019, 126: 124-132 CrossRef Google Scholar

[34] Zhang Z, Guo C H, Martinez L. Managing Multigranular Linguistic Distribution Assessments in Large-Scale Multiattribute Group Decision Making. IEEE Trans Syst Man Cybern Syst, 2017, 47: 3063-3076 CrossRef Google Scholar

[35] Dong Y C, Xiao J, Zhang H J. Managing consensus and weights in iterative multiple-attribute group decision making. Appl Soft Computing, 2016, 48: 80-90 CrossRef Google Scholar

[36] Liu B S, Shen Y H, Chen Y. A two-layer weight determination method for complex multi-attribute large-group decision-making experts in a linguistic environment. Inf Fusion, 2015, 23: 156-165 CrossRef Google Scholar

[37] Shi Z J, Wang X Q, Palomares I. A novel consensus model for multi-attribute large-scale group decision making based on comprehensive behavior classification and adaptive weight updating. Knowledge-Based Syst, 2018, 158: 196-208 CrossRef Google Scholar

[38] Pérez I J, Cabrerizo F J, Alonso S. On dynamic consensus processes in group decision making problems. Inf Sci, 2018, 459: 20-35 CrossRef Google Scholar

[39] Hajek P, Froelich W. Integrating TOPSIS with interval-valued intuitionistic fuzzy cognitive maps for effective groupdecision making. Inf Sci, 2019, 485: 394--412. Google Scholar

[40] Liu W, Li L. An approach to determining the integrated weights of decision makers based on interval number group decision matrices. Knowledge-Based Syst, 2015, 90: 92-98 CrossRef Google Scholar

[41] Tambouratzis T, Canellidis V. Reward-Penalty Assignments and Genetic Algorithms for Ordinal Interval Number Group Decision Making. Int J Intell Syst, 2014, 29: 727-750 CrossRef Google Scholar

[42] Yue Z L. Group decision making with multi-attribute interval data. Inf Fusion, 2013, 14: 551-561 CrossRef Google Scholar

[43] Roberts R, Goodwin P. Weight approximations in multi-attribute decision models. J Multi-Crit Decis Anal, 2002, 11: 291-303 CrossRef Google Scholar

[44] Yang G L, Yang J B, Xu D L. A three-stage hybrid approach for weight assignment in MADM. Omega, 2017, 71: 93-105 CrossRef Google Scholar

[45] Wang Z J, Liu F, Lin J. Fuzzy eigenvector method for obtaining normalized fuzzy weights from fuzzy pairwise comparison matrices. Fuzzy Sets Syst, 2017, 315: 26-43 CrossRef Google Scholar

[46] Shirland L E, Jesse R R, Thompson R L. Determining attribute weights using mathematical programming. Omega, 2003, 31: 423-437 CrossRef Google Scholar

[47] Fu C, Xu D L, Xue M. Determining attribute weights for multiple attribute decision analysis with discriminating power in belief distributions. Knowledge-Based Syst, 2018, 143: 127-141 CrossRef Google Scholar

[48] Barron F H, Barrett B E. Decision Quality Using Ranked Attribute Weights. Manage Sci, 1996, 42: 1515-1523 CrossRef Google Scholar

[49] Koksalmis E, Kabak O. Deriving decision makers' weights in group decision making: An overview of objective methods. Inf Fusion, 2019, 49: 146-160 CrossRef Google Scholar

[50] Wang Y M, Luo Y. Integration of correlations with standard deviations for determining attribute weights in multiple attribute decision making. Math Comput Model, 2010, 51: 1-12 CrossRef Google Scholar

[51] He Y H, Guo H W, Jin M Z. A linguistic entropy weight method and its application in linguistic multi-attribute group decision making. NOnlinear Dyn, 2016, 84: 399-404 CrossRef Google Scholar

[52] Sahin R, Liu P. Maximizing deviation method for neutrosophic multiple attribute decision making with incomplete weight information. Neural Comput Applic, 2016, 27: 2017-2029 CrossRef Google Scholar

[53] Yue Z L. Approach to group decision making based on determining the weights of experts by using projection method. Appl Math Model, 2012, 36: 2900-2910 CrossRef Google Scholar

[54] Yue C. Entropy-based weights on decision makers in group decision-making setting with hybrid preference representations. Appl Soft Computing, 2017, 60: 737-749 CrossRef Google Scholar

[55] Qi X W, Liang C Y, Zhang J L. Generalized cross-entropy based group decision making with unknown expert and attribute weights under interval-valued intuitionistic fuzzy environment. Comput Industrial Eng, 2015, 79: 52-64 CrossRef Google Scholar

[56] Zhang X, Liu P D. Method for multiple attribute decision-making under risk with interval numbers. Int J Fuzzy Syst, 2010, 12(3): 237-242. Google Scholar

[57] Tran L, Duckstein L. Comparison of fuzzy numbers using a fuzzy distance measure. Fuzzy Sets Syst, 2002, 130: 331-341 CrossRef Google Scholar

[58] de Carvalho F A T, Simoes E C. Fuzzy clustering of interval-valued data with City-Block and Hausdorff distances. Neurocomputing, 2017, 266: 659-673 CrossRef Google Scholar

[59] Zhang J, Pang J Z, Yu J F. An efficient assembly retrieval method based on Hausdorff distance. Robotics Comput-Integrated Manufacturing, 2018, 51: 103-111 CrossRef Google Scholar

[60] Ramos-Guajardo A B, Grzegorzewski P. Distance-based linear discriminant analysis for interval-valued data. Inf Sci, 2016, 372: 591-607 CrossRef Google Scholar

[61] Irpino A, Verde R. Dynamic clustering of interval data using a Wasserstein-based distance. Pattern Recognition Lett, 2008, 29: 1648-1658 CrossRef Google Scholar

[62] Li X, Zhang S L, Zhang M, et al. Rank of interval numbers based on a new distance measure. J Xihua Univ (Nat Sci), 2008, 27: 87--90. Google Scholar

[63] Winston W L. Operations research: applications and algorithms. Boston: Duxbury Press, 2003. Google Scholar

[64] Sahli Z T, Karipineni F, Hang J F. The association between the ultrasonography TIRADS classification system and surgical pathology among indeterminate thyroid nodules.. Surgery, 2019, 165: 69-74 CrossRef PubMed Google Scholar

[65] Horvath E, Silva C F, Majlis S. Prospective validation of the ultrasound based TIRADS (Thyroid Imaging Reporting And Data System) classification: results in surgically resected thyroid nodules.. Eur Radiol, 2017, 27: 2619-2628 CrossRef PubMed Google Scholar

[66] Kwak J Y, Han K H, Yoon J H. Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk.. Radiology, 2011, 260: 892-899 CrossRef PubMed Google Scholar

[67] Park J Y, Lee H J, Jang H W. A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma.. Thyroid, 2009, 19: 1257-1264 CrossRef PubMed Google Scholar

[68] Fu C, Liu W Y, Chang W J. Data-driven multiple criteria decision making for diagnosis of thyroid cancer. Ann Oper Res, https://doi.org/10.1007/s10479-018-3093-7, 2018. Google Scholar

[69] Cappelli C, Castellano M, Pirola I. The predictive value of ultrasound findings in the management of thyroid nodules.. QJM, 2006, 100: 29-35 CrossRef PubMed Google Scholar

[70] Chan B K, Desser T S, McDougall I R. Common and Uncommon Sonographic Features of Papillary Thyroid Carcinoma. J Ultrasound Med, 2003, 22: 1083-1090 CrossRef Google Scholar

[71] Frates M C, Benson C B, Charboneau J W. Management of thyroid nodules detected at US: Society of Radiologists in Ultrasound consensus conference statement.. Radiology, 2005, 237: 794-800 CrossRef PubMed Google Scholar

[72] Moon W J, Jung S L, Lee J H. Benign and malignant thyroid nodules: US differentiation--multicenter retrospective study.. Radiology, 2008, 247: 762-770 CrossRef PubMed Google Scholar

  • Figure 1

    (Color online) MCGDM process for the proposed method.

  • Figure 2

    (Color online) Movement of $\widetilde{C}_{k}^{G_{w},G_{d}}(k~=~1,\ldots,5)$ with random $\lambda_{j}(j~=~1,2,3)$. (a) $\widetilde{t}_{1}$; (b) $\widetilde{t}_{2}$; (c) $\widetilde{t}_{3}$; (d) $\widetilde{t}_{4}$; (e) $\widetilde{t}_{5}$.

  • Figure 3

    (Color online) Comparison between $\widetilde{C}_{k}^{\bar{G}_{w},G_{d}}$ and $\widetilde{C}_{k}^{G_{w},\bar{G}_{d}}(k~=~1,\ldots,5)$. (a) $\widetilde{t}_{1}$; (b) $\widetilde{t}_{2}$; (c) $\widetilde{t}_{3}$; (d) $\widetilde{t}_{4}$; (e) $\widetilde{t}_{5}$.

  • Figure 4

    Comparison between ROC curves from the group-recommended TIRADS categories and that from the overall diagnose for radiologists. (a) $\widetilde{t}_{1}$; (b) $\widetilde{t}_{2}$; (c) $\widetilde{t}_{3}$; (d) $\widetilde{t}_{4}$; (e) $\widetilde{t}_{5}$.

  • Table 1   TIRADS categories applied in the hospital
    Category Finding Cancer risk Recommendation
    TIRADS 3 Probably benign $<$3% Follow-up/FNAB
    TIRADS 4 Suspicious 3%–75%
    TIRADS 4A Low suspicion 3%–24%
    TIRADS 4A-1 Tending towards benign nodule 3%–15% Follow-up/FNAB
    TIRADS 4A-2 Not excluding the possibility of malignant nodule 16%–24% FNAB
    TIRADS 4B Intermediate suspicion 25%–75%
    TIRADS 4B-1 Not excluding the possibility of benign nodule 25%–40% FNAB
    TIRADS 4B-2 Medium possibility of malignant nodule 41%–65% FNAB
    TIRADS 4B-3 Large possibility of malignant nodule 66%–75% FNAB
    TIRADS 4C High suspicion 76%–95% FNAB
    TIRADS 5 Suggestive of malignancy $>$ 95% FNAB
  • Table 2   Details about the eight radiologists
    Radiologist Serving period Diagnostic record
    $D_{1}$ 2013–2018 591
    $D_{2}$ 2011–2018 586
    $D_{3}$ 2012–2018 628
    $D_{4}$ 2015–2018 397
    $D_{5}$ 2013–2017 179
    $D_{6}$ 2011–2016 180
    $D_{7}$ 2017–2018 202
    $D_{8}$ 2018–2018 93
  • Table 3   Weights of the five criteria for the eight radiologists
    Radiologist Learned criterion weight
    $t_{1}$ $w_{i}^{1}~~(i~=~1,\ldots,5)~=~(0.1893,~0.2386,~0.1798,~0.2119,~0.1804)$
    $t_{2}$ $w_{i}^{2}~~(i~=~1,\ldots,5)~=~(0.1901,~0.2412,~0.1778,~0.2164,~0.1745)$
    $t_{3}$ $w_{i}^{3}~~(i~=~1,\ldots,5)~=~(0.2005,~0.2352,~0.188,~0.1938,~0.1825)$
    $\widetilde{t}_{1}$ $\widetilde{w}_{i}^{1}~~(i~=~1,\ldots,5)~=~(0.2002,~0.2287,~0.1745,~0.2115,~0.1851)$
    $\widetilde{t}_{2}$ $\widetilde{w}_{i}^{2}~~(i~=~1,\ldots,5)~=~(0.1928,~0.2238,~0.1852,~0.216,~0.1822)$
    $\widetilde{t}_{3}$ $\widetilde{w}_{i}^{3}~~(i~=~1,\ldots,5)~=~(0.1981,~0.2205,~0.2126,~0.1911,~0.1778)$
    $\widetilde{t}_{4}$ $\widetilde{w}_{i}^{4}~~(i~=~1,\ldots,5)~=~(0.2027,~0.2314,~0.1668,~0.2291,~0.1699)$
    $\widetilde{t}_{5}$ $\widetilde{w}_{i}^{5}~~(i~=~1,\ldots,5)~=~(0.1811,~0.251,~0.1713,~0.2276,~0.169)$
  • Table 4   Distributions of the overall diagnoses on the TIRADS categories for the eight radiologists
    Radiologist Nodule Distributions of overall diagnoses on $T_{c}$
    $t_{1}$ Malignant $d_{1,c}^{m}~~(c~=~1,\ldots,~8)~=~(10,~14,~15,~25,~42,~25,~36,~66)$
    $t_{1}$ Benign $d_{1,c}^{b}~~(c~=~1,\ldots,~8)~=(261,~29,~15,~10,~24,~12,~2,~5)$
    $t_{2}$ Malignant $d_{2,c}^{m}~~(c~=~1,\ldots,~8)~=~(10,~1,~19,~1,~17,~49,~26,~84)$
    $t_{2}$ Benign $d_{2,c}^{b}~~(c~=~1,\ldots,~8)~=~(254,~12,~33,~11,~20,~33,~8,~8)$
    $t_{3}$ Malignant $d_{3,c}^{m}~(c~=~1,\ldots,~8)~=~(5,~1,~18,~0,~32,~76,~17,~18)$
    $t_{3}$ Benign $d_{3,c}^{b}~(c~=~1,\ldots,~8)~=~(155,~18,~23,~7,~13,~10,~2,~2)$
    $\widetilde{t}_{1}$ Malignant $\widetilde{d}_{1,c}^{m}~(c~=~1,\ldots,~8)~=~(17,~1,~21,~2,~36,~71,~52,~42)$
    $\widetilde{t}_{1}$ Benign $\widetilde{d}_{1,c}^{b}~(c~=~1,\ldots,~8)~=~(206,~31,~47,~26,~42,~20,~9,~5)$
    $\widetilde{t}_{2}$ Malignant $\widetilde{d}_{2,c}^{m}~(c~=~1,\ldots,~8)~=~(9,~3,~5,~0,~8,~14,~8,~9)$
    $\widetilde{t}_{2}$ Benign $\widetilde{d}_{2,c}^{b}~(c~=~1,\ldots,~8)~=~(66,~11,~11,~11,~13,~9,~2,~0)$
    $\widetilde{t}_{3}$ Malignant $\widetilde{d}_{3,c}^{m}~(c~=~1,\ldots,~8)~=~(3,~1,~3,~2,~5,~9,~2,~12)$
    $\widetilde{t}_{3}$ Benign $\widetilde{d}_{3,c}^{b}~~(c~=~1,\ldots,~8)~=~(84,~2,~7,~9,~20,~16,~2,~3)$
    $\widetilde{t}_{4}$ Malignant $\widetilde{d}_{4,c}^{m}~(c~=~1,\ldots,~8)~=~(3,~5,~14,~1,~14,~46,~19,~14)$
    $\widetilde{t}_{4}$ Benign $\widetilde{d}_{4,c}^{b}~(c~=~1,\ldots,~8)~=~(60,~7,~12,~1,~1,~4,~1,~0)$
    $\widetilde{t}_{5}$ Malignant $\widetilde{d}_{5,c}^{m}~(c~=~1,\ldots,~8)~=~(3,~2,~2,~2,~11,~27,~10,~2)$
    $\widetilde{t}_{5}$ Benign $\widetilde{d}_{5,c}^{b}~(c~=~1,\ldots,~8)~=~(17,~7,~4,~1,~3,~2,~0,~0)$
  • Table 5   Diagnostic capabilities of radiologists $\widetilde{t}_{k}~(k~=~1,\ldots,~5)$ in different situations
    Condition Diagnostic capabilities of five radiologists
    Group's weights and group's distributions $\widetilde{C}_{k}^{G_{w},G_{d}}~(k~=~1,\ldots,5)~=~(0.8088,~0.7712,~0.8104,~0.7793,~0.7721)$
    Group's weights and radiologists' distributions $\widetilde{C}_{k}^{G_{w},R_{d}}~(k~=~1,\ldots,5)~=(0.7566,~0.7469,~0.8104,~0.6821,~0.6577)$
    Radiologists' weights and group's distributions$\widetilde{C}_{k}^{R_{w},G_{d}}~(k~=~1,\ldots,5)~=(0.8012,~0.7712,~0.805,~0.7698,~0.7721)$
    Radiologists' weights and radiologists' distributions $\widetilde{C}_{k}^{R_{w},R_{d}}~(k~=~1,\ldots,5)~=~(0.7514,~0.7469,~0.805,~0.6745,~0.6577)$
    Radiologists' overall diagnoses $\widetilde{C}_{k}^{R_{0}}~(k~=~1,\ldots,5)~=(0.7505,~0.7317,~0.7451,~0.7445,~0.7025)$

Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有