<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1 20151215//EN" "http://jats.nlm.nih.gov/publishing/1.1/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="research-article" dtd-version="1.1">
<front>
<journal-meta>
<journal-id journal-id-type="pmc">CMC</journal-id>
<journal-id journal-id-type="nlm-ta">CMC</journal-id>
<journal-id journal-id-type="publisher-id">CMC</journal-id>
<journal-title-group>
<journal-title>Computers, Materials &#x0026; Continua</journal-title>
</journal-title-group>
<issn pub-type="epub">1546-2226</issn>
<issn pub-type="ppub">1546-2218</issn>
<publisher>
<publisher-name>Tech Science Press</publisher-name>
<publisher-loc>USA</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">19404</article-id>
<article-id pub-id-type="doi">10.32604/cmc.2022.019404</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Article</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Profiling Casualty Severity Levels of Road Accident Using Weighted Majority Voting</article-title>
<alt-title alt-title-type="left-running-head">Profiling Casualty Severity Levels of Road Accident Using Weighted Majority Voting</alt-title>
<alt-title alt-title-type="right-running-head">Profiling Casualty Severity Levels of Road Accident Using Weighted Majority Voting</alt-title>
</title-group>
<contrib-group content-type="authors">
<contrib id="author-1" contrib-type="author">
<name name-style="western"><surname>Awan</surname><given-names>Saba</given-names></name><xref ref-type="aff" rid="aff-1">1</xref>
</contrib>
<contrib id="author-2" contrib-type="author" corresp="yes">
<name name-style="western"><surname>Mehmood</surname><given-names>Zahid</given-names></name><xref ref-type="aff" rid="aff-2">2</xref><email>zahid.mehmood@uettaxila.edu.pk</email>
</contrib>
<contrib id="author-3" contrib-type="author">
<name name-style="western"><surname>Chaudhry</surname><given-names>Hassan Nazeer</given-names></name><xref ref-type="aff" rid="aff-3">3</xref>
</contrib>
<contrib id="author-4" contrib-type="author">
<name name-style="western"><surname>Tariq</surname><given-names>Usman</given-names></name><xref ref-type="aff" rid="aff-4">4</xref>
</contrib>
<contrib id="author-5" contrib-type="author">
<name name-style="western"><surname>Rehman</surname><given-names>Amjad</given-names></name><xref ref-type="aff" rid="aff-5">5</xref>
</contrib>
<contrib id="author-6" contrib-type="author">
<name name-style="western"><surname>Saba</surname><given-names>Tanzila</given-names></name><xref ref-type="aff" rid="aff-5">5</xref>
</contrib>
<contrib id="author-7" contrib-type="author">
<name name-style="western"><surname>Rashid</surname><given-names>Muhammad</given-names></name><xref ref-type="aff" rid="aff-6">6</xref>
</contrib>
<aff id="aff-1"><label>1</label><institution>Department of Software Engineering, University of Engineering and Technology</institution>, <addr-line>Taxila, 47050</addr-line>, <country>Pakistan</country></aff>
<aff id="aff-2"><label>2</label><institution>Department of Computer Engineering, University of Engineering and Technology</institution>, <addr-line>Taxila, 47050</addr-line>, <country>Pakistan</country></aff>
<aff id="aff-3"><label>3</label><institution>Department of Electronics, Information and Bioengineering, Politecnico di Milano</institution>, <addr-line>Milano, 20122</addr-line>, <country>Italy</country></aff>
<aff id="aff-4"><label>4</label><institution>College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University</institution>, <addr-line>Al-Kharj, 11942</addr-line>, <country>Saudi Arabia</country></aff>
<aff id="aff-5"><label>5</label><institution>College of Computer and Information Sciences, Prince Sultan University</institution>, <addr-line>Riyadh</addr-line>, <country>Saudi Arabia</country></aff>
<aff id="aff-6"><label>6</label><institution>Department of Computer Engineering, Umm Al-Qura University</institution>, <addr-line>Makkah, 21421</addr-line>, <country>Saudi Arabia</country></aff>
</contrib-group>
<author-notes>
<corresp id="cor1"><label>&#x002A;</label>Corresponding Author: Zahid Mehmood. Email: <email>zahid.mehmood@uettaxila.edu.pk</email></corresp>
</author-notes>
<pub-date pub-type="epub" date-type="pub" iso-8601-date="2022-01-13"><day>13</day>
<month>01</month>
<year>2022</year></pub-date>
<volume>71</volume>
<issue>3</issue>
<fpage>4609</fpage>
<lpage>4626</lpage>
<history>
<date date-type="received"><day>12</day><month>4</month><year>2021</year></date>
<date date-type="accepted"><day>25</day><month>10</month><year>2021</year></date>
</history>
<permissions>
<copyright-statement>&#x00A9; 2022 Awan et al.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Awan et al.</copyright-holder>
<license xlink:href="https://creativecommons.org/licenses/by/4.0/">
<license-p>This work is licensed under a <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</ext-link>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
</license>
</permissions>
<self-uri content-type="pdf" xlink:href="TSP_CMC_19404.pdf"></self-uri>
<abstract>
<p>To determine the individual circumstances that account for a road traffic accident, it is crucial to consider the unplanned connections amongst various factors related to a crash that results in high casualty levels. Analysis of the road accident data concentrated mainly on categorizing accidents into different types using individually built classification methods which limit the prediction accuracy and fitness of the model. In this article, we proposed a multi-model hybrid framework of the weighted majority voting (WMV) scheme with parallel structure, which is designed by integrating individually implemented multinomial logistic regression (MLR) and multilayer perceptron (MLP) classifiers using three different accident datasets i.e., IRTAD, NCDB, and FARS. The proposed WMV hybrid scheme overtook individual classifiers in terms of modern evaluation measures like ROC, RMSE, Kappa rate, classification accuracy, and performs better than state-of-the-art approaches for the prediction of casualty severity level. Moreover, the proposed WMV hybrid scheme adds up to accident severity analysis through knowledge representation by revealing the role of different accident-related factors which expand the risk of casualty in a road crash. Critical aspects related to casualty severity recognized by the proposed WMV hybrid approach can surely support the traffic enforcement agencies to develop better road safety plans and ultimately save lives.</p>
</abstract>
<kwd-group kwd-group-type="author">
<kwd>Prediction</kwd>
<kwd>hybrid framework</kwd>
<kwd>severity</kwd>
<kwd>class</kwd>
<kwd>casualty</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec id="s1"><label>1</label><title>Introduction</title>
<p>From the preceding era, accidents caused by road traffic have emerged as a widespread difficulty. The currently published &#x201C;global status report on road safety&#x201D; highlights that 50 million people suffered and 1.35 million deceased in 178 countries due to road mishaps and that goes down even more severely in under-developed countries [<xref ref-type="bibr" rid="ref-1">1</xref>] (<xref ref-type="fig" rid="fig-1">Fig. 1</xref>). A road traffic accident (RTA) does not occur at random. Its complexity encompasses the interconnections of the different traits of the driver, vehicle, road, and environmental factors. Substantial developments have therefore been undertaken in the field of accident analysis, especially when it comes to the prevention of injury and modeling of accident prediction. Traditionally, the enormous mass of research [<xref ref-type="bibr" rid="ref-2">2</xref>] relevant to accident evaluations is based on different forms of regression modeling with the mainly concerned with accident occurrence rather than on the estimation of accident intensity. Moreover, prior research employing computational modeling approaches [<xref ref-type="bibr" rid="ref-3">3</xref>&#x2013;<xref ref-type="bibr" rid="ref-5">5</xref>], shows the unpredictable effects due to the socio-economic circumstances of a specific location using their site-specific accident data.</p>
<fig id="fig-1"><label>Figure 1</label><caption><title>Global status report on road safety 2018</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_19404-fig-1.png"/></fig>
<p>Many prevailing studies [<xref ref-type="bibr" rid="ref-6">6</xref>,<xref ref-type="bibr" rid="ref-7">7</xref>] are somewhat constrained by the problem of limited dataset availability and model over or underfitting due to the usage of single-level classification modeling. The proposed WMV hybrid scheme is trained and tested over multiple accident datasets through multi-level hybrid modeling, which makes it a well-fitted approach for generalizing similar data to that on which it was trained. Hence producing more accurate outcomes. Moreover, a hybrid approach to combine single classifiers has occasionally been employed except in a hierarchical structure of majority voting schemes. Besides this, in state of the art approaches are incorporate the ordinary parameters generating typical prediction accuracy instead of using fine hyper-parameter tuning. Further, limited evaluation measures have been practiced for results comparison like percentage accuracy and root mean squared error only. Hence, revealing few discriminating factors for correct class prediction. So, the proposed research work adds novelty in accident analysis by implementing a well-fitted multi-level hybrid approach which is trained and tested over multiple accident datasets to develop a more generic framework to address the weaknesses in prior studies. It critically considers the impact of road traffic accidents in several levels of casualties as a multi-classification problem rather than modeling the frequency of crashes on a specified section over a long epoch. As a result, the level of accident severity with limited injuries is stated as &#x201C;Slight,&#x201D; more injuries as &#x201C;Serious&#x201D; and death as &#x201C;Fatal&#x201D; severity. In assessing accidents with such comprehensive complexity, essential dynamics in highly predictor variables are identified while the relevance between predictor variables and accident characteristics is consistent when the final prediction is rendered. Moreover, the weighted majority voting (WMV) [<xref ref-type="bibr" rid="ref-8">8</xref>] scheme with parallel structure provides better accuracy as compared with prevailing cascading methodologies that don&#x0027;t reveal reasonable accuracy in terms of &#x201C;Casualty Severity&#x201D; prediction. This paper improves the preceding effort in road accident data analysis by considering the strong relationships between accident characteristics and different &#x201C;Casualty Severity&#x201D; levels of a particular RTA. In our study, we used general and unbalanced datasets of road accidents from different accident repositories to make comparisons of numerous sophisticated methods to solve a multiple classification problem.</p>
<p>The major contribution of the proposed approach are as follows:
<list list-type="simple">
<list-item><label>1.</label><p>It assesses accident casualty severity level instead of accident frequency count.</p></list-item>
<list-item><label>2.</label><p>It uses a multi-level statistical model for supervised learning which is based on multinomial logistic regression (MLR) and multilayer perceptron (MLP) classifiers.</p></list-item>
<list-item><label>c)</label><p>It selects features based on correlations using statistical resampling and dimensionality reduction.</p></list-item>
<list-item><label>d)</label><p>It performs hyperparameter tuning of multi-level models for adequate regulation of the developed classifiers.</p></list-item>
<list-item><label>e)</label><p>It can accurately predict the unknown casualty severity of an RTA.</p></list-item>
<list-item><label>f)</label><p>It uses a generic hybrid framework by integration of individually developed models using the WMV ensemble modeling approach with Parallel structure.</p></list-item>
<list-item><label>g)</label><p>It utilizes knowledge discovery by exploring the individual behavioral characteristics, highway aspects, environmental aspects, and vehicle attributes related to a specific casualty severity.</p></list-item>
</list></p>
<p>The remaining sections of this article are arranged in the following way. The critical analysis of existing state-of-the-art methods is described in Section 2. The detailed methodology of the proposed WMV hybrid scheme is presented in Section 3. The performance assessment of the proposed WMV hybrid scheme is presented in Section 4 alongside a thorough analysis of the research results. A brief conclusion of the proposed WMV hybrid scheme followed by future directions is discussed in Section 5.</p>
</sec>
<sec id="s2"><label>2</label><title>Literature Review</title>
<p>Vast investigative state-of-the-art approaches have been used to examine the consequences of several possible causes that affect the degree of injuries caused by traffic collisions. Similarly, statistical and traditional classification methods were used to assess the severity of an accident&#x0027;s injuries. To evaluate the relationships between predictive variables (significant risk variables) and the outcome variable (level of injury), conventional regression techniques have prevailed over other models.</p>
<p>Research work on characterization and severity estimation of traffic accidents in Spain is also conducted by [<xref ref-type="bibr" rid="ref-9">9</xref>], which builds predictive models using naive Bayes, gradient method with boosting trees, and deep machine learning approach. The comparative study of multiple outcomes reveals that the deep learning algorithm outperforms other methods in statistical measures. However, comparing regression models to deep strategies becomes less suitable since deep learning-based frameworks involve substantial scale datasets for learning and fine-tuning of hyperparameters [<xref ref-type="bibr" rid="ref-10">10</xref>] develops a hybrid-based approach to forecast the magnitude of the RTA dataset. The <italic>k</italic>-means clustering is being used in the study to aggregate crash datasets based on their similarities, and the random forest is being used to group road accident factors into intensity parameters, which increases class accuracy results for logistic regression, random forest, support vector machine (SVM), and <italic>k</italic>-nearest neighbor (KNN). [<xref ref-type="bibr" rid="ref-11">11</xref>] performed the exploration of traffic violation severity by defining the link among driver sex, age, years of driving, vehicle type, and traffic offense severity using bayesian network (BN), cumulative logistic regression (CLR), and neural network (NN) models. The performance comparison indicates the Bayesian network&#x0027;s performance as higher than other utilized approaches. Due to the limited data usage, the consequences of variables like climate and road conditions and analysis of the relation between traffic violations and road accidents for road accident predictions have not been taken into account. For identifying the determinants of road accidents and estimating the extent of road accidents, [<xref ref-type="bibr" rid="ref-12">12</xref>] applies different classification algorithms including J48, ID3, CART, and Na&#x00EF;ve Bayes. The outcome of the analysis shows that fatal accidents occur during rainy weather conditions that drive at midnight and serious accidents occur in foggy weather conditions in one-lane roads. The performance comparison indicates the predictability of J48 as higher than that of other categorization methods. Nevertheless, the modeling approach to ensembles individual statistical methods can further improve the crash seriousness prediction by integrating utilized learning models. The study [<xref ref-type="bibr" rid="ref-13">13</xref>] evolved macro-level collision prediction models utilizing decision tree regression (DTR) models to investigate pedestrian and bicycle collapse. The DTR models revealed major predictor variables in three broad categories: traffic, road, and socio-demographic characteristics. Furthermore, spatial predictor variables of neighboring crashes are considered along with the targeted crashes in both the DTR model&#x0027;s spatial and aspatial DTR models. The model comparison results revealed that the prediction accuracy of the spatial DTR model was higher than the aspatial DTR model. However, specific techniques (i.e., bagging, random forest, and gradient boosting) can be used to further increase the predictive performance of DTR models as they are known to be slow learners. The research study of [<xref ref-type="bibr" rid="ref-14">14</xref>] categorizes the severity of an accident into four types: deadly, grievous, simple damage, and motor collision. The severity of an accident is determined through the Decision Tree, <italic>k</italic>-nearest neighbors (KNN), na&#x00EF;ve Bays, and adaptive boosting algorithms of accidents in Bangladesh. Results found that the number of accidents gets increased based on the condition of surface effect features and at rush hour (06&#x2013;18) accident rate is very high as compared to other times. Among these four methods, healthy performance is attained by AdaBoost. Moreover, the precise parameters of hyper tuning of utilized statistical learning models can advance their performance. The research work of [<xref ref-type="bibr" rid="ref-15">15</xref>] utilizes the traffic and hazard information from a simulation experiment for each modeling stage to train the backpropagation neural network system. The model is a two-staged framework with the first stage identifies risk and no-risk status, and the second stage identifies high-risk and low-risk status. However, the simulation can not be completely optimized without real traffic data and better calibration. To predict the severity of the crash injury, a two-layer &#x201C;Stacking Framework&#x201D; is proposed by [<xref ref-type="bibr" rid="ref-16">16</xref>]. The first category incorporates the Random Forest, GBDT, and AdaBoost approach and the additional category achieves an accident injury level classification relying on a logistic regression model. The calibration phase automates different model specifications through a systematic grid search method. In association with many state-of-the-art approaches, the performance of the stacking model is healthier demonstrated by its precision and recall metric. Nevertheless, improving the quality of the accident datasets still requires further consideration to improve crash severity. [<xref ref-type="bibr" rid="ref-17">17</xref>] employed and compared several statistical learning techniques including Regression of Logistics, Random Forest, Adaptive Regression Multivariates, and the Support Vector Machines as well as the Bayesian neural network to deal with binary classification problems. An imbalanced high-resolution database of road accidents in Austria is used to analyze the consequences of 40 different incident variables. Findings showed that the tree-based ensemble is better than classical approaches such as logistic regression. The conclusions, however, support a compromise between accuracy and sensitivity inherent from the context of the inherently uneven existence of the data sets which challenge and complicate the study of the data sets. Their emphasis has been on investigating driver and pedestrian collisions, with little attention paid to the impact of machine learning precision in properly identifying major risks causing traffic injuries. Because of the current increase in accident frequency, there is a tremendous need for expanding road safety preventive research at this stage. As a result, we attempted to create a new hybrid system to characterizing traffic crash severity by integrating or coordinating &#x201C;Multinomial Logistic Regression&#x201D; MLR and &#x201C;Multilayer Perceptron&#x201D; MLP classifiers with a weighted majority voting scheme, which yields impressive prediction performance in road accident analysis. It offers implicit classification integration to obtain The findings of this research experiment do provide an understanding of the possible trigger factors that lead to traffic injury accidents. By determining the risk factors, these results will assist transportation institutions and police forces in reducing the serious or fatal injuries involved with traffic collisions. As a result, policymakers may enact new laws or upgrade road networks to reduce deadly or serious traffic incidents. As a result, the overall road collision casualties will be reduced. Moreover, this research work has an alliance with topical application fields as well [<xref ref-type="bibr" rid="ref-18">18</xref>&#x2013;<xref ref-type="bibr" rid="ref-21">21</xref>].</p>
</sec>
<sec id="s3"><label>3</label><title>Methodology</title>
<p>To identify the exact conditions related to particular casualty severity and to improve road safety, a multi-level hybrid framework of the WMV scheme is proposed to predict the accident casualty severity level of a particular RTA. The WMV scheme is designed with a parallel structure by integrating individually built classifiers into a hybrid system. The main concentration is on the impact of road, environment, and vehicle-related aspects for categorizing the casualty severity. The proposed WMV hybrid scheme consists of different phases as shown in <?A3B2 "fig2",5,"anchor"?><xref ref-type="fig" rid="fig-2">Fig. 2</xref>. The major steps of the proposed WMV hybrid scheme are as follows:
<list list-type="order">
<list-item><label>1.</label><p>Acquiring training examples of the multiple datasets.</p></list-item>
<list-item><label>2.</label><p>Performing data preprocessing using;
<list list-type="alpha-lower">
<list-item><p>Co-relation-based feature selection.</p></list-item>
<list-item><p>Synthetic minority oversampling technique (SMOTE).</p></list-item>
<list-item><p>Missing value replacement filter.</p></list-item>
<list-item><p>10 fold cross-validation with 60&#x0025;--40&#x0025; rule of training and testing.</p></list-item>
</list></p></list-item>
<list-item><label>3.</label><p>Performing model implementation using;
<list list-type="simple">
<list-item><label>a.</label><p>MLR.</p></list-item>
<list-item><label>b.</label><p>MLP.</p></list-item>
<list-item><label>c.</label><p>Hybrid modeling of WMV.</p></list-item>
</list></p></list-item>
<list-item><label>4.</label><p>Performing model evaluation using;
<list list-type="simple">
<list-item><label>a.</label><p>Testing.</p></list-item>
<list-item><label>b.</label><p>Predictions.</p></list-item>
</list></p></list-item>
<list-item><label>5.</label><p>Casualty severity type prediction.</p></list-item>
<list-item><label>6.</label><p>Go to step 2 for the next dataset.</p></list-item>
</list></p>
<fig id="fig-2"><label>Figure 2</label><caption><title>Block diagram of proposed approach of WMV</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_19404-fig-2.png"/></fig>
<p>The proposed approach considers accident severity analysis by observing the associations between the accident attributes and uniting the prediction decisions made by individually developed supervised learning algorithms. It finds out the best classification approach which can make an impact on overall classification accuracy for the &#x201C;Casualty Severity&#x201D; prediction. A software tool must be selected to exercise the utilization of distinct machine learning algorithms for different phases of casualty severity analysis. The software tool selected for this research study is &#x201C;Eclipse JAVA SDK&#x201D;. Classifiers implementation, validation, evaluation, and analysis are being performed in &#x201C;JAVA&#x201D; with &#x201C;WEKA JAR 3.8&#x201D;. WEKA comprises algorithms for data pre-processing, classification, regression, clustering, association rules, and visualization.</p>
<sec id="s3_1"><label>3.1</label><title>Accident Datasets Acquisition</title>
<p>The performance of the proposed approach is examined using the different datasets whose details are provided in the following subsequent sections:</p>
<sec id="s3_1_1"><label>3.1.1</label><title>IRTAD Dataset</title>
<p>Firstly, we selected the IRTAD dataset [<xref ref-type="bibr" rid="ref-22">22</xref>] and utilized the accident records from the year 2019 to 2020. It has 11,257 accident records with 26 unique accident features having accident severity classes named &#x201C;Fatal&#x201D;, &#x201C;Slight&#x201D; and &#x201C;Serious&#x201D; injury crashes, and distinct accident characteristics related to road_type, road_user age, gender, seat position plus environmental and weather conditions at the moment of the accident.</p>
</sec>
<sec id="s3_1_2"><label>3.1.2</label><title>Fatality Analysis Reporting System (FARS) Dataset</title>
<p>Secondly, we used FARS [<xref ref-type="bibr" rid="ref-23">23</xref>] dataset having each accident record that covers 38 information components characterizing the crash, cars, and the participating persons. The selected dataset includes data concerning the 35,029 records of motor vehicle accidents of the year 2019 to 2020 on the national motorways.</p>
</sec>
<sec id="s3_1_3"><label>3.1.3</label><title>National Collision Database (NCDB)</title>
<p>For the performance analysis of the proposed WMV hybrid scheme, a third utilized dataset is NCDB [<xref ref-type="bibr" rid="ref-24">24</xref>], which consists of a total of 28984 accident records with 20 distinct car, driver, and environmental prediction characteristics for the collisions in the complete year 2017.</p>
</sec>
</sec>
<sec id="s3_2"><label>3.2</label><title>Data Pre-Processing</title>
<p>As part of the proposed approach, firstly all the accident record features are being incorporated into a distinct data matrix form before using any data mining technique [<xref ref-type="bibr" rid="ref-25">25</xref>].</p>
<sec id="s3_2_1"><label>3.2.1</label><title>Synthetic Minority Oversampling</title>
<p>To create a balanced dataset that contains the equal representation of each target class, &#x201C;Resampling&#x201D; is performed by employing the synthetic minority oversampling technique (SMOTE) [<xref ref-type="bibr" rid="ref-26">26</xref>] as a pre-processing step.</p>
</sec>
<sec id="s3_2_2"><label>3.2.2</label><title>Filter for Substituting Lost Values</title>
<p>Lost values may be a communal issue on a larger level in real existence. The accident datasets <italic>IRTAD, FARS, and NCDB</italic> comprised limited entries where quantitative amounts of particular traits are lost. For example, an omitted value in the lightning conditions attributes specifies that there were no street lights present at the time of the crash. In this manner, to bring down the issue of lost values in other attributes, the &#x201C;Mean substitution-based imputation&#x201D; approach has been utilized to substitute the lost entries with measurable approximations of the adjacent entries. The selected substitution approach i.e., &#x201C;mean substitution-based imputation&#x201D; [<xref ref-type="bibr" rid="ref-27">27</xref>] figure out the average estimate of the features and custom this average estimate to supply the lost entry.</p>
</sec>
<sec id="s3_2_3"><label>3.2.3</label><title>Correlation Based Feature Selection</title>
<p>Furthermore, the pre-processing stage is followed by the correlation-based feature selection (CFS) [<xref ref-type="bibr" rid="ref-28">28</xref>] technique with the Greedy stepwise search [<xref ref-type="bibr" rid="ref-29">29</xref>] approach. The CFS is used to recognize and eliminate unwanted, inappropriate, and repeated features from the accident record. CFS identifies the features that are more significant and potential forecasters for predicting the target attribute. The CFS criterion is defined as follows:
<disp-formula id="eqn-1"><label>(1)</label><mml:math id="mml-eqn-1" display="block"><mml:mtable columnalign="left" rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>C</mml:mi><mml:mi>F</mml:mi><mml:mi>S</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:munder><mml:mo movablelimits="true" form="prefix">max</mml:mo><mml:mrow><mml:mi>s</mml:mi><mml:msub><mml:mrow></mml:mrow><mml:mi>k</mml:mi></mml:msub></mml:mrow></mml:munder></mml:mrow><mml:mo stretchy="false">[</mml:mo><mml:mstyle displaystyle="true" scriptlevel="0"><mml:mfrac><mml:mrow><mml:mi>r</mml:mi><mml:msub><mml:mrow></mml:mrow><mml:mi>c</mml:mi></mml:msub><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:mrow><mml:mo>+</mml:mo><mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mi>c</mml:mi></mml:msub></mml:mrow><mml:mi>f</mml:mi><mml:msub><mml:mrow></mml:mrow><mml:mn>2</mml:mn></mml:msub><mml:mo>+</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>+</mml:mo><mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mi>c</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mi>k</mml:mi></mml:msub></mml:mrow></mml:mrow><mml:mrow><mml:msqrt><mml:mi>k</mml:mi><mml:mo>+</mml:mo><mml:mn>2</mml:mn><mml:mo stretchy="false">(</mml:mo><mml:mi>r</mml:mi><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:mrow><mml:mo>+</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>+</mml:mo><mml:mi>r</mml:mi><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow><mml:mo>+</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>+</mml:mo><mml:mi>r</mml:mi><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mi>k</mml:mi></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mrow><mml:mi>k</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:msqrt></mml:mrow></mml:mfrac></mml:mstyle><mml:mo stretchy="false">]</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>
The <inline-formula id="ieqn-1"><mml:math id="mml-ieqn-1"><mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mrow><mml:mi>c</mml:mi><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mi>k</mml:mi></mml:msub></mml:mrow></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> is the average value of all feature classification correlations and <inline-formula id="ieqn-2"><mml:math id="mml-ieqn-2"><mml:mi>r</mml:mi><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mrow><mml:msub><mml:mi>f</mml:mi><mml:mi>j</mml:mi></mml:msub></mml:mrow></mml:mrow></mml:msub></mml:mrow></mml:math></inline-formula> is the average value of all feature-feature correlations.</p>
</sec>
<sec id="s3_2_4"><label>3.2.4</label><title>Dimensionality Reduction</title>
<p>Lastly, principal component analysis (PCA) [<xref ref-type="bibr" rid="ref-30">30</xref>] is applied as a pre-processing step to select the set of attributes combinations to reduce the data dimensions. PCA&#x0027;s adjustment is made by subtracting the variable&#x0027;s mean from each value. New variables which are called the factors or principal components are constructed as weighted averages of the original variables. Their specific values on a specific row are referred to as the scores. The matrix of scores is referred to as the matrix <italic>Y</italic>. The basic equation of PCA is, in matrix notation, given by:
<disp-formula id="eqn-2"><label>(2)</label><mml:math id="mml-eqn-2" display="block"><mml:mi>y</mml:mi><mml:mspace width="thickmathspace" /><mml:mi>i</mml:mi><mml:mi>j</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:msub><mml:mi>w</mml:mi><mml:mrow><mml:mn>1</mml:mn><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mrow><mml:mn>1</mml:mn><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo>+</mml:mo><mml:mrow><mml:msub><mml:mi>w</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mo>+</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>+</mml:mo><mml:mrow><mml:msub><mml:mi>w</mml:mi><mml:mrow><mml:mi>p</mml:mi><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mrow><mml:mi>p</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:math></disp-formula>where <italic>w</italic> is a matrix of coefficients that are determined by PCA with a data matrix as <italic>x</italic> which consists of n observations (rows) on <italic>p</italic> variables (columns).</p>
</sec>
<sec id="s3_2_5"><label>3.2.5</label><title>Statistical Resampling Using K-Fold Cross-Validation</title>
<p>Besides the strategies connected for dataset handling and classifiers, <italic>k</italic>-fold cross approval resampling [<xref ref-type="bibr" rid="ref-31">31</xref>] of a dataset is utilized in aggregation. The procedure is utilized to part the input dataset into preparing and test information. Preparing information is utilized to instruct the dataset whereas test information is utilized to assess the trained classifier. We have chosen a number of folds to be 10 as a cross-validation method with 10-folds, partition the input dataset subjectively into 10 identical small-scale divisions. Furthermore, in any cross-validation using <italic>k</italic>-fold, commonly one test is utilized as approval data/testing information whereas the remaining <italic>k&#x2212;</italic>1 tests of information are used as preparing information. The sampling technique is mathematically expressed as follows:
<disp-formula id="eqn-3"><label>(3)</label><mml:math id="mml-eqn-3" display="block"><mml:mi>C</mml:mi><mml:mi>V</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mrow><mml:msup><mml:mi>f</mml:mi><mml:mrow><mml:mo>&#x2227;</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>N</mml:mi></mml:mfrac><mml:munderover><mml:mrow><mml:mo movablelimits="false">&#x2211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>N</mml:mi></mml:munderover><mml:mo>&#x2061;</mml:mo><mml:mi>L</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mrow><mml:msub><mml:mi>y</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mo>,</mml:mo></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:msup><mml:mi>f</mml:mi><mml:mrow><mml:mo>&#x2227;</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mi>k</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>i</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:msup></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula>where <inline-formula id="ieqn-3"><mml:math id="mml-ieqn-3"><mml:mi>k</mml:mi></mml:math></inline-formula>: &#x007B;<italic>1,&#x2026;, N</italic>&#x007D; be an indexing function that indicates the partition to which observation <italic>i</italic> is allocated by randomization. <inline-formula id="ieqn-4"><mml:math id="mml-ieqn-4"><mml:mrow><mml:msup><mml:mi>f</mml:mi><mml:mrow><mml:mo>&#x2227;</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mi>k</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>i</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:msup></mml:mrow></mml:math></inline-formula> is the fitted function. Typical Choices of <italic>k</italic> are 5 to 10. In the case <italic>k(i) &#x003D; i</italic> and for the <inline-formula id="ieqn-5"><mml:math id="mml-ieqn-5"><mml:mrow><mml:msup><mml:mi>i</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:math></inline-formula> observation, the fit is computed using all the data except the <inline-formula id="ieqn-6"><mml:math id="mml-ieqn-6"><mml:mrow><mml:msup><mml:mi>i</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:mrow></mml:msup></mml:mrow></mml:math></inline-formula> observation.</p>
</sec>
</sec>
</sec>
<sec id="s4"><label>4</label><title>Predictive Modelling</title>
<p>To learn high-level representations from the data and classifying the injury severity of road traffic accidents, this research proposes the implementation of Multi-Model network architectures including two distinct machine learning classifiers i.e., MLR, MLP which are trained and implemented using the attributes selected by PCA and CFS using 10-fold cross-validation technique.</p>
<sec id="s4_1"><label>4.1</label><title>Multinomial Logistic Regression Classifier</title>
<p>The first utilized classifier as part of the predictive modeling stage is the MLR classifier [<xref ref-type="bibr" rid="ref-32">32</xref>]. In statistics, MLR is a classification method that generalizes logistic regression to multiclass problems, i.e., with more than two possible discrete outcomes. It is a model that is used to predict the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables. The objective is to construct as a linear predictor function that constructs a score from a set of weights that are linearly combined with the explanatory variables (features) of a given observation using a dot product, mathematically defined as follows:
<disp-formula id="eqn-4"><label>(4)</label><mml:math id="mml-eqn-4" display="block"><mml:mi>S</mml:mi><mml:mi>c</mml:mi><mml:mi>o</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mspace width="thickmathspace" /></mml:mrow></mml:msub></mml:mrow><mml:mo>,</mml:mo><mml:mspace width="thickmathspace" /><mml:mi>k</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mrow><mml:msub><mml:mi>B</mml:mi><mml:mi>k</mml:mi></mml:msub></mml:mrow><mml:mspace width="thickmathspace" /><mml:mo>.</mml:mo><mml:mspace width="thickmathspace" /><mml:mi>X</mml:mi></mml:math></disp-formula>where <inline-formula id="ieqn-7"><mml:math id="mml-ieqn-7"><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:math></inline-formula> is the vector of explanatory variables describing observation <italic>i</italic>, <inline-formula id="ieqn-8"><mml:math id="mml-ieqn-8"><mml:mrow><mml:msub><mml:mi>&#x03B2;</mml:mi><mml:mi>k</mml:mi></mml:msub></mml:mrow></mml:math></inline-formula> is a vector of weights (or regression coefficients) corresponding to outcome <italic>k</italic>, and score <inline-formula id="ieqn-9"><mml:math id="mml-ieqn-9"><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo>,</mml:mo><mml:mspace width="thickmathspace" /><mml:mi>k</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is the score associated with assigning observation <italic>i</italic> to category <italic>k</italic>. As a part of the model implementation and classification phase, chosen classifier MLR is applied on all three distinct accident datasets to predict our target attribute &#x201C;Casualty Severity&#x201D; which is nominal and having three values i.e., &#x201C;Slight&#x201D;. &#x201C;Serious&#x201D; and &#x201C;Fatal&#x201D;. Classification using MLR has been performed with the following parameters: a maximum number of iterations is set to 10 and &#x201C;Ridge Value&#x201D; in the &#x201C;log-likelihood&#x201D; set to &#x201C;1.0E&#x2212;8&#x201D;.</p>
</sec>
<sec id="s4_2"><label>4.2</label><title>Multilayer Perceptron Classifier</title>
<p>One of the categories of feedforward artificial neural network (ANN) is the multilayer perceptron (MLP) classifier [<xref ref-type="bibr" rid="ref-33">33</xref>]. It comprises at least ternary layers of connection: an input tier, a concealed tier, and a turnout tier. But for the input connections, each connection could be a neuron that employs a nonlinear actuation function. Controlled learning practice named &#x201C;Backpropagation&#x201D; is employed by MLP for preparing the input data. In case an MLP encompasses a straight actuation work in all neurons, that&#x0027;s, a direct work that maps the weighted inputs to the outcome of each neuron, at that point direct variable-based calculations show that any number of layers can be diminished to a two-layer input-output classifier. The historically common activation function is the sigmoid function and is mathematically defined by:
<disp-formula id="eqn-5"><label>(5)</label><mml:math id="mml-eqn-5" display="block"><mml:mi>y</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mrow><mml:msub><mml:mi>v</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>t</mml:mi><mml:mi>a</mml:mi><mml:mi>n</mml:mi><mml:mi>h</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mrow><mml:msub><mml:mi>v</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula>where <inline-formula id="ieqn-10"><mml:math id="mml-ieqn-10"><mml:mrow><mml:msub><mml:mi>y</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:math></inline-formula> the output of the ith node (neuron) and <inline-formula id="ieqn-11"><mml:math id="mml-ieqn-11"><mml:mrow><mml:msub><mml:mi>v</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow></mml:math></inline-formula> is the weighted sum of the input connections. It is a hyperbolic tangent that ranges from &#x2212;1 to 1, while the other is the logistic function, which is similar in shape but ranges from 0 to 1. As a second classification model, the MLP classifier is MLP model is applied on accident records from selected accident datasets to predict the outcome i.e., Casualty Severity. MLP classifier performed the classification with the parameter settings as follows: &#x201C;hidden layers&#x201D; to be calculated as &#x201C;a&#x0027;&#x003D;(attributes&#x002B;classes)/2&#x201D;, &#x201C;decay&#x201D; is set to false to increase the learning rate, &#x201C;normalizeAttributes&#x201D; is set to &#x201C;True&#x201D; that will normalize the attributes, &#x201C;learning rate&#x201D; is set to 0.3 and &#x201C;momentum&#x201D; is set to 0.2.</p>
</sec>
<sec id="s4_3"><label>4.3</label><title>Hybrid Model of Weighted Majority Voting Scheme</title>
<p>The second stage of the proposed approach employs the hybrid classification approach which is preferably employed because of its better accuracy and precision as compared to an individual classifier. Rendering to the &#x201C;No Free Lunch Theorem&#x201D; [<xref ref-type="bibr" rid="ref-34">34</xref>] the finest machine learning technique which is best for any prediction problem doesn&#x0027;t exist. So, the integration of various classifiers in the form of a hybrid classification model provides better results as specified by [<xref ref-type="bibr" rid="ref-35">35</xref>]. <xref ref-type="fig" rid="fig-2">Fig. 2</xref> illustrates the hybrid modeling architecture for unknown &#x201C;Casualty Severity&#x201D; prediction.</p>
<fig id="fig-3"><label>Figure 3</label><caption><title>Architecture of the hybrid modeling of the proposed approach</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_19404-fig-3.png"/></fig>
<p>The major approaches of hybrid modeling being utilized are cascading [<xref ref-type="bibr" rid="ref-36">36</xref>], hierarchical [<xref ref-type="bibr" rid="ref-37">37</xref>], and parallel [<xref ref-type="bibr" rid="ref-38">38</xref>] (<xref ref-type="fig" rid="fig-3">Fig. 3</xref>). In cascading approach, the outcome of one classifier is used as feedback for the subsequent classifier to perform the classification. Employing a collection of two-fold classification methods organized by way of a &#x201C;tree&#x201D; in class orbit, a hierarchical classification strategy resolves multiple categorical complexities in higher dimensional areas. In contrast to that, the parallel approach receives an identical input for all the selected models and combine their outcome employing specified decision reasoning. As discussed in [<xref ref-type="bibr" rid="ref-39">39</xref>] decision reasoning can be direct which includes average and weighted average of the outcomes or indirect that includes voting, probabilistic, and rank cantered approaches. In the proposed approach, the parallel ensemble approach is being utilized to integrate the individual classifiers with the WMV scheme. The proposed WMV hybrid scheme also performs well in a case where all individual classifiers provide less effective results. A weight factor is assigned to each model. For each model, predicted class likelihoods are accumulated, then its product is taken with the model&#x0027;s weight, and the average is calculated. Based on these weighted average likelihoods, the class tag is assigned using the mathematical equation as follows:
<disp-formula id="eqn-6"><label>(5)</label><mml:math id="mml-eqn-6" display="block"><mml:mrow><mml:msup><mml:mi>Y</mml:mi><mml:mrow><mml:mo>&#x2227;</mml:mo></mml:mrow></mml:msup></mml:mrow><mml:mo>=</mml:mo><mml:munderover><mml:mrow><mml:mo movablelimits="false">&#x2211;</mml:mo></mml:mrow><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mi>n</mml:mi></mml:munderover><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:msub><mml:mi>W</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mspace width="thickmathspace" /><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>s</mml:mi></mml:msub></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mrow><mml:msub><mml:mi>C</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>j</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula>where<inline-formula id="ieqn-12"><mml:math id="mml-ieqn-12"><mml:mspace width="thickmathspace" /><mml:mrow><mml:msub><mml:mi>X</mml:mi><mml:mi>S</mml:mi></mml:msub></mml:mrow></mml:math></inline-formula> is the attribute function <inline-formula id="ieqn-13"><mml:math id="mml-ieqn-13"><mml:mo stretchy="false">[</mml:mo><mml:mrow><mml:mrow><mml:msub><mml:mi>C</mml:mi><mml:mi>i</mml:mi></mml:msub></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>j</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>S</mml:mi></mml:mrow><mml:mo stretchy="false">]</mml:mo></mml:math></inline-formula> and <italic>S</italic> is the collection of distinct target class values. After the classification is performed by individual models, it is a prerequisite to reduce their distinct faults by using a WMV Hybrid classification approach. Henceforth, the hybrid model is created by integrating the individual classifiers that are MLR and MLP. To evaluate the combined effort of all the selected classifiers, the proposed WMV Hybrid model is applied on all three accident datasets to predict our target attribute i.e., Casualty Severity. A hybrid model is designed using a training dataset with parameter settings as follows: seed value is set to &#x201C;2&#x201D; which is a random number seed to be used, &#x201C;batch size&#x201D; is set to &#x201C;100&#x201D; which is the preferred number of instances to process, this combination rule is termed as &#x201C;Weighted Majority Voting&#x201D;.</p>
</sec>
<sec id="s4_4"><label>4.4</label><title>Model Testing and Predictions</title>
<p>This section provides the details about testing activity that has been performed for each classifier and the WMV Hybrid model for Casualty Severity prediction of road traffic accident datasets <italic>IRTAD, FARS, and NCDB</italic> using the 10 fold cross-validation method. With this method, we produced one data set of every accident dataset which we divided randomly into 10 parts. We used 9 of those parts for training and reserved one-tenth for testing. We repeated this procedure 10 times, each time reserving a different tenth for testing. Through this method, we have performed out-of-sample testing, for assessing that how well each individual and hybrid model generalizes to an independent dataset to obtain the best prediction accuracy.</p>
</sec>
<sec id="s4_5"><label>4.5</label><title>Model Evaluation and Discussions</title>
<p>After performing the testing and predictions activity, the next phase is to provide the model evaluation with a performance comparison of the proposed framework for Casualty Severity prediction of road traffic accident datasets <italic>IRTAD, FARS, and NCDB</italic> using individual classifiers and the WMV Hybrid model. A quantitative analysis of the results generated by the proposed framework is also presented here. We correspondingly presented the evaluation comparison of the anticipated WMV Hybrid model and individually implemented classification approaches i.e., MLR and MPL on selected accident datasets.</p>
<sec id="s4_5_1"><label>4.5.1</label><title>Quantitative Result Evaluation</title>
<p>To begin with the experimental evaluation, we first utilized the empirical assessment measures i.e., &#x201C;Precision&#x201D;, &#x201C;Recall&#x201D;, &#x201C;Classification Accuracy&#x201D;, &#x201C;Mean Absolute Error&#x201D; (MAE), &#x201C;Root Mean Squared Error&#x201D; (RMSE), and &#x201C;Relative Absolute Error&#x201D; (RAE) for execution assessment. Upon all three accident records <italic>IRTAD, FARS, and NCDB</italic>, the proposed WMV Hybrid method achieves the highest precision and recall rate respectively of 0.894, 0.996, and lowest MAE and RMSE of 0.0731, 0.2705 which is above the implemented classifiers on the IRTAD dataset as presented in <?A3B2 "tbl1",5,"anchor"?><xref ref-type="table" rid="table-1">Tab. 1</xref>.</p>
<table-wrap id="table-1"><label>Table 1</label><caption><title>Performance evaluation of the proposed WMV hybrid scheme on different datasets</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="center" colspan="6">General assessment profile</th>
</tr>
<tr>
<th align="left">Selected approaches</th>
<th align="left">Name of dataset</th>
<th align="left">Precision</th>
<th align="left">Recall</th>
<th align="left">MAE</th>
<th align="left">RMSE</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" rowspan="3">MLR</td>
<td align="left">IRTAD</td>
<td align="left">0.892</td>
<td align="left">0.994</td>
<td align="left">0.1176</td>
<td align="left">0.3439</td>
</tr>
<tr>
<td align="left">FARS</td>
<td align="left">0.712</td>
<td align="left">0.814</td>
<td align="left">0.2710</td>
<td align="left">0.3110</td>
</tr>
<tr>
<td align="left">NCDB</td>
<td align="left">0.793</td>
<td align="left">0.894</td>
<td align="left">0.2176</td>
<td align="left">0.2839</td>
</tr>
<tr>
<td align="left" rowspan="3">MLP</td>
<td align="left">IRTAD</td>
<td align="left">0.894</td>
<td align="left">0.892</td>
<td align="left">0.2249</td>
<td align="left">0.3312</td>
</tr>
<tr>
<td align="left">FARS</td>
<td align="left">0.811</td>
<td align="left">0.901</td>
<td align="left">0.1420</td>
<td align="left">0.2715</td>
</tr>
<tr>
<td align="left">NCDB</td>
<td align="left">0.601</td>
<td align="left">0.881</td>
<td align="left">0.1120</td>
<td align="left">0.2922</td>
</tr>
<tr>
<td align="left" rowspan="3">Proposed WMV hybrid scheme</td>
<td align="left">IRTAD</td>
<td align="left">0.895</td>
<td align="left">0.996</td>
<td align="left">0.0731</td>
<td align="left">0.2705</td>
</tr>
<tr>
<td align="left">FARS</td>
<td align="left">0.810</td>
<td align="left">0.921</td>
<td align="left">0.1320</td>
<td align="left">0.2850</td>
</tr>
<tr>
<td align="left">NCDB</td>
<td align="left">0.884</td>
<td align="left">0.794</td>
<td align="left">0.1752</td>
<td align="left">0.2809</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="s4_5_2"><label>4.5.2</label><title>Performance Comparison</title>
<p>For the subsequent evaluation approach, confusion matrix [<xref ref-type="bibr" rid="ref-40">40</xref>] examination is undertaken to exhibit the accurateness of predicted estimates performed by selected approaches. Along with the prediction accuracy, other evaluation metrics i.e., F-measure and ROC Area and Kappa metric [<xref ref-type="bibr" rid="ref-41">41</xref>] are also being utilized to access the performance of implemented methods. Kappa Rate is a measure of agreement between the predictions and the real outputs. ROC area provides an effective way to choose better classifiers and reject others. A perfect prediction model generates ROC area rate approaches towards 1. It represents the assessment of the total accuracy to the estimated random chance accuracy. Kappa rate larger than 0 indicates that the model performs better than the random chance classifier of the proposed method and individual classifiers for each target class on selected datasets. Although all of the implemented classifiers performed reasonably well, however, the proposed WMV Hybrid classifier outperformed the individual classifiers for the &#x201C;Casualty Severity&#x201D; prediction of an RTA. The comparison between the reported metrics using the MLR, MLP, and proposed WMV Hybrid model is presented in <?A3B2 "tbl2",5,"anchor"?><xref ref-type="table" rid="table-2">Tab. 2</xref>. As classifiers with the highest prediction accuracy, F-Measure and ROC area are preferred. So, the outcomes of implementing the specified framework indicate that the proposed WMV Hybrid approach attains the uppermost prediction accuracy of 89.0281&#x0025;, F-measure of 0.942, Roc Area of 0.513, and Kappa Rate of 0.0395 among the implemented classifiers on the IRTAD dataset as presented in <xref ref-type="table" rid="table-2">Tab. 2</xref>.</p>
<table-wrap id="table-2"><label>Table 2</label><caption><title>Performance analysis of the proposed approach in terms of performance parameters</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Selected approaches</th>
<th align="left">Name of dataset</th>
<th align="left">Prediction accuracy</th>
<th align="left">F-Measure</th>
<th align="left">Roc Area</th>
<th align="left">Kappa rate</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" rowspan="3">MLR</td>
<td align="left">IRTAD</td>
<td align="left">88.7971&#x0025;</td>
<td align="left">0.941</td>
<td align="left">0.553</td>
<td align="left">0.0211</td>
</tr>
<tr>
<td align="left">FARS</td>
<td align="left">87.591&#x0025;</td>
<td align="left">0.850</td>
<td align="left">0.511</td>
<td align="left">0.0143</td>
</tr>
<tr>
<td align="left">NCDB</td>
<td align="left">87.247&#x0025;</td>
<td align="left">0.912</td>
<td align="left">0.652</td>
<td align="left">0.0202</td>
</tr>
<tr>
<td align="left" rowspan="3">MLP</td>
<td align="left">IRTAD</td>
<td align="left">88.8149&#x0025;</td>
<td align="left">0.941</td>
<td align="left">0.542</td>
<td align="left">0.0329</td>
</tr>
<tr>
<td align="left">FARS</td>
<td align="left">88.8012&#x0025;</td>
<td align="left">0.752</td>
<td align="left">0.421</td>
<td align="left">0.0451</td>
</tr>
<tr>
<td align="left">NCDB</td>
<td align="left">85.214&#x0025;</td>
<td align="left">0.779</td>
<td align="left">0.665</td>
<td align="left">0.0321</td>
</tr>
<tr>
<td align="left" rowspan="3">Proposed WMV hybrid scheme</td>
<td align="left">IRTAD</td>
<td align="left">89.0281&#x0025;</td>
<td align="left">0.942</td>
<td align="left">0.751</td>
<td align="left">0.0629</td>
</tr>
<tr>
<td align="left">FARS</td>
<td align="left">89.0132&#x0025;</td>
<td align="left">0.889</td>
<td align="left">0.701</td>
<td align="left">0.0612</td>
</tr>
<tr>
<td align="left">NCDB</td>
<td align="left">88.8241&#x0025;</td>
<td align="left">0.901</td>
<td align="left">0.712</td>
<td align="left">0.0541</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="s4_5_3"><label>4.5.3</label><title>Relative Assessment of the Presented and Prevailing Approaches</title>
<p>As the third step of experimental evaluation, a relative assessment of the presented and prevailing approaches for accident severity prediction in terms of precision has been specified. In this regard, <?A3B2 "tbl3",5,"anchor"?><xref ref-type="table" rid="table-3">Tab. 3</xref> below projects the comprehensive evaluation. Since the assessment outcomes undoubtedly confirm the ascendance of a specified framework as it overtakes all prevailing methods in terms of prediction for unknown &#x201C;Casualty Severity&#x201D; of an RTA. This indicates the validity of the presented WMV Hybrid classifier in terms of accurate prediction of an unknown &#x201C;Casualty Severity&#x201D;. The subsequent fine technique of SVG and GMM attains the values of precision as 0.98. Lastly, the flawed approach of the fuzzy decision obtained the lowest precision of 0.61.</p>
<table-wrap id="table-3"><label>Table 3</label><caption><title>The performance comparison of the proposed WMV hybrid scheme with different state-of-the-art approaches</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Statistical approach</th>
<th align="left">Model</th>
<th align="left">Executional assessment approach</th>
<th align="left">Precision</th>
<th align="left">Accuracy</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Fuzzy decision approaches [<xref ref-type="bibr" rid="ref-42">42</xref>]</td>
<td align="left">DT, Bayesian network, and linear SVM</td>
<td align="left">OOB C-Statistics</td>
<td align="left">0.61</td>
<td align="left">69.96&#x0025;</td>
</tr>
<tr>
<td align="left">Hierarchical modeling scheme [<xref ref-type="bibr" rid="ref-43">43</xref>]</td>
<td align="left">Random Forest (RF), Support Vector Machine (SVM), and Neural<break/>Network (NN)</td>
<td align="left">accuracy (ACC) and<break/>precision (PPV)</td>
<td align="left">0.81</td>
<td align="left">81.37&#x0025;</td>
</tr>
<tr>
<td align="left">Qualitative modeling [<xref ref-type="bibr" rid="ref-44">44</xref>]</td>
<td align="left">Logistic regression model</td>
<td align="left">Cox &#x0026; Snell R-square<break/>&#x0026; Log-likelihood</td>
<td align="left">0.69</td>
<td align="left">70.00&#x0025;</td>
</tr>
<tr>
<td align="left">Hybrid model [<xref ref-type="bibr" rid="ref-45">45</xref>]</td>
<td align="left">SVG &#x0026; GMM</td>
<td align="left">RBF kernel performance metrics</td>
<td align="left">0.98</td>
<td align="left">88.5167&#x0025;</td>
</tr>
<tr>
<td align="left">Proposed WMV hybrid scheme</td>
<td align="left">WMV Hybrid of MLR and MLP Model</td>
<td align="left">Goodness-of-fit</td>
<td align="left">1.00</td>
<td align="left">89.0281&#x0025;</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="s4_5_4"><label>4.5.4</label><title>Computational Complexity Comparison</title>
<p>As the final step of experimental evaluation, computational complexity comparison between the proposed WMV Hybrid classifier and prevailing approaches has been carried out. In this regard, <?A3B2 "tbl4",5,"anchor"?><xref ref-type="table" rid="table-4">Tab. 4</xref> below projects the comparative evaluation. The results showed that, while the computation time of adaptive algorithms differs slightly, the proposed approach is carried out through hybrid integration of two individual models MLR and MLP using a weighted majority voting scheme, resulting in more complex results with higher prediction accuracy than previous studies that used individual algorithms. It produces rapid global and precise local optimal findings, displaying a better comprehension of the entire model description, and finally, the feature analysis revealed that non-road-related variables, notably driver variables, are more essential than highway variables. The methodology established in this work can be extended to big data predictive analytics of road accident fatalities and used by traffic policy regulators and traffic safety experts as a rapid tool.</p>
<table-wrap id="table-4"><label>Table 4</label><caption><title>Computational complexity comparison of the proposed WMV hybrid scheme with existing approaches</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left" rowspan="2">Severity prediction approaches</th>
<th align="center" colspan="5">Complexity attributes</th>
</tr>
<tr>
<th align="left">RAM</th>
<th align="left">Implementation tool</th>
<th align="left">Processor</th>
<th align="left">Hard disk</th>
<th align="left">Model computation time<break/>(Seconds)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Proposed WMV hybrid scheme</td>
<td align="left">6GB</td>
<td align="left">JDMP [<xref ref-type="bibr" rid="ref-46">46</xref>]<break/>WEKA [<xref ref-type="bibr" rid="ref-47">47</xref>]</td>
<td align="left">Core i7</td>
<td align="left">600GB</td>
<td align="left">155.96</td>
</tr>
<tr>
<td align="left">Fuzzy decision approaches</td>
<td align="left">6GB</td>
<td align="left">Matlab</td>
<td align="left">NA</td>
<td align="left">NA</td>
<td align="left">159.41</td>
</tr>
<tr>
<td align="left">Hierarchical modeling scheme</td>
<td align="left">5GB</td>
<td align="left">WEKA</td>
<td align="left">Core i5</td>
<td align="left">500</td>
<td align="left">165.52</td>
</tr>
<tr>
<td align="left">Qualitative modeling</td>
<td align="left">NA</td>
<td align="left">WEKA</td>
<td align="left">NA</td>
<td align="left">NA</td>
<td align="left">155.99</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="s4_5_5"><label>4.5.5</label><title>Knowledge Representation</title>
<p>The findings of result evaluation and performance comparisons show that the age group of 18 to 30 years is identified as the most vulnerable age group involved in traffic accidents of various severity levels. Besides, the Type of Vehicle is also found to be an important factor to discriminate among different casualty severities. Mostly 50cc motorcycles are found to be involved in Slight casualty accidents and 500cc motorcycles are identified to be involved in Serious casualty accidents. Moreover, Goods vehicles with 7.5 tons are identified to be involved in fatal accidents. Most of the female drivers are observed to be involved in slight casualty crashes and male drivers are being involved in both serious and fatal casualty accidents. The road type and road surface condition are also found to be distinguishing attributes for predicting the &#x201C;Casualty Severity&#x201D; as both attributes show the highest classifier related to target attribute generated weights by MLR. The proposed approach provides a comprehensive analysis and findings of important factors that cause accidents of different severity levels.</p>
</sec>
</sec>
</sec>
<sec id="s5"><label>5</label><title>Conclusion and Future Work</title>
<p>The proposed approach analyzes the RTA records to discover the underlying patterns responsible for a particular type of causality severity that occurs in road traffic accidents. Individually designed prediction models and proposed WMV Hybrid model predict the unknown &#x201C;Casualty Severity&#x201D; of an accident from a selected accident recordset. Performance comparison indicates the proposed WMV Hybrid classifier as the best prediction approach due to its enhanced evaluation statistics including precision and prediction accuracy as compared to individual classifiers on selected accident datasets. The results of the proposed WMV hybrid model support the road safety policymakers for rendering their decisions in the identification of the most critical aspect related to &#x201C;Casualty Severity&#x201D;. Finally, the consequences of this research provide the prospective study related to accident severity analysis using hybrid machine learning techniques, and different security issues [<xref ref-type="bibr" rid="ref-48">48</xref>&#x2013;<xref ref-type="bibr" rid="ref-51">51</xref>] particularly in the perspective of highway safety. A possible future work direction is to further expand the current research work using deep learning approaches like recurrent neural networks (RNN) and convolutional neural networks (CNN) that require learning at different layers with substantial accident data to get the profound insight analysis of potential risk factors of a road accident. This has the potential to be a useful method for estimating the seriousness of injuries in road collisions.</p>
</sec>
</body>
<back>
<fn-group>
<fn fn-type="other"><p><bold>Funding Statement:</bold> The authors received no specific funding for this study.</p></fn>
<fn fn-type="conflict"><p><bold>Conflicts of Interest:</bold> The authors declare that they have no conflicts of interest to report regarding the present study.</p></fn>
</fn-group>
<ref-list content-type="authoryear">
<title>References</title>
<ref id="ref-1"><label>[1]</label><mixed-citation publication-type="web"><person-group person-group-type="author"><collab>WHO</collab></person-group>, &#x201C;<article-title>Global status report on road safety 2018</article-title>,&#x201D; <source>World Health Organization</source>, no. <issue>WHO/NMH/NVI/18.20</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>20</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-2"><label>[2]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>P. B.</given-names> <surname>Silva</surname></string-name>, <string-name><given-names>P.</given-names> <surname>Barbosa</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Andrade</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Ferreira</surname></string-name></person-group>, &#x201C;<article-title>Machine learning applied to road safety modeling: A systematic literature review</article-title>,&#x201D; <source>Journal of Traffic and Transportation Engineering (English Edition): Elsevier</source>, vol. <volume>7</volume>, no. <issue>6</issue>, pp. <fpage>775</fpage>&#x2013;<lpage>790</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-3"><label>[3]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>L.</given-names> <surname>Eboli</surname></string-name>, <string-name><given-names>G.</given-names> <surname>Mazzulla</surname></string-name> and <string-name><given-names>C.</given-names> <surname>Forciniti</surname></string-name></person-group>, &#x201C;<article-title>Factors influencing accident severity: An analysis by road accident type</article-title>,&#x201D; <source>Transportation Research Procedia:Elsevier</source>, vol. <volume>47</volume>, no. <issue>3</issue>, pp. <fpage>449</fpage>&#x2013;<lpage>456</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-4"><label>[4]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>N.</given-names> <surname>Mor</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Sood</surname></string-name> and <string-name><given-names>T.</given-names> <surname>Goyal</surname></string-name></person-group>, &#x201C;<chapter-title>A critical review on use of data mining technique for prediction of road accidents</chapter-title>,&#x201D; in <source>Computational Methods and Data Engineering</source>, <publisher-loc>Netherlands</publisher-loc>: <publisher-name>Springer</publisher-name>, vol. <volume>7</volume>, no. <issue>2</issue>, pp. <fpage>141</fpage>&#x2013;<lpage>149</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-5"><label>[5]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>S. A.</given-names> <surname>Joshi</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Senanayake</surname></string-name>, <string-name><given-names>S. A.</given-names> <surname>Prasad</surname></string-name>, <string-name><given-names>P.</given-names> <surname>Yong</surname></string-name>, <string-name><given-names>S. Y.</given-names> <surname>Elchouemi</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>Pattern mining predictor system for road accidents</article-title>,&#x201D; in <conf-name>Proc. Int. Conf. on Computational Collective Intelligence</conf-name>, <publisher-name>Springer</publisher-name>, <conf-loc>Da Nang, Vietnam</conf-loc>, vol. <volume>1287</volume><conf-loc>, no.</conf-loc> <issue>3</issue>, pp. <fpage>605</fpage>&#x2013;<lpage>615</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-6"><label>[6]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>C.</given-names> <surname>Gutierrez-Osorio</surname></string-name> and <string-name><given-names>C.</given-names> <surname>Pedraza</surname></string-name></person-group>, &#x201C;<chapter-title>Modern data sources and techniques for analysis and forecast of road accidents: A review</chapter-title>,&#x201D; in <source>Journal of Traffic and Transportation Engineering (English Edition)</source>, <publisher-loc>Peoples R China</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>7</volume>, no. <issue>4</issue>, pp. <fpage>432</fpage>&#x2013;<lpage>446</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-7"><label>[7]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Pradhan</surname></string-name> and <string-name><given-names>M. I.</given-names> <surname>Sameen</surname></string-name></person-group>, &#x201C;<chapter-title>Predicting injury severity of road traffic accidents using a hybrid extreme gradient boosting and deep neural network approach</chapter-title>,&#x201D; in <source>Laser Scanning Systems in Highway and Safety Assessment</source>, <edition>1st ed</edition>. <publisher-loc> Switzerland</publisher-loc>: <publisher-name>Springer</publisher-name>, vol. <volume>1</volume>, no. <issue>3</issue>, pp. <fpage>119</fpage>&#x2013;<lpage>127</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-8"><label>[8]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>R. F. N.</given-names> <surname>Nielsen</surname></string-name>, <string-name><given-names>N.</given-names> <surname>Sillesen</surname></string-name>, <string-name><given-names>L. W.</given-names> <surname>Andersson</surname></string-name>, <string-name><given-names>M. P.</given-names> <surname>Gernaey</surname></string-name>, <string-name><given-names>K.</given-names> <surname>V</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>Hybrid machine learning assisted modelling framework for particle processes</article-title>,&#x201D; <source>Computers &#x0026; Chemical Engineering:Elsevier</source>, vol. <volume>140</volume>, no. <issue>4</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>19</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-9"><label>[9]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Ijaz</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Zahid</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Jamal</surname></string-name></person-group>, &#x201C;<article-title>A comparative study of machine learning classifiers for injury severity prediction of crashes involving three-wheeled motorized rickshaw</article-title>,&#x201D; <source>Accident Analysis &#x0026; Prevention: Elsevier</source>, vol. <volume>154</volume>, no. <issue>3</issue>, pp. <fpage>106094</fpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-10"><label>[10]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S. S.</given-names> <surname>Yassin</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Seid</surname></string-name></person-group>, &#x201C;<article-title>Road accident prediction and model interpretation using a hybrid K-means and random forest algorithm approach</article-title>,&#x201D; <source>SN Applied Sciences: Springer</source>, vol. <volume>2</volume>, no. <issue>9</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>13</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-11"><label>[11]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Saracoglu</surname></string-name>, <string-name><surname>Abdulsamet</surname> and <given-names>H.</given-names></string-name> <string-name><surname>Ozen</surname></string-name></person-group>, &#x201C;<chapter-title>Estimation of traffic incident duration: A comparative study of decision tree models</chapter-title>,&#x201D; <source>Arabian Journal for Science and Engineering</source>, <publisher-loc>Germany</publisher-loc>: <publisher-name>Springer</publisher-name>, vol. <volume>45</volume>, no. <issue>3</issue>, pp. <fpage>8099</fpage>&#x2013;<lpage>8110</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-12"><label>[12]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>T. K.</given-names> <surname>Bahiru</surname></string-name>, <string-name><given-names>D. K.</given-names> <surname>Singh</surname></string-name> and <string-name><given-names>E. A.</given-names> <surname>Tessfaw</surname></string-name></person-group>, &#x201C;<article-title>Comparative study on data mining classification algorithms for predicting road traffic accident severity</article-title>,&#x201D; in <conf-name>Proc. Int. Conf. on Computation, Automation and Knowledge Management (ICCAKM)</conf-name>, <conf-loc>Dubai, United Arab Emirates</conf-loc>, pp. <fpage>373</fpage>&#x2013;<lpage>376</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-13"><label>[13]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Zhu</surname></string-name></person-group>, &#x201C;<article-title>Analysis of the severity of vehicle-bicycle crashes with data mining techniques</article-title>,&#x201D; <source>Journal of Safety Research: Elsevier</source>, vol. <volume>76</volume>, no. <issue>3</issue>, pp. <fpage>218</fpage>&#x2013;<lpage>227</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-14"><label>[14]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>P.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Abdel-Aty</surname></string-name> and <string-name><given-names>J.</given-names> <surname>Yuan</surname></string-name></person-group>, &#x201C;<chapter-title>Real-time crash risk prediction on arterials based on LSTM-CNN</chapter-title>,&#x201D; in <source>Accident Analysis and Prevention</source>, <publisher-loc>England</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>135</volume>, no. <issue>4</issue>, pp. <fpage>105371</fpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-15"><label>[15]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>C.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>J.</given-names> <surname>He</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Yan</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Zhang</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<chapter-title>A crash severity prediction method based on improved neural network and factor Analysis</chapter-title>,&#x201D; in <source>Discrete Dynamics in Nature and Society</source>, <publisher-loc>England</publisher-loc>: <publisher-name>Elseiver</publisher-name>, vol. <volume>2020</volume>, no. <issue>3</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>13</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-16"><label>[16]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>L.</given-names> <surname>Liu</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Zhao</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Liu</surname></string-name>, <string-name><given-names>W.</given-names> <surname>Zhu</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>An ensemble of multiple boosting methods based on classifier-specific soft voting for intelligent vehicle crash injury severity prediction</article-title>,&#x201D; in <conf-name>Proc. IEEE Sixth Int. Conf. on Big Data Computing Service and Applications (BigDataService)</conf-name>, <conf-loc>Oxford, UK</conf-loc>, pp. <fpage>17</fpage>&#x2013;<lpage>24</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-17"><label>[17]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Gulino</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Gangi</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Sortino</surname></string-name> and <string-name><given-names>D.</given-names> <surname>Vangi</surname></string-name></person-group>, &#x201C;<chapter-title>Injury risk assessment based on pre-crash variables: The role of closing velocity and impact eccentricity</chapter-title>,&#x201D; in <source>Information Sciences</source>, <publisher-loc>Amsterdam</publisher-loc>: <publisher-name>Elseiver</publisher-name>, vol. <volume>479</volume>, no. <issue>2</issue>, pp. <fpage>432</fpage>&#x2013;<lpage>447</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-18"><label>[18]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>D.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Deng</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Choi</surname></string-name>, <string-name><given-names>B. B.</given-names> <surname>Gupta</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<chapter-title>A novel CNN based security guaranteed image watermarking generation scenario for smart city applications</chapter-title>,&#x201D; in <source>Accident Analysis &#x0026; Prevention</source>, <publisher-loc>Amsterdam</publisher-loc>: <publisher-name>Elseiver</publisher-name>, vol. <volume>150</volume>, no. <issue>3</issue>, pp. <fpage>105864</fpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-19"><label>[19]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>S. R.</given-names> <surname>Sahoo</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Ranjan</surname></string-name> and <string-name><given-names>B. B.</given-names> <surname>Gupta</surname></string-name></person-group>, &#x201C;<chapter-title>Multiple features based approach for automatic fake news detection on social networks using deep learning</chapter-title>,&#x201D; in <source>Applied Soft Computing</source>, <publisher-loc>England</publisher-loc>: <publisher-name>Elseiver</publisher-name>, vol. <volume>100</volume>, no. <issue>3</issue>, pp. <fpage>106983</fpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-20"><label>[20]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>F.</given-names> <surname>Mirsadeghi</surname></string-name>, <string-name><given-names>M. K.</given-names> <surname>Rafsanjani</surname></string-name> and <string-name><given-names>B. B.</given-names> <surname>Gupta</surname></string-name></person-group>, &#x201C;<article-title>A trust infrastructure based authentication method for clustered vehicular ad hoc networks</article-title>,&#x201D; <source>Peer-to-Peer Networking and Applications</source>, vol. <volume>14</volume>, no. <issue>4</issue>, pp. <fpage>2537</fpage>&#x2013;<lpage>2553</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-21"><label>[21]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>V. A.</given-names> <surname>Memos</surname></string-name>, <string-name><given-names>K. E.</given-names> <surname>Psannis</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Ishibashi</surname></string-name>, <string-name><given-names>B. G.</given-names> <surname>Kim</surname></string-name> and <string-name><given-names>B. B.</given-names> <surname>Gupta</surname></string-name></person-group>, &#x201C;<chapter-title>An efficient algorithm for media-based surveillance system (EAMSuS) in IoT smart city framework</chapter-title>,&#x201D; in <source>Future Generation Computer Systems</source>, <publisher-loc>Netherlands</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>83</volume>, no. <issue>4</issue>, pp. <fpage>619</fpage>&#x2013;<lpage>628</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-22"><label>[22]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>I. T.</given-names> <surname>Forum</surname></string-name></person-group>, &#x201C;<article-title>IRTAD road safety annual database report 2020</article-title>,&#x201D; <source>International Traffic Safety Data and Analysis Group (IRTAD)</source>, vol. <volume>1</volume>, pp. <fpage>1</fpage>&#x2013;<lpage>65</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-23"><label>[23]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>U. S. D. O.</given-names> <surname>Transportation</surname></string-name></person-group>, &#x201C;<article-title>Fatality analysis reporting system (FARS) accident datasets</article-title>,&#x201D; <source>National Highway Traffic Safety Administration (NHTSA)</source>, <year>2020</year>. [Online]. Available: <uri xlink:href="https://www.nhtsa.gov/research-data/fatality-analysis-reporting-system-fars">https://www.nhtsa.gov/research-data/fatality-analysis-reporting-system-fars</uri>.</mixed-citation></ref>
<ref id="ref-24"><label>[24]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>O. G.</given-names> <surname>Portal</surname></string-name></person-group>, &#x201C;<chapter-title>National collision database 2017</chapter-title>,&#x201D; <source>(NCDB) National Collision Database</source>, <publisher-loc>Canada</publisher-loc>: <publisher-name>Transport Canada</publisher-name>, <year>2020</year>. [Online]. Available: <uri xlink:href="https://open.canada.ca/data/en/dataset/1eb9eba7-71d1-4b30-9fb1-30cbdab7e63a">https://open.canada.ca/data/en/dataset/1eb9eba7-71d1-4b30-9fb1-30cbdab7e63a</uri>.</mixed-citation></ref>
<ref id="ref-25"><label>[25]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>N.</given-names> <surname>Shehab</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Badawy</surname></string-name> and <string-name><given-names>H.</given-names> <surname>Arafat</surname></string-name></person-group>, &#x201C;<chapter-title>Big data analytics and preprocessing</chapter-title>,&#x201D; in <source>Machine Learning and Big Data Analytics Paradigms: Analysis, Applications and Challenges</source>, <publisher-loc>Switzerland</publisher-loc>: <publisher-name>Springer</publisher-name>, vol. <volume>4</volume>, no. <issue>10</issue>, pp. <fpage>25</fpage>&#x2013;<lpage>43</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-26"><label>[26]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>S. S.</given-names> <surname>Nimankar</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Shivaji</surname></string-name> and <string-name><given-names>D.</given-names> <surname>Vora</surname></string-name></person-group>, &#x201C;<chapter-title>Designing a model to handle imbalance data classification using smote and optimized classifier</chapter-title>,&#x201D; in <source>Data Management, Analytics and Innovation</source>, <publisher-loc>Singapore</publisher-loc>: <publisher-name>Springer</publisher-name>, vol. <volume>16</volume>, no. <issue>3</issue>, pp. <fpage>323</fpage>&#x2013;<lpage>334</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-27"><label>[27]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>C.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>N.</given-names> <surname>Shakhovska</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Sachenk</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Komar</surname></string-name></person-group>, &#x201C;<article-title>A new approach for missing data imputation in big data interface</article-title>,&#x201D; <source>Information Technology and Control</source>, vol. <volume>49</volume>, no. <issue>4</issue>, pp. <fpage>541</fpage>&#x2013;<lpage>555</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-28"><label>[28]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Thaseen</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Banu</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Lavanya</surname></string-name>, <string-name><given-names>K.</given-names> <surname>Rukunuddin</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Ghalib</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>An integrated intrusion detection system using correlation-based attribute selection and artificial neural network</article-title>,&#x201D; <source>Transactions on Emerging Telecommunications Technologies</source>, vol. <volume>32</volume>, no. <issue>2</issue>, pp. <fpage>14</fpage>&#x2013;<lpage>40</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-29"><label>[29]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>&#x015E;</given-names> <surname>Y&#x00FC;celba&#x015F;</surname></string-name></person-group>, &#x201C;<article-title>Simple logistic hybrid system based on greedy stepwise algorithm for feature analysis to diagnose Parkinson&#x0027;s disease according to gender</article-title>,&#x201D; <source>Arabian Journal for Science Engineering: Elseiver</source>, vol. <volume>45</volume>, no. <issue>3</issue>, pp. <fpage>2001</fpage>&#x2013;<lpage>2016</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-30"><label>[30]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J.</given-names> <surname>Pino-Ortega</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Rojas-Valverde</surname></string-name>, <string-name><given-names>D.</given-names> <surname>G&#x00F3;mez-Carmona</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Rico-Gonz&#x00E1;lez.</surname></string-name></person-group>, &#x201C;<article-title>Raining design, performance analysis and talent identification&#x2014;a systematic review about the most relevant variables through the principal component analysis in soccer, basketball and rugby</article-title>,&#x201D; <source>International Journal of Environmental Research and Public Health</source>, vol. <volume>18</volume>, no. <issue>5</issue>, pp. <fpage>26</fpage>&#x2013;<lpage>42</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-31"><label>[31]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>W. A.</given-names> <surname>Yousef</surname></string-name></person-group>, &#x201C;<chapter-title>Estimating the standard error of cross-Validation-Based estimators of classifier performance</chapter-title>,&#x201D; in <source>Pattern Recognition Letters</source>, <publisher-loc>Netherlands</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>146</volume>, no. <issue>3</issue>, pp. <fpage>115</fpage>&#x2013;<lpage>125</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-32"><label>[32]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>F.</given-names> <surname>Abramovich</surname></string-name>, <string-name><given-names>V.</given-names> <surname>Grinshtein</surname></string-name> and <string-name><given-names>T.</given-names> <surname>Levy</surname></string-name></person-group>, &#x201C;<article-title>Multiclass classification by sparse multinomial logistic regression</article-title>,&#x201D; <source>IEEE Transactions on Information Theory</source>, vol. <volume>3</volume>, no. <issue>4</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>15</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-33"><label>[33]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>A. C.</given-names> <surname>Cinar</surname></string-name></person-group>, &#x201C;<chapter-title>Training feed-forward multi-layer perceptron artificial neural networks with a tree-seed algorithm</chapter-title>,&#x201D; in <source>Arabian Journal for Science and Engineering</source>, <publisher-loc>Germany</publisher-loc>: <publisher-name>Springer</publisher-name>, vol. <volume>45</volume>, no. <issue>12</issue>, pp. <fpage>10915</fpage>&#x2013;<lpage>10938</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-34"><label>[34]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>C. G.</given-names> <surname>Chehbi</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Gamoura</surname></string-name> and <string-name><given-names>H.</given-names> <surname>Koruca</surname></string-name></person-group>, &#x201C; <article-title>The theory of complexity and&#x201D; No free lunch</article-title>,&#x201D; in <conf-name>Proc. Int. Conf. on Artificial Intelligence and Applied Mathematics in Engineering (ICAIAME)</conf-name>, <conf-loc>Antalya Turkey</conf-loc>, vol. <volume>3</volume>, no. <issue>2</issue>, pp. <fpage>216</fpage>&#x2013;<lpage>228</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-35"><label>[35]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Husin</surname></string-name></person-group>, &#x201C;<article-title>Designing multiple classifier combinations a survey</article-title>,&#x201D; <source>Journal of Theoretical Applied Information Technology</source>, vol. <volume>97</volume>, no. <issue>20</issue>, pp. <fpage>2386</fpage>&#x2013;<lpage>2405</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-36"><label>[36]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>E.</given-names> <surname>Moradi</surname></string-name> and <string-name><given-names>L.</given-names> <surname>Miranda-Moreno</surname></string-name></person-group>, &#x201C;<chapter-title>Vehicular fuel consumption estimation using real-world measures through cascaded machine learning modeling</chapter-title>,&#x201D; in <source>Transportation Research Part D: Transport and Environment</source>, <publisher-loc>England</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>88</volume>, no. <issue>3</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>17</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-37"><label>[37]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>K.</given-names> <surname>Jiang</surname></string-name>, <string-name><given-names>W.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Wang</surname></string-name> and <string-name><given-names>H.</given-names> <surname>Wu</surname></string-name></person-group>, &#x201C;<article-title>Network intrusion detection combined hybrid sampling with deep hierarchical network</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>8</volume>, pp. <fpage>32464</fpage>&#x2013;<lpage>32476</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-38"><label>[38]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Taiwo</surname></string-name>, <string-name><given-names>L. O.</given-names> <surname>Jolaoso</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Olakunle</surname></string-name> and <string-name><given-names>O. T.</given-names> <surname>Mewomo</surname></string-name></person-group>, &#x201C;<chapter-title>Parallel hybrid algorithm for solving pseudomonotone equilibrium and split common fixed point problems</chapter-title>,&#x201D; in <source>Bulletin of the Malaysian Mathematical Sciences Society</source>, <publisher-loc>England</publisher-loc>: <publisher-name>Springer</publisher-name>, vol. <volume>43</volume>, no. <issue>2</issue>, pp. <fpage>1893</fpage>&#x2013;<lpage>1918</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-39"><label>[39]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>X. W.</given-names> <surname>Shi</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Chai</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Li</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Zhi-Feng</surname></string-name></person-group>, &#x201C;<article-title>An automated machine learning (AutoML) method of risk prediction for decision-making of autonomous vehicles</article-title>,&#x201D; <source>IEEE Transactions on Intelligent Transportation Systems</source>, vol. <volume>1</volume>, no. <issue>2</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>10</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-40"><label>[40]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>J.</given-names> <surname>Xu</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Zhang</surname></string-name> and <string-name><given-names>D. J. I. S.</given-names> <surname>Miao</surname></string-name></person-group>, &#x201C;<chapter-title>Three-way confusion matrix for classification: a measure driven view</chapter-title>,&#x201D; in <source>Information Sciences</source>, <publisher-loc>NY, USA</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>507</volume>, no. <issue>2</issue>, pp. <fpage>772</fpage>&#x2013;<lpage>794</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-41"><label>[41]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>P. R.</given-names> <surname>Verma</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Srivastava</surname></string-name> and <string-name><given-names>P. K.</given-names> <surname>Raghubanshi</surname></string-name></person-group>, &#x201C;<chapter-title>Appraisal of kappa-based metrics and disagreement indices of accuracy assessment for parametric and nonparametric techniques used in LULC classification and change detection</chapter-title>,&#x201D; in <source>Modeling Earth Systems Environment</source>, <publisher-loc>Germany</publisher-loc>: <publisher-name>Springer</publisher-name>, vol. <volume>2</volume>, no. <issue>3</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>15</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-42"><label>[42]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Janssens</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Cecile</surname></string-name> and <string-name><given-names>F. K.</given-names> <surname>Martens</surname></string-name></person-group>, &#x201C;<chapter-title>Reflection on modern methods: revisiting the area under the ROC curve</chapter-title>,&#x201D; in <source>International Journal of Epidemiology</source>, <publisher-loc>Oxford, England</publisher-loc>: <publisher-name>Oxford University Press</publisher-name>, vol. <volume>49</volume>, no. <issue>4</issue>, pp. <fpage>1397</fpage>&#x2013;<lpage>1403</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-43"><label>[43]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Sangare</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Gupta</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Bouzefrane</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Banerjee</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Muhlethaler</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<chapter-title>Exploring the forecasting approach for road accidents: analytical measures with hybrid machine learning</chapter-title>,&#x201D; in <source>Expert Systems with Applications</source>, <publisher-loc>Kidlington, Oxford, England</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>3</volume>, no. <issue>1</issue>, pp. <fpage>113855</fpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-44"><label>[44]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>AlKheder</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Aiash</surname></string-name> and <string-name><given-names>F.</given-names> <surname>AlRukaibi</surname></string-name></person-group>, &#x201C;<chapter-title>Risk analysis of traffic accidents&#x2019; severities: an application of three data mining models</chapter-title>,&#x201D; in <source>ISA Transactions</source>, <publisher-loc>Newyork, USA </publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>106</volume>, no. <issue>3</issue>, pp. <fpage>213</fpage>&#x2013;<lpage>220</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-45"><label>[45]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>Y. L.</given-names> <surname>Lin</surname></string-name> and <string-name><given-names>R.</given-names> <surname>Li</surname></string-name></person-group>, &#x201C;<chapter-title>Real-time traffic accidents post-impact prediction: Based on crowdsourcing data</chapter-title>,&#x201D; in <source>Accident Analysis</source>, <publisher-loc>Oxford, England</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>145</volume>, no. <issue>2</issue>, pp. <fpage>105696</fpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-46"><label>[46]</label><mixed-citation publication-type="web"><person-group person-group-type="author"><string-name><given-names>J. D. M.</given-names> <surname>Package</surname></string-name></person-group>, <person-group person-group-type="author"><collab>JDMP</collab></person-group>, &#x201C;<article-title>A library for machine learning and Big data analytics</article-title>,&#x201D; <year>2019</year>. [Online]. Available: <uri xlink:href="https://jdmp.org/">https://jdmp.org/</uri>.</mixed-citation></ref>
<ref id="ref-47"><label>[47]</label><mixed-citation publication-type="web"><person-group person-group-type="author"><collab>WEKA</collab></person-group>, &#x201C;<article-title>WEKA 3, the workbench for machine learning</article-title>,&#x201D; <year>2019</year>. [Online]. Available: <uri xlink:href="https://www.cs.waikato.ac.nz/ml/weka/">https://www.cs.waikato.ac.nz/ml/weka/</uri>.</mixed-citation></ref>
<ref id="ref-48"><label>[48]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>D.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Dang</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Choi</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>B. B.</given-names> <surname>Gupta</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<chapter-title>A novel CNN based security guaranteed image watermarking generation scenario for smart city applications</chapter-title>,&#x201D; in <source>Information Sciences</source>, <publisher-loc>NY, USA</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>479</volume>, no. <issue>2</issue>, pp. <fpage>432</fpage>&#x2013;<lpage>447</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-49"><label>[49]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>B. B.</given-names> <surname>Gupta</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Quamara</surname></string-name></person-group>, &#x201C;<chapter-title>An overview of Internet of Things (IoT): Architectural aspects, challenges, and protocols</chapter-title>,&#x201D; in <source>Concurrency and Computation: Practice and Experience</source>, <publisher-loc>Hoboken, USA</publisher-loc>: <publisher-name>Wiley Online Library</publisher-name>, vol. <volume>32</volume>, no. <issue>21</issue>, pp. <fpage>e4946</fpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-50"><label>[50]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>H.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>Z.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Choi</surname></string-name>, <string-name><given-names>B. B.</given-names> <surname>Gupta</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<chapter-title>Visual saliency guided complex image retrieval</chapter-title>,&#x201D; in <source>Pattern Recognition Letters</source>, <publisher-loc>Netherlands</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>130</volume>, no. <issue>2</issue>, pp. <fpage>64</fpage>&#x2013;<lpage>72</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-51"><label>[51]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>C.</given-names> <surname>Espositoa</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Ficcob</surname></string-name> and <string-name><given-names>B. B.</given-names> <surname>Gupta</surname></string-name></person-group>, &#x201C;<chapter-title>Blockchain-based authentication and authorization for smart city applications</chapter-title>,&#x201D; in <source>Information Processing &#x0026; Management</source>, <publisher-loc>Oxford, England</publisher-loc>: <publisher-name>Elsevier</publisher-name>, vol. <volume>68</volume>, no. <issue>2</issue>, pp. <fpage>102468</fpage>, <year>2021</year>.</mixed-citation></ref>
</ref-list>
</back>
</article>