<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1 20151215//EN" "http://jats.nlm.nih.gov/publishing/1.1/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="research-article" dtd-version="1.1">
<front>
<journal-meta>
<journal-id journal-id-type="pmc">CMC</journal-id>
<journal-id journal-id-type="nlm-ta">CMC</journal-id>
<journal-id journal-id-type="publisher-id">CMC</journal-id>
<journal-title-group>
<journal-title>Computers, Materials &#x0026; Continua</journal-title>
</journal-title-group>
<issn pub-type="epub">1546-2226</issn>
<issn pub-type="ppub">1546-2218</issn>
<publisher>
<publisher-name>Tech Science Press</publisher-name>
<publisher-loc>USA</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">19029</article-id>
<article-id pub-id-type="doi">10.32604/cmc.2022.019029</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Article</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>An Eigenspace Method for Detecting Space-Time Disease Clusters with Unknown Population-Data</article-title>
<alt-title alt-title-type="left-running-head">An Eigenspace Method for Detecting Space-Time Disease Clusters with Unknown Population-Data</alt-title>
<alt-title alt-title-type="right-running-head">An Eigenspace Method for Detecting Space-Time Disease Clusters with Unknown Population-Data</alt-title>
</title-group>
<contrib-group content-type="authors">
<contrib id="author-1" contrib-type="author" corresp="yes"><name name-style="western"><surname>Ullah</surname><given-names>Sami</given-names></name><xref ref-type="aff" rid="aff-1">1</xref><email>sami.khan3891@gmail.com</email>
</contrib>
<contrib id="author-2" contrib-type="author"><name name-style="western"><surname>Nor</surname><given-names>Nurul Hidayah Mohd</given-names></name><xref ref-type="aff" rid="aff-1">1</xref>
</contrib>
<contrib id="author-3" contrib-type="author"><name name-style="western"><surname>Daud</surname><given-names>Hanita</given-names></name><xref ref-type="aff" rid="aff-1">1</xref>
</contrib>
<contrib id="author-4" contrib-type="author"><name name-style="western"><surname>Zainuddin</surname><given-names>Nooraini</given-names></name><xref ref-type="aff" rid="aff-1">1</xref>
</contrib>
<contrib id="author-5" contrib-type="author"><name name-style="western"><surname>Fanaee-T</surname><given-names>Hadi</given-names></name><xref ref-type="aff" rid="aff-2">2</xref>
</contrib>
<contrib id="author-6" contrib-type="author"><name name-style="western"><surname>Khalil</surname><given-names>Alamgir</given-names></name><xref ref-type="aff" rid="aff-3">3</xref>
</contrib>
<aff id="aff-1"><label>1</label><institution>Department of Fundamental &#x0026; Applied Sciences, Universiti Teknologi PETRONAS</institution>, <addr-line>Seri Iskandar, 32610, Perak</addr-line>, <country>Malaysia</country></aff>
<aff id="aff-2"><label>2</label><institution>Center for Applied Intelligent Systems Research (CAISR), Halmstad University</institution>, <addr-line>Halmstad, SE-301 18</addr-line>, <country>Sweden</country></aff>
<aff id="aff-3"><label>3</label><institution>Department of Statistics, University of Peshawar</institution>, <country>Pakistan</country></aff>
</contrib-group>
<author-notes>
<corresp id="cor1"><label>&#x002A;</label>Corresponding Author: Sami Ullah. Email: <email>sami.khan3891@gmail.com</email></corresp>
</author-notes>
<pub-date pub-type="epub" date-type="pub" iso-8601-date="2021-08-30"><day>30</day><month>08</month><year>2021</year>
</pub-date>
<volume>70</volume>
<issue>1</issue>
<fpage>1945</fpage>
<lpage>1953</lpage>
<history>
<date date-type="received"><day>30</day><month>3</month><year>2021</year>
</date>
<date date-type="accepted"><day>01</day><month>5</month><year>2021</year>
</date>
</history>
<permissions>
<copyright-statement>&#x00A9; 2022 Ullah et al.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Ullah et al.</copyright-holder>
<license xlink:href="https://creativecommons.org/licenses/by/4.0/">
<license-p>This work is licensed under a <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</ext-link>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
</license>
</permissions>
<self-uri content-type="pdf" xlink:href="TSP_CMC_19029.pdf"></self-uri>
<abstract>
<p>Space-time disease cluster detection assists in conducting disease surveillance and implementing control strategies. The state-of-the-art method for this kind of problem is the Space-time Scan Statistics (SaTScan) which has limitations for non-traditional/non-clinical data sources due to its parametric model assumptions such as Poisson or Gaussian counts. Addressing this problem, an Eigenspace-based method called Multi-EigenSpot has recently been proposed as a nonparametric solution. However, it is based on the population counts data which are not always available in the least developed countries. In addition, the population counts are difficult to approximate for some surveillance data such as emergency department visits and over-the-counter drug sales, where the catchment area for each hospital/pharmacy is undefined. We extend the population-based Multi-EigenSpot method to approximate the potential disease clusters from the observed/reported disease counts only with no need for the population counts. The proposed adaptation uses an estimator of expected disease count that does not depend on the population counts. The proposed method was evaluated on the real-world dataset and the results were compared with the population-based methods: Multi-EigenSpot and SaTScan. The result shows that the proposed adaptation is effective in approximating the important outputs of the population-based methods.</p>
</abstract>
<kwd-group kwd-group-type="author">
<kwd>Space-time disease clusters</kwd>
<kwd>Eigenspace method</kwd>
<kwd>nontraditional data sources</kwd>
<kwd>nonparametric methods</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec id="s1">
<label>1</label>
<title>Introduction</title>
<p>With the advent of electronic medical records, syndromic data sources, and low-cost location sensors, data on disease occurrences or other health-related events are increasingly encoded with both spatial and temporal information. Based on this data, Health authorities conduct surveillance to search for the potential clusters of disease or other health-related events. In public health, cluster detection aims to identify those spatiotemporal regions that contain unexpected counts of disease cases or other health-related events. The detection of such potential clusters facilitates the health officials&#x2019; efforts to identify their targets of interest for possible interventions. Such clusters show the over-density anomalies in the spatiotemporal space which assist epidemiologists in finding the environmental factors responsible for a particular disease outbreak in the area.</p>
<p>A number of parametric methods have been developed for detecting space-time clusters in public health data. The examples are Space-time Scan Statistic (SaTScan) [<xref ref-type="bibr" rid="ref-1">1</xref>,<xref ref-type="bibr" rid="ref-2">2</xref>], Expectation-based Scan Statistic [<xref ref-type="bibr" rid="ref-3">3</xref>,<xref ref-type="bibr" rid="ref-4">4</xref>], Flexible Space-time Scan Statistic [<xref ref-type="bibr" rid="ref-5">5</xref>,<xref ref-type="bibr" rid="ref-6">6</xref>], Space-time Permutation Scan Statistic [<xref ref-type="bibr" rid="ref-7">7</xref>,<xref ref-type="bibr" rid="ref-8">8</xref>], and EvoGridStatistic [<xref ref-type="bibr" rid="ref-9">9</xref>,<xref ref-type="bibr" rid="ref-10">10</xref>]. All these methods are based on Maximum Likelihood Estimation (MLE) which put some constraints on the distribution and quality of data that are valid only for clinical data which are collected from the hospitals and are not necessarily valid for non-traditional/nonclinical data sources. For example, data collected from social media [<xref ref-type="bibr" rid="ref-11">11</xref>], pharmacy sales, and school health surveys are non-traditional or non-clinical data sources for public health surveillance [<xref ref-type="bibr" rid="ref-12">12</xref>], where the parametric model might be very restrictive <italic>i.e</italic>., difficult to be followed. For such data sources, MLE-based methods like SaTScan are not an ideal choice for disease cluster detection. Addressing this problem, the nonparametric methods called EigenSpot [<xref ref-type="bibr" rid="ref-13">13</xref>] and Multi-EigenSpot [<xref ref-type="bibr" rid="ref-14">14</xref>] have recently been developed that make no assumption about the distribution and quality of data. However, these nonparametric methods require that the population counts be available. This is a big limitation, because, in some least developed countries census population data are not available. In addition, the population counts are difficult to approximate for some surveillance data such as emergency department visits and over-the-counter drugs sales where the catchment area for each hospital/pharmacy is undefined. Even if the population counts are available, the catchment area population would not be a good denominator since there can be natural geographical disparity in health-care utilization data, due to disparities in disease prevalence, access to health care, and consumer behavior [<xref ref-type="bibr" rid="ref-15">15</xref>].</p>
<p>In order to address this problem, we adapt the Multi-EigenSpot algorithm to be applicable for disease surveillance in such a realistic scenario. Multi-EigenSpot uses a population-based estimator for expected disease occurrences that has been frequently used in prior arts [<xref ref-type="bibr" rid="ref-9">9</xref>,<xref ref-type="bibr" rid="ref-16">16</xref>]. We propose an adaptation by using a different estimator of the expected disease occurrences in the algorithm which does not depend on the population counts. The proposed adaptation infers the expected disease counts from the observed disease counts only. The experimental evaluation on real-world data shows that the proposed adaptation is effective in approximating the significant outputs of the population-based methods.</p>
<p>Some nonparametric alternatives to the MLE-based scan statistics have also been proposed such as [<xref ref-type="bibr" rid="ref-17">17</xref>&#x2013;<xref ref-type="bibr" rid="ref-19">19</xref>]. However, these are purely spatial techniques that can detect purely spatial clusters while this research focuses on the space-time cluster detection problem. It is evident from the literature that the Eigenspace-based methods [<xref ref-type="bibr" rid="ref-13">13</xref>,<xref ref-type="bibr" rid="ref-14">14</xref>] are the latest nonparametric technique in the spatiotemporal class of methods for areal-count data.</p>
</sec>
<sec id="s2">
<label>2</label>
<title>Materials and Methods</title>
<p>The stepwise process of the proposed approach is given below:</p>
<p>Step 1: Given the observed disease counts, estimate the spatiotemporal matrices of expected disease cases, <italic>E</italic> and Risk measures, <italic>R</italic> according to <xref ref-type="disp-formula" rid="eqn-1">Eqs. (1)</xref> and <xref ref-type="disp-formula" rid="eqn-3">(2)</xref>, respectively.</p>
<p><disp-formula id="eqn-1">
<label>(1)</label>
<mml:math id="mml-eqn-1" display="block"><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:munder><mml:msub><mml:mi>C</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>&#x00D7;</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>j</mml:mi></mml:mrow></mml:munder><mml:msub><mml:mi>C</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:mrow><mml:mrow><mml:mi>C</mml:mi><mml:mo>.</mml:mo><mml:mo>.</mml:mo></mml:mrow></mml:mfrac></mml:math>
</disp-formula></p>
<p><disp-formula id="eqn-2">
<mml:math id="mml-eqn-2" display="block"><mml:mi>E</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:mtable columnalign="left left left left" rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mn>11</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>&#x2026;</mml:mo></mml:mtd><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mn>1</mml:mn><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mn>21</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mn>22</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>&#x2026;</mml:mo></mml:mtd><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>&#x22EE;</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x22EE;</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x2026;</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x22EE;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mi>m</mml:mi><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mi>m</mml:mi><mml:mn>2</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>&#x2026;</mml:mo></mml:mtd><mml:mtd><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mi>m</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr></mml:mtable><mml:mo>]</mml:mo></mml:mrow></mml:math>
</disp-formula></p>
<p>where <italic>E</italic><sub><italic>ij</italic></sub> is the expected disease count for <italic>i</italic><sup><italic>th</italic></sup> sub-region over the <italic>j</italic><sup><italic>th</italic></sup> time-point; <inline-formula id="ieqn-1"><mml:math id="mml-ieqn-1"><mml:msub><mml:mi>C</mml:mi><mml:mrow><mml:mo>.</mml:mo><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> denotes the total observed/reported cases in the whole study-area at the <italic>j</italic><sup><italic>th</italic></sup> time-point; <inline-formula id="ieqn-2"><mml:math id="mml-ieqn-2"><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mo>.</mml:mo><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> the total population counts in the whole study-area at the <italic>j</italic><sup><italic>th</italic></sup> time-point; <inline-formula id="ieqn-3"><mml:math id="mml-ieqn-3"><mml:msub><mml:mi>p</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> the population counts in the <italic>i</italic><sup><italic>th</italic></sup> sub-region at the <italic>j</italic><sup><italic>th</italic></sup> time-point.</p>
<p><disp-formula id="eqn-3">
<label>(2)</label>
<mml:math id="mml-eqn-3" display="block"><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:msub><mml:mi>C</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:msub><mml:mi>E</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:mfrac></mml:math>
</disp-formula></p>
<p><disp-formula id="eqn-4">
<mml:math id="mml-eqn-4" display="block"><mml:mi>R</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:mtable columnalign="left left left left" rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mn>11</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>&#x2026;</mml:mo></mml:mtd><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mn>1</mml:mn><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mn>21</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mn>22</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>&#x2026;</mml:mo></mml:mtd><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>&#x22EE;</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x22EE;</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x2026;</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x22EE;</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mi>m</mml:mi><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mi>m</mml:mi><mml:mn>2</mml:mn></mml:mrow></mml:msub></mml:mtd><mml:mtd><mml:mo>&#x2026;</mml:mo></mml:mtd><mml:mtd><mml:msub><mml:mi>R</mml:mi><mml:mrow><mml:mi>m</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:mtd></mml:mtr></mml:mtable><mml:mo>]</mml:mo></mml:mrow></mml:math>
</disp-formula></p>
<p>where <italic>E</italic><sub><italic>ij</italic></sub> is the expected disease count for the <italic>i</italic><sup><italic>th</italic></sup> sub-region over the <italic>j</italic><sup><italic>th</italic></sup> time-point; <inline-formula id="ieqn-4"><mml:math id="mml-ieqn-4"><mml:msub><mml:mi>C</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> is the observed/reported disease count in the <italic>i</italic><sup><italic>th</italic></sup> sub-region at the <italic>j</italic><sup><italic>th</italic></sup> time-point; <inline-formula id="ieqn-5"><mml:math id="mml-ieqn-5"><mml:mi>C</mml:mi><mml:mo>.</mml:mo><mml:mo>.</mml:mo></mml:math></inline-formula> is the grand total of the observed/reported disease counts and is calculated as in <xref ref-type="disp-formula" rid="eqn-5">Eq. (3)</xref>.</p>
<p><disp-formula id="eqn-5">
<label>(3)</label>
<mml:math id="mml-eqn-5" display="block"><mml:mi>C</mml:mi><mml:mo>.</mml:mo><mml:mo>.</mml:mo><mml:mo>=</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:munder><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>j</mml:mi></mml:mrow></mml:munder><mml:msub><mml:mi>C</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math>
</disp-formula></p>
<p>Step 2: Calculate the principal-left and principal-right singular vectors of matrices <italic>C</italic> and <italic>E</italic> using one-rank singular value decomposition. For matrix <italic>C</italic>, the principal-left singular vector is denoted by <inline-formula id="ieqn-6"><mml:math id="mml-ieqn-6"><mml:mi>S</mml:mi><mml:mi>C</mml:mi></mml:math></inline-formula> and the principal-right singular vector by <inline-formula id="ieqn-7"><mml:math id="mml-ieqn-7"><mml:mi>T</mml:mi><mml:mi>C</mml:mi></mml:math></inline-formula>. Similarly, for matrix <italic>E</italic>, the principal-left singular vector is denoted by <inline-formula id="ieqn-8"><mml:math id="mml-ieqn-8"><mml:mi>S</mml:mi><mml:mi>E</mml:mi></mml:math></inline-formula> and the principal-right singular vector by <inline-formula id="ieqn-9"><mml:math id="mml-ieqn-9"><mml:mi>T</mml:mi><mml:mi>E</mml:mi></mml:math></inline-formula>.</p>
<p>Step 3: Compute the difference vector of the left-singular vectors as <inline-formula id="ieqn-10"><mml:math id="mml-ieqn-10"><mml:mi>D</mml:mi><mml:mi>S</mml:mi><mml:mo>&#x003A;</mml:mo><mml:mo>=</mml:mo><mml:mi>S</mml:mi><mml:mi>C</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mi>S</mml:mi><mml:mi>E</mml:mi></mml:math></inline-formula>, and that of the right-singular vectors as <inline-formula id="ieqn-11"><mml:math id="mml-ieqn-11"><mml:mi>D</mml:mi><mml:mi>T</mml:mi><mml:mo>=</mml:mo><mml:mi>T</mml:mi><mml:mi>C</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mi>T</mml:mi><mml:mi>E</mml:mi></mml:math></inline-formula>.</p>
<p>Step 4: Find the abnormally higher elements in each subtract vector <italic>DS</italic> and <italic>DT</italic> by applying the Z-control chart with the significance level alpha. The abnormally higher elements in the vector <italic>DS</italic> are associated with the spatial component of the cluster and in vector <italic>DT</italic> to the temporal component.</p>
<p>Step 5: If the abnormally higher elements are found in spatial as well as temporal dimension, upgrade matrix <italic>C</italic> by replacing the elements corresponding to the out-of-control components with the respective expected cases to remove the previous cluster. Simultaneously, matrix <italic>R</italic> is upgraded by replacing the elements corresponding to the out-of-control components by their average value.</p>
<p>Step 6: To approximate the additional clusters, if exist, reiterate Steps (2&#x2013;5) until no out-of-control element is found in each difference vector.</p>
<p>Step 7: In the upgraded matrix <italic>R</italic>, replace the elements corresponding to the components that are not found to be abnormal by 1 to distinguish clearly between the normal and abnormal regions.</p>
<p>Step 8: Visualize the resultant matrix <italic>R</italic> as a heatmap to show multiple clusters with different colors.</p>
<p>What is novel with the proposed adaptation is the strategy used for estimating the expected disease counts. Population-based Multi-EigenSpot uses the historical temporal information for population-at-risk while our proposed method infers this indirectly from the geographical neighborhood. For each region and time point, we calculate the expected number of a particular disease counts conditioning on the observed marginal.</p>
<fig id="fig-1">
<label>Figure 1</label>
<caption>
<title>An example illustrating the proposed approach</title>
</caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="CMC_19029-fig-1.png"/>
</fig>
<sec id="s2_1">
<label>2.1</label>
<title>Illustrative Example</title>
<p><?A3B2 "fig1",5,"anchor"?><xref ref-type="fig" rid="fig-1">Fig. 1</xref> shows the detailed process that how our proposed method detects multiple clusters in a spatiotemporal space with no requirement for population counts. For instance, assume that two different hotspots exist in a 3 &#x00D7; 4 spatiotemporal space. The two shaded areas in matrix C (<xref ref-type="fig" rid="fig-1">Fig. 1</xref>) are the two clusters of interest to be approximated by our proposed approach. The intersection of the third row with the first-second columns denotes the most likely hotspot and the second-third rows with the fourth column the secondary (additional) cluster. The input is only the spatiotemporal matrix of the observed disease counts denoted by C. Given the matrix C, the proposed method approximates these two clusters in two iterations. The most likely cluster is detected in the first iteration. The detected hotspot is then removed by replacing the observed counts with the corresponding expected counts, and the method is repeated for the secondary cluster. In the last upgraded matrix R, the cells containing the value M1 represent one cluster and that containing the value M2 represents the other cluster.</p>
<fig id="fig-2">
<label>Figure 2</label>
<caption>
<title>Heatmap</title>
</caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="CMC_19029-fig-2.png"/>
</fig>
</sec>
</sec>
<sec id="s3">
<label>3</label>
<title>Results and Discussion</title>
<sec id="s3_1">
<label>3.1</label>
<title>Experiment with the Real-World Dataset</title>
<p>In this section, the proposed approach is applied to the measles case data in Khyber&#x2013;Pakhtunkhwa, Pakistan (Jan 2016&#x2013;Dec 2016), assuming the population is unknown. This dataset has been described in detail elsewhere [<xref ref-type="bibr" rid="ref-14">14</xref>]. The proposed method is executed in MATLAB (version R2014a). Based on the spatiotemporal data on the observed measles cases, the proposed method with alpha &#x003D; 0.10, results in a heatmap as shown in <?A3B2 "fig2",5,"anchor"?><xref ref-type="fig" rid="fig-2">Fig. 2</xref>, showing the potential measles hotspots. The resulting heatmap shows three potential measles clusters in Khyber-Pakhtunkhwa in the period from January 2016 to December 2016. The most likely cluster is seen in the district of Bannu for May, October, and December with an average Relative Risk (RR) &#x003D; 1.677, denoted with a dark red color on the heatmap. The secondary cluster is seen in the district Bannu for April with an average RR &#x003D; 1.614, denoted by a light red color on the heatmap. The third cluster is seen in the two districts (Kohat and D. I. Khan) for March and April with an average RR &#x003D; 1.58, represented with a yellow color on the heatmap. These hotspot regions have also been detected by the Multi-EigenSpot and Space-time Scan Statistics in the previous study on the same dataset [<xref ref-type="bibr" rid="ref-14">14</xref>] and hence confirm that the proposed approach is effective for surveillance data with unknown population-at-risk information.</p>
<fig id="fig-3">
<label>Figure 3</label>
<caption>
<title>Geographical map of the study area showing the locations of Measles clusters with red color</title>
</caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="CMC_19029-fig-3.png"/>
</fig>
<p>It is obvious from <?A3B2 "fig3",5,"anchor"?><xref ref-type="fig" rid="fig-3">Fig. 3</xref> that all the hotspots&#x2019; regions identified by the proposed approach are adjacent to Federally Administrative Tribal Areas (FATA). Indeed, due to military operations during the years 2014&#x2013;2016, the Internally Displaced People (IDP) from FATA were settled in the neighboring districts which might have caused the measles outbreak in the hosting districts. Because FATA and IDP camps suffer from a low vaccination rate due to lack of awareness [<xref ref-type="bibr" rid="ref-20">20</xref>,<xref ref-type="bibr" rid="ref-21">21</xref>].</p>
</sec>
<sec id="s3_2">
<label>3.2</label>
<title>Performance Comparison with Population-Based Methods</title>
<p>In this section, we compare the outputs of our proposed method with Multi-EigenSpot and SaTScan which have already been applied to the same dataset [<xref ref-type="bibr" rid="ref-14">14</xref>]. The outputs of these three methods are presented in <?A3B2 "tbl1",5,"anchor"?><xref ref-type="table" rid="table-1">Tab. 1</xref>. It is obvious from <xref ref-type="table" rid="table-1">Tab. 1</xref> that the regions detected by our proposed method were also detected by Multi-EigenSpot and SaTScan. Our proposed method detects (Bannu, May, Oct, Dec,) as the most likely cluster and (Bannu, Apr) as the secondary cluster. It is very interesting to know that the most likely and secondary clusters of the proposed approach are the same as detected by the population-based Muti-EigenSpot. Moreover, our approach detects (Kohat, D. I. Khan, Mar, Apr) as the third cluster while Multi-EigenSpot detects (Bannu, Kohat, D. I. Khan, Mar) as the third cluster, showing the two districts and one month in common.</p>
<p>The outputs of the proposed approach are also included in the significant outputs of the SaTScan. The Space-time Scan Statistics detects (Bannu, Apr-May) as the most likely cluster. This cluster is covered by the first two clusters of the proposed method. The secondary cluster of the SaTScan (Kohat, Mar-Apr) is covered in the third cluster of our proposed method.</p>
<table-wrap id="table-1">
<label>Table 1</label>
<caption>
<title>The outputs of the proposed method, Multi-EigenSpot, and SaTScan</title>
</caption>
<table>
<colgroup>
<col/>
<col charoff="25"/>
</colgroup>
<thead>
<tr>
<th>Method</th>
<th>The detected clusters</th>
</tr>
</thead>
<tbody>
<tr>
<td>The proposed method</td>
<td>03<break/>(Bannu, May, Oct, Dec), (Bannu, Apr), (Kohat, D. I. Khan, Mar, Apr)</td>
</tr>
<tr>
<td>Multi-EigenSpot</td>
<td>08<break/>(Bannu, May, Oct, Dec), (Bannu-Apr), (Bannu, Kohat, D.I,Khan, Mar), (Kohat, D.I.khan, Feb), (Kohat, D.I.Khan, Swat, Jan), (Kohat, D.I.Khan, Swat, Nov), (Kohat, D.I.Khan, Jun, Jul), (Kohat, D.I.Khan, Swat, Aug).</td>
</tr>
<tr>
<td>Space-time scan statistic</td>
<td>08<break/>(Bannu, Apr&#x2013;May), (Kohat, Mar&#x2013;Apr), (Shangla, Sep), (Swat, Nov&#x2013;Dec), (Buner, Feb), (Charsada, Feb), (Sawabi, Dec), (Haripur, May).</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>The proposed approach detects the first three high-risk clusters while using the population counts, the detection ratio can be increased up to 8 clusters. This suggests that if the population counts are is possible to be approximated, then using this extra information, Multi-EigenSpot performs better than our proposed approach.</p>
</sec>
</sec>
<sec id="s4">
<label>4</label>
<title>Conclusion</title>
<p>We proposed the first Eigenspace-based method which allows the nonparametric practice to detect clusters in the scenarios where the population counts are unavailable or difficult to approximate. Our proposed method replaces the temporal inference in methods like EigenSpot [<xref ref-type="bibr" rid="ref-13">13</xref>] and Multi-EigenSpot [<xref ref-type="bibr" rid="ref-14">14</xref>] with geographical inference which ultimately results in a method that can be used for hotspots detection in the least developed countries where population data is not available or very expensive to obtain. The results indicate that the proposed approach can detect the significant clusters with no need for the population counts. The proposed adaptation can delineate the boundaries of a disease outbreak and its potential to guide the control efforts in many least developed countries where the population data are not available or difficult to access. In addition, the proposed method can be used as a nonparametric solution for cluster detection in many research fields such as criminology [<xref ref-type="bibr" rid="ref-22">22</xref>,<xref ref-type="bibr" rid="ref-23">23</xref>], network [<xref ref-type="bibr" rid="ref-24">24</xref>], and environment [<xref ref-type="bibr" rid="ref-25">25</xref>] where the population data is not relevant.</p>
<p>The proposed method does not account for the spatial and temporal covariates which would make it impractical to examine all &#x2018;unusual&#x2019; events, implicitly diminishing the significance of the surveillance. Extending the proposed method to adjust the population-at-risk-data for spatial and temporal covariate is recommended for future work in this area.</p>
</sec>
</body>
<back>
<ack>
<p>The authors grateful to Universiti Teknologi PETRONAS for providing facilities for the research.</p>
</ack>
<fn-group>
<fn fn-type="other">
<p><bold>Funding Statement:</bold> This article was funded by a Fundamental Research Grant Scheme (FRGS) from the Ministry of Education, Malaysia (Ref: FRGS/1/2018/STG06/UTP/02/1) and a Yayasan Universiti Teknologi PETRONAS-Fundamental Research Grant (cost center of 015LC0-013) received by Hanita Daud, URLs: <uri xlink:href="https://www.mohe.gov.my/en/initiatives-2/187-program-utama/penyelidikan/548-research-grants-information">https://www.mohe.gov.my/en/initiatives-2/187-program-utama/pen-yelidikan/548-research-grants-information</uri>; <uri>https://www.utp.edu.my/yayasan/Pages/default.aspx</uri>.</p>
</fn>
<fn fn-type="conflict">
<p><bold>Conflicts of Interest:</bold> The authors declare that they have no conflicts of interest to report regarding the present study.</p>
</fn>
</fn-group>
<ref-list content-type="authoryear">
<title>References</title>
<ref id="ref-1"><label>[1]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Kulldorff</surname></string-name></person-group>, &#x201C;<article-title>Prospective time periodic geographical disease surveillance using a scan statistic</article-title>,&#x201D; <source>Journal of the Royal Statistical Society: Series A (Statistics in Society)</source>, vol. <volume>164</volume>, no. <issue>1</issue>, pp. <fpage>61</fpage>&#x2013;<lpage>72</lpage>, <year>2001</year>.</mixed-citation></ref>
<ref id="ref-2"><label>[2]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Kulldorff</surname></string-name>, <string-name><given-names>W. F.</given-names> <surname>Athas</surname></string-name>, <string-name><given-names>E. J.</given-names> <surname>Feuer</surname></string-name>, <string-name><given-names>B. A.</given-names> <surname>Miller</surname></string-name> and <string-name><given-names>C. R.</given-names> <surname>Key</surname></string-name></person-group>, &#x201C;<article-title>Evaluating cluster alarms: A space-time scan statistic and brain cancer in Los Alamos, New Mexico</article-title>,&#x201D; <source>American Journal of Public Health</source>, vol. <volume>88</volume>, no. <issue>9</issue>, pp. <fpage>1377</fpage>&#x2013;<lpage>1380</lpage>, <year>1998</year>.</mixed-citation></ref>
<ref id="ref-3"><label>[3]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>D. B.</given-names> <surname>Neill</surname></string-name>, <string-name><given-names>A. W.</given-names> <surname>Moore</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Sabhnani</surname></string-name> and <string-name><given-names>K.</given-names> <surname>Daniel</surname></string-name></person-group>, &#x201C;<article-title>Detection of emerging space-time clusters</article-title>,&#x201D; in <conf-name>Proc. of the Eleventh ACM SIGKDD Int. Conf. on Knowledge Discovery in Data Mining</conf-name>, <publisher-loc>Chicago Illinois, USA</publisher-loc>, pp. <fpage>218</fpage>, <year>2005</year>. </mixed-citation></ref>
<ref id="ref-4"><label>[4]</label><mixed-citation publication-type="other"><person-group person-group-type="author"><string-name><given-names>D. B.</given-names> <surname>Neill</surname></string-name></person-group>, &#x201C;<article-title>Detection of spatial and spatio-temporal clusters</article-title>,&#x201D; <comment>Ph.D thesis</comment>, <publisher-name>Carnegie Mellon University</publisher-name>, <year>2006</year>. </mixed-citation></ref>
<ref id="ref-5"><label>[5]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>T.</given-names> <surname>Tango</surname></string-name>, <string-name><given-names>K.</given-names> <surname>Takahashi</surname></string-name> and <string-name><given-names>K.</given-names> <surname>Kohriyama</surname></string-name></person-group>, &#x201C;<article-title>A space-time scan statistic for detecting emerging outbreaks</article-title>,&#x201D; <source>Biometrics</source>, vol. <volume>67</volume>, no. <issue>1</issue>, pp. <fpage>106</fpage>&#x2013;<lpage>115</lpage>, <year>2011</year>.</mixed-citation></ref>
<ref id="ref-6"><label>[6]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>K.</given-names> <surname>Takahashi</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Kulldorff</surname></string-name>, <string-name><given-names>T.</given-names> <surname>Tango</surname></string-name> and <string-name><given-names>K.</given-names> <surname>Yih</surname></string-name></person-group>, &#x201C;<article-title>A flexibly shaped space-time scan statistic for disease outbreak detection and monitoring</article-title>,&#x201D; <source>International Journal of Health Geographics</source>, vol. <volume>7</volume>, no. <issue>1</issue>, pp. <fpage>14</fpage>, <year>2008</year>.</mixed-citation></ref>
<ref id="ref-7"><label>[7]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Kulldorff</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Heffernan</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Hartman</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Assun&#x00E7;&#x00E3;o</surname></string-name> and <string-name><given-names>F.</given-names> <surname>Mostashari</surname></string-name></person-group>, &#x201C;<article-title>A space-time permutation scan statistic for disease outbreak detection</article-title>,&#x201D; <source>PLOS Medicine</source>, vol. <volume>2</volume>, no. <issue>3</issue>, pp. <fpage>216</fpage>&#x2013;<lpage>224</lpage>, <year>2005</year>.</mixed-citation></ref>
<ref id="ref-8"><label>[8]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M. A.</given-names> <surname>Costa</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Kulldorff</surname></string-name></person-group>, &#x201C;<article-title>Maximum linkage space-time permutation scan statistics for disease outbreak detection</article-title>,&#x201D; <source>International Journal of Health Geographics</source>, vol. <volume>13</volume>, no. <issue>1</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>14</lpage>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-9"><label>[9]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>W.</given-names> <surname>Dong</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Sun</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Shi</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Detecting irregularly shaped significant spatial and spatio-temporal clusters</article-title>,&#x201D; in <conf-name>Proc. of the 2012 SIAM Int. Conf. on Data Mining: Society for Industrial and Applied Mathematics</conf-name>, <publisher-loc>New York, USA</publisher-loc>, pp. <fpage>732</fpage>&#x2013;<lpage>743</lpage>, <year>2012</year>. </mixed-citation></ref>
<ref id="ref-10"><label>[10]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>W.</given-names> <surname>Dong</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>Z.</given-names> <surname>Jiang</surname></string-name>, <string-name><given-names>W.</given-names> <surname>Sun</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Xie</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Detect irregularly shaped spatio-temporal clusters for decision support</article-title>,&#x201D; in <conf-name>Proc. of 2011 IEEE Int. Conf. on Service Operations, Logistics and Informatics</conf-name>, <publisher-loc>USA</publisher-loc>, pp. <fpage>231</fpage>&#x2013;<lpage>236</lpage>, <year>2011</year>. </mixed-citation></ref>
<ref id="ref-11"><label>[11]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J. S. P.</given-names> <surname>Tulloch</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Vivancos</surname></string-name>, <string-name><given-names>R. M.</given-names> <surname>Christley</surname></string-name>, <string-name><given-names>A. D.</given-names> <surname>Radford</surname></string-name> and <string-name><given-names>J. C.</given-names> <surname>Warner</surname></string-name></person-group>, &#x201C;<article-title>Mapping tweets to a known disease epidemiology; A case study of Lyme disease in the United Kingdom and Republic of Ireland</article-title>,&#x201D; <source>Journal of Biomedical Informatics</source>, vol. <volume>X</volume>, no. <issue>4</issue>, pp. <fpage>100060</fpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-12"><label>[12]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Mancuso</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Yao</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Otchere</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Rasali</surname></string-name>, <string-name><given-names>E.</given-names> <surname>Clark</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Proof of concept paper: Non-traditional data sources for public health surveillance</article-title>,&#x201D; in <conf-name>Proc. of the 6th Int. Conf. on Digital Health Conf.-DH&#x2019; 16</conf-name>, <publisher-loc>Montr&#x00E9;al Qu&#x00E9;bec, Canada</publisher-loc>, vol. <volume>176</volume>, pp. <fpage>91</fpage>&#x2013;<lpage>92</lpage>, <year>2016</year>. </mixed-citation></ref>
<ref id="ref-13"><label>[13]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>H.</given-names> <surname>Fanaee-T</surname></string-name> and <string-name><given-names>J.</given-names> <surname>Gama</surname></string-name></person-group>, &#x201C;<article-title>Eigenspace method for spatiotemporal hotspot detection</article-title>,&#x201D; <source>Expert System</source>, vol. <volume>32</volume>, no. <issue>3</issue>, pp. <fpage>454</fpage>&#x2013;<lpage>464</lpage>, <year>2015</year>.</mixed-citation></ref>
<ref id="ref-14"><label>[14]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Ullah</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Daud</surname></string-name>, <string-name><given-names>S. C.</given-names> <surname>Dass</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Fanaee-T</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Khalil</surname></string-name></person-group>, &#x201C;<article-title>An Eigenspace approach for detecting multiple space-time disease clusters: Application to measles hotspots detection in Khyber-Pakhtunkhwa, Pakistan</article-title>,&#x201D; <source>PLOS One</source>, vol. <volume>13</volume>, no. <issue>6</issue>, pp. <fpage>e0199176</fpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-15"><label>[15]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>K.</given-names> <surname>Kleinman</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Lazarus</surname></string-name> and <string-name><given-names>R.</given-names> <surname>Platt</surname></string-name></person-group>, &#x201C;<article-title>A generalized linear mixed models approach for detecting incident clusters of disease in small areas, with an application to biological terrorism</article-title>,&#x201D; <source>American Journal of Epidemiology</source>, vol. <volume>159</volume>, no. <issue>3</issue>, pp. <fpage>217</fpage>&#x2013;<lpage>224</lpage>, <year>2004</year>.</mixed-citation></ref>
<ref id="ref-16"><label>[16]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Kulldorff</surname></string-name></person-group>, &#x201C;<article-title>A spatial scan statistic</article-title>,&#x201D; <source>Communications in Statistics-Theory and Methods</source>, vol. <volume>26</volume>, no. <issue>6</issue>, pp. <fpage>1481</fpage>&#x2013;<lpage>1496</lpage>, <year>1997</year>.</mixed-citation></ref>
<ref id="ref-17"><label>[17]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>E.</given-names> <surname>Arias-Castro</surname></string-name>, <string-name><given-names>R. M.</given-names> <surname>Castro</surname></string-name>, <string-name><given-names>E.</given-names> <surname>T&#x00E1;nczos</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Wang</surname></string-name></person-group>, &#x201C;<article-title>Distribution-free detection of structured anomalies: Permutation and rank-based scans</article-title>,&#x201D; <source>Journal of the American Statistical Association</source>, vol. <volume>113</volume>, no. <issue>522</issue>, pp. <fpage>789</fpage>&#x2013;<lpage>801</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-18"><label>[18]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>L.</given-names> <surname>Cucala</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Genin</surname></string-name>, <string-name><given-names>F.</given-names> <surname>Occelli</surname></string-name> and <string-name><given-names>J.</given-names> <surname>Soula</surname></string-name></person-group>, &#x201C;<article-title>A multivariate nonparametric scan statistic for spatial data</article-title>,&#x201D; <source>Spatial Statistics</source>, vol. <volume>29</volume>, no. <issue>6</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>14</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-19"><label>[19]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>I.</given-names> <surname>Jung</surname></string-name> and <string-name><given-names>H. J.</given-names> <surname>Cho</surname></string-name></person-group>, &#x201C;<article-title>A nonparametric spatial scan statistic for continuous data</article-title>,&#x201D; <source>International Journal of Health Geographics</source>, vol. <volume>14</volume>, no. <issue>1</issue>, pp. <fpage>30</fpage>, <year>2015</year>.</mixed-citation></ref>
<ref id="ref-20"><label>[20]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S. S.</given-names> <surname>Ali</surname></string-name>, <string-name><given-names>E. A.</given-names> <surname>Khan</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Mushtaq</surname></string-name> and <string-name><given-names>M. U.</given-names> <surname>Rahman</surname></string-name></person-group>, &#x201C;<article-title>Childhood immunization among internally displaced persons (IDPS) of under five-years from north Waziristan agency, Pakistan: A cross sectional study</article-title>,&#x201D; <source>Pakistan Journal of Public Health</source>, vol. <volume>6</volume>, no. <issue>4</issue>, pp. <fpage>36</fpage>&#x2013;<lpage>40</lpage>, <year>2016</year>.</mixed-citation></ref>
<ref id="ref-21"><label>[21]</label><mixed-citation publication-type="other"><person-group person-group-type="author"><collab>Sci-Dev-Net</collab></person-group>, &#x201C;<article-title>Pakistan boosts drive against measles</article-title>,&#x201D; <year>2012</year>. [Online]. Available: <uri>https://www.scidev.net/global/health/news/pakistan-boosts-drive-against-measles.html</uri>.</mixed-citation></ref>
<ref id="ref-22"><label>[22]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>T.</given-names> <surname>Nakaya</surname></string-name> and <string-name><given-names>K.</given-names> <surname>Yano</surname></string-name></person-group>, &#x201C;<article-title>Visualising crime clusters in a space-time cube: An exploratory data-analysis approach using space-time kernel density estimation and scan statistics</article-title>,&#x201D; <source>Transactions in GIS</source>, vol. <volume>14</volume>, no. <issue>3</issue>, pp. <fpage>223</fpage>&#x2013;<lpage>239</lpage>, <year>2010</year>.</mixed-citation></ref>
<ref id="ref-23"><label>[23]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>R.</given-names> <surname>Minamisava</surname></string-name>, <string-name><given-names>S. S.</given-names> <surname>Nouer</surname></string-name>, <string-name><given-names>O. L.</given-names> <surname>de Morais Neto</surname></string-name>, <string-name><given-names>L. K.</given-names> <surname>Melo</surname></string-name> and <string-name><given-names>A. L. S. S.</given-names> <surname>Andrade</surname></string-name></person-group>, &#x201C;<article-title>Spatial clusters of violent deaths in a newly urbanized region of Brazil: Highlighting the social disparities</article-title>,&#x201D; <source>International Journal of Health Geographics</source>, vol. <volume>8</volume>, no. <issue>1</issue>, pp. <fpage>66</fpage>, <year>2009</year>.</mixed-citation></ref>
<ref id="ref-24"><label>[24]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>L.</given-names> <surname>Duczmal</surname></string-name>, <string-name><given-names>G. J. P.</given-names> <surname>Moreira</surname></string-name>, <string-name><given-names>S. J.</given-names> <surname>Ferreira</surname></string-name> and <string-name><given-names>R. H. C.</given-names> <surname>Takahashi</surname></string-name></person-group>, &#x201C;<article-title>Dual graph spatial cluster detection for syndromic surveillance in networks</article-title>,&#x201D; <source>Advances in Disease Surveillance</source>, vol. <volume>4</volume>, pp. <fpage>88</fpage>, <year>2007</year>.</mixed-citation></ref>
<ref id="ref-25"><label>[25]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Tonini</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Tuia</surname></string-name> and <string-name><given-names>F.</given-names> <surname>Ratle</surname></string-name></person-group>, &#x201C;<article-title>Detection of clusters using space-time scan statistics</article-title>,&#x201D; <source>International Journal of Wildland Fire</source>, vol. <volume>18</volume>, no. <issue>7</issue>, pp. <fpage>830</fpage>&#x2013;<lpage>836</lpage>, <year>2009</year>.</mixed-citation></ref>
</ref-list>
</back>
</article>