<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1 20151215//EN" "http://jats.nlm.nih.gov/publishing/1.1/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="research-article" dtd-version="1.1">
<front>
<journal-meta>
<journal-id journal-id-type="pmc">CMC</journal-id>
<journal-id journal-id-type="nlm-ta">CMC</journal-id>
<journal-id journal-id-type="publisher-id">CMC</journal-id>
<journal-title-group>
<journal-title>Computers, Materials &#x0026; Continua</journal-title>
</journal-title-group>
<issn pub-type="epub">1546-2226</issn>
<issn pub-type="ppub">1546-2218</issn>
<publisher>
<publisher-name>Tech Science Press</publisher-name>
<publisher-loc>USA</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">30262</article-id>
<article-id pub-id-type="doi">10.32604/cmc.2023.030262</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Article</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Sentiment Analysis and Classification Using Deep Semantic Information and Contextual Knowledge</article-title>
<alt-title alt-title-type="left-running-head">Sentiment Analysis and Classification Using Deep Semantic Information and Contextual Knowledge</alt-title>
<alt-title alt-title-type="right-running-head">Sentiment Analysis and Classification Using Deep Semantic Information and Contextual Knowledge</alt-title>
</title-group>
<contrib-group content-type="authors">
<contrib id="author-1" contrib-type="author">
<name name-style="western"><surname>Al-Absi</surname><given-names>Ahmed Abdulhakim</given-names></name><xref ref-type="aff" rid="aff-1">1</xref></contrib>
<contrib id="author-2" contrib-type="author" corresp="yes">
<name name-style="western"><surname>Kang</surname><given-names>Dae-Ki</given-names></name><xref ref-type="aff" rid="aff-2">2</xref><email>dkkang@dongseo.ac.kr</email></contrib>
<contrib id="author-3" contrib-type="author">
<name name-style="western"><surname>Al-Absi</surname><given-names>Mohammed Abdulhakim</given-names></name><xref ref-type="aff" rid="aff-3">3</xref></contrib>
<aff id="aff-1"><label>1</label><institution>Department of Smart Computing, Kyungdong University 46 4-gil</institution>, <addr-line>Bongpo, Gosung, Gangwon-do, 24764</addr-line>, <country>Korea</country></aff>
<aff id="aff-2"><label>2</label><institution>Division of Computer Engineering, Dongseo University</institution>, <addr-line>47 Jurye-ro, Sasang-gu, Busan, 47011</addr-line>, <country>Korea</country></aff>
<aff id="aff-3"><label>3</label><institution>Department of Computer Engineering, Dongseo University</institution>, <addr-line>47 Jurye-ro, Sasang-gu, Busan, 47011</addr-line>, <country>Korea</country></aff>
</contrib-group>
<author-notes>
<corresp id="cor1"><label>&#x002A;</label>Corresponding Author: Dae-Ki Kang. Email: <email>dkkang@dongseo.ac.kr</email></corresp>
</author-notes>
<pub-date pub-type="epub" date-type="pub" iso-8601-date="2022-08-16"><day>16</day>
<month>08</month>
<year>2022</year></pub-date>
<volume>74</volume>
<issue>1</issue>
<fpage>671</fpage>
<lpage>691</lpage>
<history>
<date date-type="received"><day>22</day><month>3</month><year>2022</year></date>
<date date-type="accepted"><day>09</day><month>6</month><year>2022</year></date>
</history>
<permissions>
<copyright-statement>&#x00A9; 2023 Al-Absi et al.</copyright-statement>
<copyright-year>2023</copyright-year>
<copyright-holder>Al-Absi et al.</copyright-holder>
<license xlink:href="https://creativecommons.org/licenses/by/4.0/">
<license-p>This work is licensed under a <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</ext-link>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
</license>
</permissions>
<self-uri content-type="pdf" xlink:href="TSP_CMC_30262.pdf"></self-uri>
<abstract>
<p>Sentiment analysis (AS) is one of the basic research directions in natural language processing (NLP), it is widely adopted for news, product review, and politics. Aspect-based sentiment analysis (ABSA) aims at identifying the sentiment polarity of a given target context, previous existing model of sentiment analysis possesses the issue of the insufficient exaction of features which results in low accuracy. Hence this research work develops a deep-semantic and contextual knowledge networks (DSCNet). DSCNet tends to exploit the semantic and contextual knowledge to understand the context and enhance the accuracy based on given aspects. At first temporal relationships are established then deep semantic knowledge and contextual knowledge are introduced. Further, a deep integration layer is introduced to measure the importance of features for efficient extraction of different dimensions. Novelty of DSCNet model lies in introducing the deep contextual. DSCNet is evaluated on three datasets i.e., Restaurant, Laptop, and Twitter dataset considering different deep learning (DL) metrics like precision, recall, accuracy, and Macro-F1 score. Also, comparative analysis is carried out with different baseline methods in terms of accuracy and Macro-F1 score. DSCNet achieves 92.59&#x0025; of accuracy on restaurant dataset, 86.99&#x0025; of accuracy on laptop dataset and 78.76&#x0025; of accuracy on Twitter dataset.</p>
</abstract>
<kwd-group kwd-group-type="author">
<kwd>Sentiment analysis</kwd>
<kwd>aspect-sa</kwd>
<kwd>deep learning</kwd>
<kwd>DSCNet</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec id="s1"><label>1</label><title>Introduction</title>
<p>Rapid development in information technology (IT) and the internet has made people not only retrieve the information but express their opinions as well. There is a different platform available for people to express their opinion; opinions like online reviews comprises valuable information for consumers as well as business. A large amount of data is accumulated through these opinions over the internet, opinions, and sentiment can be extracted through NLP and big data analysis [<xref ref-type="bibr" rid="ref-1">1</xref>]. Sentiment analysis is a process of understanding text meaning and analyzing the sentiment hidden in a given text, NLP is one of the popular technology to model disassemble reason extract, and classify. The main aim of sentiment analysis is to identify the attitudes of bipolar views based on targets in sentences [<xref ref-type="bibr" rid="ref-2">2</xref>].</p>
<p>Sentiment analysis is categorized on three distinctive levels that are shown in <xref ref-type="fig" rid="fig-1">Fig. 1</xref> and discussed further.
   <fig id="fig-1"><label>Figure 1</label><caption><title>Different categories of sentiment analysis</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_30262-fig-1.png"/></fig>
<list list-type="alpha-upper">
<list-item><label>A.</label><p>Document-SA: In this type of task, the aim is to determine the positive or negative opinions based on overall documents. It is one of the simplest ways to analyze sentiments. This task considers each document expresses an opinion on a single entity. However, it does not hold for a scenario where multiple views are evaluated in a single document.</p></list-item>
<list-item><label>B.</label><p>Sentence-SA: This type of task determines positive, negative, and neutral opinion in a given sentence, it comprises neutral opinion which is not available in document-SA. A neutral opinion is a sentiment that has no effecting opinion i.t. neither positive nor negative. Sentence-SA applies a similar assumption as of document-SA except that it is considered for a single sentence.</p></list-item>
<list-item><label>C.</label><p>ABSA (Aspect-SA): Sentence-SA and Document-SA are very effective when there is a sentence or document on a single entity. Further research shows that people talk with different attributes (aspect). In the case of each aspect, there are different opinions. Aspect-based-SA is a fine-grained analysis that usually exists in product reviews for product-based companies such as product review such as phones, cars, and others. It is considered as one of the fine approaches for business development, hence it has become one of the popular and widely available AS.</p></list-item>
</list></p>
<p>Aspect-based-SA is referred to as the task of identifying the various aspect in review and distinguishing the sentiment polarities (SP) towards aspects in an individual manner for extracting the fine-grained information. ABSA task possesses wide research value as it enables consumers to an evaluation of product or service from a comprehensive perspective to understand detailed and explicit knowledge of it [<xref ref-type="bibr" rid="ref-2">2</xref>]. Moreover, Neural network-based mechanisms have made a significant impact in comparison with the traditional approach of sentiment analysis. Recent studies on neural networks (NNs) have observed significant growth in Natural Language processing such as text summarization and machine translation, however still ABSA&#x2013;NN required to be studied in depth [<xref ref-type="bibr" rid="ref-3">3</xref>]. ABSA is categorized into two tasks, mining (aspect mining) and classification (aspect classification); aspect mining tends to extract the aspect words from each sentence, aspect mining has been exploited through unsupervised models, semi-supervised and supervised model whereas aspect classification aims at polarities prediction on the aspect. However, getting accurate sentiment is a major challenge in sentiment classification; for instance, &#x201C;Food was delicious, but the place was small&#x201D;. In this sentence, there are two distinctive aspects i.e., food and place and opinion words like &#x201C;delicious&#x201D; and &#x201C;small &#x201C;corresponds to polarity where &#x201C;delicious&#x201D; is positive sentiment polarity towards food and &#x201C;small&#x201D; is negative sentiment polarity towards the room.</p>
<p><xref ref-type="fig" rid="fig-2">Fig. 2</xref> shows the general framework followed by the research for classification of aspect-based sentiments, this general framework comprises three-phase namely aspect extraction, aspect sentiment analysis, and sentiment evaluation. Furthermore, apart from these three important blocks it also includes pre-processing steps such as discarding unnecessary information from corpus, tokenization stemming, and so on [<xref ref-type="bibr" rid="ref-4">4</xref>].</p>
<fig id="fig-2"><label>Figure 2</label><caption><title>General framework of sentiment analysis</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_30262-fig-2.png"/></fig>
<p>Through research it was observed that conventional sentiment models fail to observe the dynamic polarity recognition due to bilateral information lack, also emotions accompany different emotions. Moreover, in classification, 40&#x0025; of error occurs when aspects are not considered. In recent years, deep neural networks (DNN) have achieved great success in several applications. Deep learning is classified into convolutional neural networks (CNNs) [<xref ref-type="bibr" rid="ref-5">5</xref>], recurrent neural networks (RNNs) [<xref ref-type="bibr" rid="ref-6">6</xref>], reconvolutional neural networks (ReCNNs) [<xref ref-type="bibr" rid="ref-7">7</xref>], and memory networks (MNs) [<xref ref-type="bibr" rid="ref-8">8</xref>], detailed classification of deep learning is shown in <xref ref-type="fig" rid="fig-3">Fig. 3</xref></p>
<fig id="fig-3"><label>Figure 3</label><caption><title>Deep neural network categorization for Aspect-SA</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_30262-fig-3.png"/></fig>
<p>Moreover, <xref ref-type="fig" rid="fig-3">Fig. 3</xref> includes the direct deep learning architecture adopted for Aspect-SA, in addition to that attention mechanism has been adopted widely along with the above-mentioned architecture to enhance the feature extraction and context [<xref ref-type="bibr" rid="ref-9">9</xref>]. Other types of Aspect-SA mechanisms include the embedding approach and the transfer learning approach. Furthermore, considering the deep learning architecture, CNN is considered effective and possesses the capability to extract meaningful patterns. However, CNN fails to balance the contextual information and sequential order, this problem can be solved through RNN. Hence recent study on Aspect-SA has focused on RNN, all basic RNN model considers the particular situation, which is formed through many words, however, the target is not restricted to a single word. Thus, attention with long short- term memory (LSTM) is used where targets and context are into LSTM to understand the context, however, it is restricted to single instances [<xref ref-type="bibr" rid="ref-9">9</xref>]. Bidirectional-LSTM is introduced which comprises embedding layer attention layer, contextual layer, and output layer, aspect, and context can be embedded into to get the aspect and contextual output. Hence Bi-LSTM architecture has been exploited widely to exploit the Sentiment analysis, meanwhile, Bi-LSTM based architecture also has a limitation, and considering the review of other research work (discussed in related work), several challenges still exist such as ignoring the other data types such as emoji as it is heavily used by user; also classifying sentiments when there are different targets, pre-processing of data could also improvise accuracy [<xref ref-type="bibr" rid="ref-4">4</xref>]. Other major challenges are deep knowledge of semantic information and context knowledge; thus, this research work focuses on deep semantic information and deep context knowledge [<xref ref-type="bibr" rid="ref-2">2</xref>,<xref ref-type="bibr" rid="ref-10">10</xref>].</p>
<p>Word expresses various sentiments as it holds various semantics in various contexts; for instance, the word &#x201C;blue&#x201D; holds different semantics in sentences like &#x201C;He is so blue that none going to get better&#x201D; or &#x201C;I bought blue jeans&#x201D; or She said it was just a blue. Each sentence conveys different sentiments; moreover, the existing embedding mechanism tends to directly embed the lexicon to word representation this makes no difference in words in different sentences, this fails to provide proposed information of words in various sentences to utilize absolute embedding. Hence motivated by its application and challenges, this research work proposes DSCNet architecture. A further contribution of research work is given below.
<list list-type="bullet">
<list-item><p>DSCNet architecture incorporates semantic information and contextual knowledge in deep to enhance the aspect extraction and understanding of context.</p></list-item>
<list-item><p>Deep semantic knowledge helps in sharing the semantic information of different aspects and focuses on generating optimal features.</p></list-item>
<list-item><p>Deep contextual knowledge helps in focusing on the specific words in a more precise way, also a deep integration layer is introduced to integrate the efficiently extracted parts from each dimension.</p></list-item>
<list-item><p>DSCNet is developed to remember and provide contextual knowledge for given targeted aspects.</p></list-item>
<list-item><p>DSCNet is evaluated on SemEval dataset 2014 task 4 and twitter dataset, evaluation is carried out through on accuracy and Macro-F1</p></list-item>
</list></p>
<p>This research work is organized in the following way: the first section starts with an understanding of sentiment analysis, its types, and different challenges and approach used. This section concludes with the motivation and contribution of research work. The second section discusses different related work based on aspect-based sentiment analysis, third section designs and develops a neural network-based mechanism. The fourth section evaluates the proposed methodology considering different online reviews dataset.</p>
</sec>
<sec id="s2"><label>2</label><title>Related Work</title>
<p>Aspect Based Sentiment analysis is a basic task in SA field; it comprises two subtasks i.e., extraction and classification. ABSA considers targets and sentiments both at one time and the aim is for polarity determination of given aspect and sentence. Furthermore, with the development of NLP, SA has been given wide attention by researchers, this section discusses different aspect-based-SA work along with their shortcomings.</p>
<p>In general Aspect based-SA is categorized into the category of lexicon based [<xref ref-type="bibr" rid="ref-11">11</xref>], machine learning (ML) [<xref ref-type="bibr" rid="ref-12">12</xref>] based, and DL-based. A traditional approach such as parts of speech (PoS) [<xref ref-type="bibr" rid="ref-13">13</xref>] and bag of words (BoW) (Bag of Words) [<xref ref-type="bibr" rid="ref-14">14</xref>] focus on utilizing the feature engineering phenomena; also for training machine learning classifiers were used such as NN, support vector machine (SVM) and Na&#x00EF;ve Bayesian. Above discussed approach is dependent on handcrafted features and thus it is labor-intensive. Further, the deep learning approach has been an evolutionary phenomenon for sentiment analysis through word embedding, here we discuss some of the recent successful research along with their shortcoming. In [<xref ref-type="bibr" rid="ref-15">15</xref>], Bidirectional gated recurrent unit (GRU) model was developed for complete feature mining for the ABSA task, Bidirectional GRU network is used for acquiring the dependency-based sentence SA along with their corresponding terms. Further, it proposes the learning feature of sentiment polarity in given sentences; despite enhancement in feature mining, accuracy did not improve much, and also it was proven to be time-consuming. Hence [<xref ref-type="bibr" rid="ref-16">16</xref>] developed a novel integration method of CNN and GRU to utilize the local features for efficient processing. Moreover, GRU is used for long-term dependency learning and CNN is used for the generation of local features. However, this model faces the issue of dependency learning which is enhanced in [<xref ref-type="bibr" rid="ref-17">17</xref>], it introduced dependency-related phenomena to identify the dependency-related feature for given aspect term; dependency parse tree is designed and dependency-related feature is integrated into BiLSTM and CNN. Furthermore, this research work observed that sentiment features of a given aspect help in discriminating the sentiment polarity. Designing a tree makes the model too complicated for enhancement in dependency-related phenomena. In [<xref ref-type="bibr" rid="ref-18">18</xref>], a model named effective adaptive transfer network (EATN) was presented, EATN utilized the understanding of the incorporation of correlation between different domains. Furthermore, it proposes domain attention module (DAM) for learning the common feature from a given source domain and further targeting the classification. DAM is designed for two tasks first is the sentiment classification task for sentiment knowledge and the second is domain classification. To reduce the feature dependency multiple kernel mechanism is selected. This mechanism faces an issue with labeling, thus [<xref ref-type="bibr" rid="ref-19">19</xref>] introduces Ecolabels which is a semiautomatic mechanism for the textual ER to provide the large-scale annotation considering English emotion corporate in different generations aiming robust reliability. It comprises two steps, first step includes the automatic process of pre-annotation of the unlabeled sentence along with optimized emotional categories. The second step includes a manual process of refinement where humans determine which is more dominant emotions in given pre-defined possibilities. Further, [<xref ref-type="bibr" rid="ref-20">20</xref>] proposed CNN along with a sentiment module known as SentiCNN, SentiCNN analyses different sentiments in sentences with sentiment and contextual information of words. Contextual words are identified through word embeddings and sentiment information is identified through lexicons. Moreover, a highway network is incorporated for combining the contextual and sentiment information from given sentences through developing the connection among features of sentences and words; also attention mechanism is used based on lexicons to identify the eminent sentiment indicators to make more effective predictions. [<xref ref-type="bibr" rid="ref-21">21</xref>] adopted different mechanism of end-to-end segmentation model named SEGBOT, SEGBOT uses BiRNN for encoding input text sequence, alter another RNN is used along with pointer network to identify the text boundaries in the given input sequence, hence it is not dependent on handcrafted feature and SEGBOT handles variable size issue and sparse boundary tags. To evaluate the model, document level and sentence level analysis are considered.</p>
<p>There have been enormous efforts in the fields of ABSA, however, still there exists research gap analysis that needs to be solved. These gaps include improvisation in the extraction capability of the ABSA model. Hence, in this research work, we used deep semantic and deep contextual knowledge for extraction and contextual understanding. The architecture formulation of DSCNet is designed in the next section of the research work.</p>
</sec>
<sec id="s3"><label>3</label><title>Proposed Methodology</title>
<p>The deep learning approach has shown wide application in ABSA task classification as deep learning exploits the aspect and context to classify the sentiment polarity. Hence, this research adopts the deep learning domain and introduces DSCNet. DSCNet exploits the aspect and target sentiment.</p>
<p><xref ref-type="fig" rid="fig-4">Fig. 4</xref> shows DSCNet architecture, it comprises deep features, deep attention layer, DSCNet and Deep Integration layer. DSCNet is primary component of our research model, and it is capable of delivering the contextual knowledge of an aspect that has similar meanings through deep exploration of its features.</p>
<fig id="fig-4"><label>Figure 4</label><caption><title>DSCNet architecture</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_30262-fig-4.png"/></fig>
<sec id="s3_1"><label>3.1</label><title>Designing Task</title>
<p>Let&#x2019;s consider two distinctive entity sentences denoted through <inline-formula id="ieqn-1"><mml:math id="mml-ieqn-1"><mml:mi>u</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mspace width="thickmathspace" /><mml:msub><mml:mi>y</mml:mi><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>.</mml:mo><mml:mo>,</mml:mo><mml:mspace width="thickmathspace" /><mml:msub><mml:mi>y</mml:mi><mml:mrow><mml:mi>P</mml:mi></mml:mrow></mml:msub><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula> and aspects <inline-formula id="ieqn-2"><mml:math id="mml-ieqn-2"><mml:mi>c</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mspace width="thickmathspace" /><mml:mspace width="thickmathspace" /><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>.</mml:mo><mml:mo>.</mml:mo><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mi>O</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula> in a pair of <inline-formula id="ieqn-3"><mml:math id="mml-ieqn-3"><mml:mrow><mml:mo>[</mml:mo><mml:mi>u</mml:mi><mml:mo>,</mml:mo><mml:mi>b</mml:mi><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula>. Aspect-based sentiment analysis and classification aim to analyze the sentiment polarity of a given aspect in the sentence.</p>
</sec>
<sec id="s3_2"><label>3.2</label><title>Word Interpretation and Temporal Relation Analysis</title>
<p>To interpret the words inefficient manner through deep contextual knowledge, a glove model is utilized where each word is mapped into the vector space as <inline-formula id="ieqn-4"><mml:math id="mml-ieqn-4"><mml:mi>z</mml:mi><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>&#x03F1;</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mrow><mml:mo>&#x230A;</mml:mo><mml:mi>X</mml:mi><mml:mo>&#x230B;</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:math></inline-formula> is designed and lookup matrix concerning embedding <inline-formula id="ieqn-5"><mml:math id="mml-ieqn-5"><mml:mi>L</mml:mi><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>&#x03F1;</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mrow><mml:mo>&#x230A;</mml:mo><mml:mi>V</mml:mi><mml:mo>&#x230B;</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:math></inline-formula> is designed. In introduced terms, <inline-formula id="ieqn-6"><mml:math id="mml-ieqn-6"><mml:mrow><mml:mo>&#x230A;</mml:mo><mml:mi>X</mml:mi><mml:mo>&#x230B;</mml:mo></mml:mrow></mml:math></inline-formula> indicates the inherent size of vocabulary and <inline-formula id="ieqn-7"><mml:math id="mml-ieqn-7"><mml:mi>&#x03F1;</mml:mi></mml:math></inline-formula> indicates the embedding dimension. Further, temporal relationship is observed through Bi-LSTM, Bi-LSTM leverages long term dependencies to establish the aspect and hidden semantics of words. Let&#x2019;s Consider <inline-formula id="ieqn-8"><mml:math id="mml-ieqn-8"><mml:msubsup><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> as the aspect concerning <inline-formula id="ieqn-9"><mml:math id="mml-ieqn-9"><mml:mi>l</mml:mi><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:math></inline-formula> aspect word and <inline-formula id="ieqn-10"><mml:math id="mml-ieqn-10"><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> as the word based features; word based features are achieved through the integration of backward and forward LSTM.</p>
<p><disp-formula id="eqn-1"><label>(1)</label><mml:math id="mml-eqn-1" display="block"><mml:mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mml:mtr><mml:mtd><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msup><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">L</mml:mi></mml:mrow></mml:mrow><mml:mrow><mml:mo stretchy="false">&#x2191;</mml:mo></mml:mrow></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>z</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>;</mml:mo><mml:msup><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">L</mml:mi></mml:mrow></mml:mrow><mml:mrow><mml:mo stretchy="false">&#x2193;</mml:mo></mml:mrow></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>z</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>]</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msubsup><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msup><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">L</mml:mi></mml:mrow></mml:mrow><mml:mrow><mml:mo stretchy="false">&#x2191;</mml:mo></mml:mrow></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:msubsup><mml:mrow><mml:mi>z</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>)</mml:mo></mml:mrow><mml:mo>;</mml:mo><mml:msup><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">L</mml:mi></mml:mrow></mml:mrow><mml:mrow><mml:mo stretchy="false">&#x2193;</mml:mo></mml:mrow></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:msubsup><mml:mrow><mml:mi>z</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>)</mml:mo></mml:mrow><mml:mo>]</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>In the above equation, <inline-formula id="ieqn-11"><mml:math id="mml-ieqn-11"><mml:msubsup><mml:mrow><mml:mi>z</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> indicates embedding concerning <inline-formula id="ieqn-12"><mml:math id="mml-ieqn-12"><mml:mi>l</mml:mi><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:math></inline-formula> word and <inline-formula id="ieqn-13"><mml:math id="mml-ieqn-13"><mml:msub><mml:mi>z</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> indicates embedding concerning <inline-formula id="ieqn-14"><mml:math id="mml-ieqn-14"><mml:mi>k</mml:mi><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:math></inline-formula> word; indicates the vector-based integration of both sides of LSTM. Considering sentence <italic>s</italic> and aspect <italic>a</italic>, its context and output are computed through the below equation.
<disp-formula id="eqn-2"><label>(2)</label><mml:math id="mml-eqn-2" display="block"><mml:mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mml:mtr><mml:mtd><mml:mi>J</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msub><mml:mo>]</mml:mo></mml:mrow><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x03F1;</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mi>P</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msup><mml:mi>J</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msub><mml:mo>]</mml:mo></mml:mrow><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x03F1;</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mi>O</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>Matrix is designed for encoding positions for the weights of words and it is denoted as <inline-formula id="ieqn-15"><mml:math id="mml-ieqn-15"><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>g</mml:mi></mml:mrow></mml:msup><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>&#x03F1;</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mrow><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:mrow></mml:msup><mml:mspace width="thickmathspace" /></mml:math></inline-formula> with <inline-formula id="ieqn-16"><mml:math id="mml-ieqn-16"><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> being the maximum threshold length of sentence. Further, in <inline-formula id="ieqn-17"><mml:math id="mml-ieqn-17"><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> position embedding <inline-formula id="ieqn-18"><mml:math id="mml-ieqn-18"><mml:msubsup><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> indicates the distance <italic>i</italic> of word to defined aspect, thus position weight is computed as:
<disp-formula id="eqn-3"><label>(3)</label><mml:math id="mml-eqn-3" display="block"><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mi>v</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mi mathvariant="normal">&#x039E;</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>d</mml:mi><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:msubsup><mml:mrow><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup><mml:mo>)</mml:mo></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula></p>
<p>The above equation comprises bias and weighted matrix denoted by <inline-formula id="ieqn-19"><mml:math id="mml-ieqn-19"><mml:msub><mml:mi>d</mml:mi><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> and <inline-formula id="ieqn-20"><mml:math id="mml-ieqn-20"><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msub><mml:mspace width="thickmathspace" /></mml:math></inline-formula> respectively; also <inline-formula id="ieqn-21"><mml:math id="mml-ieqn-21"><mml:mi mathvariant="normal">&#x039E;</mml:mi></mml:math></inline-formula> is the activation function. To optimize as the learnable parameter, <inline-formula id="ieqn-22"><mml:math id="mml-ieqn-22"><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> is initialized and updated. Context output is obtained as:
<disp-formula id="eqn-4"><label>(4)</label><mml:math id="mml-eqn-4" display="block"><mml:mi>J</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>.</mml:mo><mml:mo>,</mml:mo><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mi>P</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:msubsup><mml:mrow><mml:mi>y</mml:mi></mml:mrow><mml:mrow><mml:mi>P</mml:mi></mml:mrow><mml:mrow><mml:mi>r</mml:mi></mml:mrow></mml:msubsup><mml:mo>]</mml:mo></mml:mrow></mml:math></disp-formula></p>
</sec>
<sec id="s3_3"><label>3.3</label><title>Deep Attention Layer</title>
<p>To understand the aspect considering context, the influence of different aspects on the context has to be measured, hence parameter score is assigned to the words and computed. Attention score helps in getting the aspect-based features. Considering aspect word as <inline-formula id="ieqn-23"><mml:math id="mml-ieqn-23"><mml:msubsup><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> with context <inline-formula id="ieqn-24"><mml:math id="mml-ieqn-24"><mml:msub><mml:mi>j</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula>, is computed.
<disp-formula id="eqn-5"><label>(5)</label><mml:math id="mml-eqn-5" display="block"><mml:msubsup><mml:mrow><mml:mi>w</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mi mathvariant="normal">&#x039E;</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>A</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msubsup><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi></mml:mrow><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msubsup><mml:mo>;</mml:mo><mml:msub><mml:mi>l</mml:mi><mml:mrow><mml:mi>m</mml:mi></mml:mrow></mml:msub><mml:mo>]</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:msub><mml:mi>f</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula></p>
<p>Similarly, considering both context and aspect, attention weights are computed with bias and context words; also <inline-formula id="ieqn-25"><mml:math id="mml-ieqn-25"><mml:mi mathvariant="normal">&#x039E;</mml:mi></mml:math></inline-formula> indicates the hyperbolic tangent function.
<disp-formula id="eqn-6"><label>(6)</label><mml:math id="mml-eqn-6" display="block"><mml:msubsup><mml:mrow><mml:mi>c</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:munderover><mml:mrow><mml:mo movablelimits="false">&#x2211;</mml:mo></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>O</mml:mi></mml:mrow></mml:munderover><mml:mo>&#x2061;</mml:mo><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>Considering the above two equations, another parameter <inline-formula id="ieqn-26"><mml:math id="mml-ieqn-26"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> is considered which represents the aspect-based feature; meanwhile <inline-formula id="ieqn-27"><mml:math id="mml-ieqn-27"><mml:mi>exp</mml:mi></mml:math></inline-formula> repesents mathematical exponential function, aspect based feature are computed considering whole context
<disp-formula id="eqn-7"><label>(7)</label><mml:math id="mml-eqn-7" display="block"><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:munderover><mml:mrow><mml:mo movablelimits="false">&#x2211;</mml:mo></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>O</mml:mi></mml:mrow></mml:munderover><mml:mo>&#x2061;</mml:mo><mml:msubsup><mml:mrow><mml:mi>c</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>.</mml:mo><mml:msubsup><mml:mrow><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mspace width="thickmathspace" /><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>Further aspect-based feature with max-pooling and context words are computed through <inline-formula id="ieqn-28"><mml:math id="mml-ieqn-28"><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>b</mml:mi></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>b</mml:mi></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>.</mml:mo><mml:mo>.</mml:mo><mml:mo>,</mml:mo><mml:msubsup><mml:mrow><mml:mi>t</mml:mi></mml:mrow><mml:mrow><mml:mi>P</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>]</mml:mo></mml:mrow><mml:mspace width="thickmathspace" /><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>m</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mi>O</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula>. Max-pooling operation is carried out for eminent feature extraction from a different dimension and formed with <inline-formula id="ieqn-29"><mml:math id="mml-ieqn-29"><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mspace width="thickmathspace" /><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>m</mml:mi></mml:mrow></mml:msup><mml:mspace width="thickmathspace" /></mml:math></inline-formula> where <inline-formula id="ieqn-30"><mml:math id="mml-ieqn-30"><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> comprises recent aspect information, this information is stored in DSCNet.
<disp-formula id="eqn-8"><label>(8)</label><mml:math id="mml-eqn-8" display="block"><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mo movablelimits="true" form="prefix">max</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula></p>
</sec>
<sec id="s3_4"><label>3.4</label><title>Context Based Feature Design with Respect to Aspect</title>
<p>Considering context influence in aspect, aspect is designed and denoted as <inline-formula id="ieqn-31"><mml:math id="mml-ieqn-31"><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> ; this comprises domain knowledge; further, in the case of each context word, weights are computed as:
<disp-formula id="eqn-9"><label>(9)</label><mml:math id="mml-eqn-9" display="block"><mml:msubsup><mml:mi>c</mml:mi><mml:mi>k</mml:mi><mml:mi>e</mml:mi></mml:msubsup><mml:mo>=</mml:mo><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msubsup><mml:mi>u</mml:mi><mml:mi>k</mml:mi><mml:mi>e</mml:mi></mml:msubsup><mml:mo stretchy="false">)</mml:mo><mml:mo>.</mml:mo><mml:mspace width="thickmathspace" /><mml:mspace width="thickmathspace" /><mml:mo stretchy="false">(</mml:mo><mml:munderover><mml:mo movablelimits="false">&#x2211;</mml:mo><mml:mrow><mml:mi>m</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>P</mml:mi></mml:mrow></mml:munderover><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mi>m</mml:mi></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup><mml:mo>)</mml:mo></mml:mrow><mml:mspace width="thinmathspace" /><mml:msub><mml:mi>W</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msub><mml:mspace width="thickmathspace" /><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>I</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mn>4</mml:mn><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>m</mml:mi></mml:mrow></mml:msup><mml:mspace width="thickmathspace" /><mml:msub><mml:mi>b</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msub><mml:mspace width="thickmathspace" /><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace" /><mml:mi>R</mml:mi></mml:math></disp-formula>
<disp-formula id="ueqn-101"><mml:math id="mml-ueqn-101" display="block"><mml:msub><mml:mi>W</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msub><mml:mspace width="thickmathspace" /><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>I</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mn>4</mml:mn><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>m</mml:mi></mml:mrow></mml:msup></mml:math></disp-formula>
<disp-formula id="ueqn-102"><mml:math id="mml-ueqn-102" display="block"><mml:msub><mml:mi>b</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msub><mml:mspace width="thickmathspace" /><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace" /><mml:mi>R</mml:mi></mml:math></disp-formula></p>
<p>Also, the score is computed as:
<disp-formula id="eqn-10"><label>(10)</label><mml:math id="mml-eqn-10" display="block"><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mi>tanh</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msub><mml:mi>i</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub><mml:mo>;</mml:mo><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>]</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:msub><mml:mi>d</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>Moreover, considering the deep contextual knowledge, context-based features is formulated as:
<disp-formula id="eqn-11"><label>(11)</label><mml:math id="mml-eqn-11" display="block"><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:msubsup><mml:mrow><mml:mo>&#x2211;</mml:mo></mml:mrow><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>P</mml:mi></mml:mrow></mml:msubsup><mml:msubsup><mml:mrow><mml:mi>c</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msubsup><mml:mo>.</mml:mo><mml:msub><mml:mi>i</mml:mi><mml:mrow><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></disp-formula></p>
</sec>
<sec id="s3_5"><label>3.5</label><title>Deep Semantic Knowledge and Deep Contextual Knowledge</title>
<p>To obtain the optimal semantic information, two embedding metrics are considered concerning domain knowledge and semantic information. These two matrices i.e., memory matrix and aspect alignment matrix are utilized for domain knowledge and semantic information. It comprises three phases extraction, updation, and integration. Furthermore, DSCNet considers two matrix i.e., optimal memory matrix and aspect metrix. Further, exploitation of semantic knowledge and contextual knowledge is carried out as given in <xref ref-type="fig" rid="fig-5">Fig. 5</xref>.</p>
<fig id="fig-5"><label>Figure 5</label><caption><title>Deep semantic and contextual knowledge exploitation process</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_30262-fig-5.png"/></fig>
<sec id="s3_5_1"><label>3.5.1</label><title>Deep Extraction</title>
<p>As discussed earlier, let&#x2019;s consider optimal memory matrix that consists of aspect embedding of <inline-formula id="ieqn-32"><mml:math id="mml-ieqn-32"><mml:msubsup><mml:mrow><mml:mi>e</mml:mi></mml:mrow><mml:mrow><mml:mi>i</mml:mi></mml:mrow><mml:mrow><mml:mi>a</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> and <italic>I</italic> as aspect embedding in <inline-formula id="ieqn-33"><mml:math id="mml-ieqn-33"><mml:msup><mml:mi>E</mml:mi><mml:mrow><mml:mi>a</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula>. Moreover, a relation among the aspect embedding and aspect is measured through the introduced parameter weight vector <italic>c</italic> which is computed as
<disp-formula id="eqn-12"><label>(12)</label><mml:math id="mml-eqn-12" display="block"><mml:mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mml:mtr><mml:mtd /><mml:mtd><mml:mi>e</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mi>&#x2136;</mml:mi></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>c</mml:mi><mml:mi>o</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:mspace width="thickmathspace" /><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>+</mml:mo><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>c</mml:mi><mml:mi>o</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mrow><mml:mtext mathvariant="italic">Where</mml:mtext></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>c</mml:mi><mml:mi>o</mml:mi></mml:mrow></mml:msub><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>I</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mn>4</mml:mn><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>m</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>c</mml:mi><mml:mi>o</mml:mi></mml:mrow></mml:msub><mml:mi>&#x03F5;</mml:mi><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>L</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>Further, mean pooling is carried out on the obtained output (aspect) to represent the aspect in the above equation. The above equation consists of bias and weight matrix denoted as <inline-formula id="ieqn-34"><mml:math id="mml-ieqn-34"><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>c</mml:mi><mml:mi>o</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> and <inline-formula id="ieqn-35"><mml:math id="mml-ieqn-35"><mml:mspace width="thickmathspace" /><mml:msub><mml:mi>c</mml:mi><mml:mrow><mml:mi>c</mml:mi><mml:mi>o</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> respectively, also <inline-formula id="ieqn-36"><mml:math id="mml-ieqn-36"><mml:mrow><mml:mi>&#x2136;</mml:mi></mml:mrow></mml:math></inline-formula> indicates the normalized exponential function. Considering vector <inline-formula id="ieqn-37"><mml:math id="mml-ieqn-37"><mml:msup><mml:mi>x</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> with weights <inline-formula id="ieqn-38"><mml:math id="mml-ieqn-38"><mml:msup><mml:mi>G</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> and vector <italic>e</italic> is represented as follows:
<disp-formula id="eqn-13"><label>(13)</label><mml:math id="mml-eqn-13" display="block"><mml:msup><mml:mi>x</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msup><mml:mi>G</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>)</mml:mo></mml:mrow><mml:mo>.</mml:mo><mml:mi>e</mml:mi></mml:math></disp-formula></p>
<p>Another matrix, optimal memory matrix denoted as <inline-formula id="ieqn-39"><mml:math id="mml-ieqn-39"><mml:msup><mml:mi>G</mml:mi><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msup><mml:mi>&#x03F5;</mml:mi><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>&#x03F1;</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mi>K</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> with <italic>K</italic> memory embedding and each one is related to aspect-based embedding ; <inline-formula id="ieqn-40"><mml:math id="mml-ieqn-40"><mml:msubsup><mml:mrow><mml:mi>g</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> holds the context information and context information is extracted through context knowledge. Extracted feature-based information is formulated as follows:
<disp-formula id="eqn-14"><label>(14)</label><mml:math id="mml-eqn-14" display="block"><mml:mi>x</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow><mml:mo>=</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>G</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>.</mml:mo><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:math></disp-formula></p>
<p>Both matrix initialization is performed randomly; the difference between these two is aspect alignment matrix acts as a learnable parameter and the memory matrix acts as a non-learnable parameter. Aspect alignment matrix is further updated using the adjacent aspects embedding and the memory matrix is updated considering the context information.</p>
</sec>
<sec id="s3_5_2"><label>3.5.2</label><title>Deep Updation</title>
<p>To compute extracted feature-based information, semantics is given to the network. Aspect based features <inline-formula id="ieqn-41"><mml:math id="mml-ieqn-41"><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> comprises the context information, further memory embedding is required to be updated is selected from the matrix <inline-formula id="ieqn-42"><mml:math id="mml-ieqn-42"><mml:msup><mml:mi>G</mml:mi><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> with a higher probability.
<disp-formula id="eqn-15"><label>(15)</label><mml:math id="mml-eqn-15" display="block"><mml:mi mathvariant="normal">&#x2135;</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mtext mathvariant="italic">argmax</mml:mtext></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mi>e</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>In the above equation, <inline-formula id="ieqn-43"><mml:math id="mml-ieqn-43"><mml:mi mathvariant="normal">&#x2135;</mml:mi><mml:mspace width="thickmathspace" /><mml:mi>&#x03F5;</mml:mi><mml:mrow><mml:mo>[</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mi>K</mml:mi><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula> index of weight vector <inline-formula id="ieqn-44"><mml:math id="mml-ieqn-44"><mml:mi>e</mml:mi><mml:mspace width="thickmathspace" /></mml:math></inline-formula>, then memory embedding concerning <inline-formula id="ieqn-45"><mml:math id="mml-ieqn-45"><mml:mi mathvariant="normal">&#x2135;</mml:mi></mml:math></inline-formula> and further updates as:
<disp-formula id="eqn-16"><label>(16)</label><mml:math id="mml-eqn-16" display="block"><mml:msubsup><mml:mrow><mml:mi>g</mml:mi></mml:mrow><mml:mrow><mml:mi mathvariant="normal">&#x2135;</mml:mi></mml:mrow><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mi>tanh</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>w</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>+</mml:mo><mml:msubsup><mml:mrow><mml:mi>g</mml:mi></mml:mrow><mml:mrow><mml:mi mathvariant="normal">&#x2135;</mml:mi></mml:mrow><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msubsup><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula></p>
<p>In above equation <inline-formula id="ieqn-46"><mml:math id="mml-ieqn-46"><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>w</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> indicates the weight matrix.
<disp-formula id="eqn-17"><label>(17)</label><mml:math id="mml-eqn-17" display="block"><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mi>tanh</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>h</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:msup><mml:mi>x</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>+</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>h</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:msup><mml:mi>x</mml:mi><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msup><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula></p>
</sec>
<sec id="s3_5_3"><label>3.5.3</label><title>Integration</title>
<p>In this step, deep semantic knowledge and deep contextual knowledge are integrated, these two are integrated to achieve context based aspect and denoted as <inline-formula id="ieqn-47"><mml:math id="mml-ieqn-47"><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula>.
<disp-formula id="eqn-18"><label>(18)</label><mml:math id="mml-eqn-18" display="block"><mml:mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mml:mtr><mml:mtd /><mml:mtd><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:mi>tanh</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>h</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:msup><mml:mi>x</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>+</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>h</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:msup><mml:mi>x</mml:mi><mml:mrow><mml:mi>o</mml:mi></mml:mrow></mml:msup><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mrow><mml:mtext mathvariant="italic">where</mml:mtext></mml:mrow><mml:mspace width="thickmathspace" /><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mi>&#x03F5;</mml:mi><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>&#x03F1;</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mi>K</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p><inline-formula id="ieqn-48"><mml:math id="mml-ieqn-48"><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>h</mml:mi></mml:mrow></mml:msub><mml:mi>&#x03F5;</mml:mi><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>&#x03F1;</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mi>K</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> indicates the weight matrix and <inline-formula id="ieqn-49"><mml:math id="mml-ieqn-49"><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> indicates the primary function for the attention to design the optimal features.</p>
</sec>
</sec>
<sec id="s3_6"><label>3.6</label><title>Deep Integration Layer</title>
<p>The fine-grained Integration layer is designed to compute the importance of features concerning <inline-formula id="ieqn-50"><mml:math id="mml-ieqn-50"><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula>; original sentence based features is computed using <inline-formula id="ieqn-51"><mml:math id="mml-ieqn-51"><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>q</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> through performing a max-pool operation. Further, score parameters concerning weights are computed as:
<disp-formula id="eqn-19"><label>(19)</label><mml:math id="mml-eqn-19" display="block"><mml:mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mml:mtr><mml:mtd /><mml:mtd><mml:msup><mml:mi>u</mml:mi><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mi>N</mml:mi><mml:mi>E</mml:mi><mml:mi>F</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>q</mml:mi></mml:mrow></mml:msup><mml:mo>]</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mrow><mml:mover><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup><mml:mo>&#x00A8;</mml:mo></mml:mover></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mi>s</mml:mi><mml:mi>u</mml:mi><mml:mi>c</mml:mi><mml:mi>h</mml:mi><mml:mspace width="thinmathspace" /><mml:mi>t</mml:mi><mml:mi>h</mml:mi><mml:mi>a</mml:mi><mml:mi>t</mml:mi><mml:mspace width="thinmathspace" /><mml:msup><mml:mi>u</mml:mi><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msup><mml:mspace width="thickmathspace" /><mml:mo>&#x2208;</mml:mo><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>m</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mn>3</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mi>a</mml:mi><mml:mi>n</mml:mi><mml:mi>d</mml:mi><mml:mspace width="thinmathspace" /><mml:mrow><mml:mo>[</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>q</mml:mi></mml:mrow></mml:msup><mml:mo>]</mml:mo></mml:mrow><mml:mspace width="thickmathspace" /><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mi>d</mml:mi><mml:mi>i</mml:mi><mml:mi>m</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mn>3</mml:mn></mml:mrow></mml:msup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>In above equation <inline-formula id="ieqn-52"><mml:math id="mml-ieqn-52"><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> indicates the weight matrix and <inline-formula id="ieqn-53"><mml:math id="mml-ieqn-53"><mml:msubsup><mml:mrow><mml:mi>J</mml:mi></mml:mrow><mml:mrow><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">G</mml:mi></mml:mrow></mml:mrow></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> is utilized from the previous equation, <inline-formula id="ieqn-54"><mml:math id="mml-ieqn-54"><mml:mi>N</mml:mi><mml:mi>E</mml:mi><mml:mi>F</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mrow><mml:mtext mathvariant="italic">Normalized</mml:mtext></mml:mrow><mml:mspace width="thickmathspace" /><mml:mrow><mml:mtext mathvariant="italic">exponential</mml:mtext></mml:mrow><mml:mspace width="thickmathspace" /><mml:mrow><mml:mtext mathvariant="italic">funciton</mml:mtext></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> is applied on the dimension level for different features and optimization is observed through integrating <inline-formula id="ieqn-55"><mml:math id="mml-ieqn-55"><mml:mrow><mml:mo>[</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>e</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>q</mml:mi></mml:mrow></mml:msup><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula>
<disp-formula id="eqn-20"><label>(20)</label><mml:math id="mml-eqn-20" display="block"><mml:mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mml:mtr><mml:mtd /><mml:mtd><mml:mrow><mml:mi mathvariant="double-struck">Z</mml:mi></mml:mrow><mml:mo>=</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msubsup><mml:mo>&#x2299;</mml:mo><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>t</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>+</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msubsup><mml:mo>&#x2299;</mml:mo><mml:msup><mml:mi>r</mml:mi><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msup><mml:mo>+</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mn>3</mml:mn></mml:mrow><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msubsup><mml:mo>&#x2299;</mml:mo><mml:msup><mml:mi>r</mml:mi><mml:mrow><mml:mi>q</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mrow><mml:mtext mathvariant="italic">where</mml:mtext></mml:mrow><mml:mspace width="thickmathspace" /><mml:msup><mml:mi>u</mml:mi><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msup><mml:mo>=</mml:mo><mml:mrow><mml:mo>[</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:msubsup><mml:mrow><mml:mi>u</mml:mi></mml:mrow><mml:mrow><mml:mn>3</mml:mn></mml:mrow><mml:mrow><mml:mi>h</mml:mi><mml:mi>w</mml:mi></mml:mrow></mml:msubsup><mml:mo>]</mml:mo></mml:mrow></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>In above equations <inline-formula id="ieqn-56"><mml:math id="mml-ieqn-56"><mml:mo>&#x2299;</mml:mo></mml:math></inline-formula> indicates element base multiplication, later <inline-formula id="ieqn-57"><mml:math id="mml-ieqn-57"><mml:mrow><mml:mi mathvariant="double-struck">Z</mml:mi></mml:mrow></mml:math></inline-formula> is given to label sampling along with polarity.
<disp-formula id="eqn-21"><label>(21)</label><mml:math id="mml-eqn-21" display="block"><mml:mtable columnalign="right left right left right left right left right left right left" rowspacing="3pt" columnspacing="0em 2em 0em 2em 0em 2em 0em 2em 0em 2em 0em" displaystyle="true"><mml:mtr><mml:mtd /><mml:mtd><mml:mi>r</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mi>&#x2136;</mml:mi></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>y</mml:mi><mml:mrow><mml:mi>e</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>.</mml:mo><mml:mrow><mml:mi mathvariant="double-struck">Z</mml:mi></mml:mrow><mml:mo>+</mml:mo><mml:msub><mml:mi>d</mml:mi><mml:mrow><mml:mi>e</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mrow><mml:mtext mathvariant="italic">where</mml:mtext></mml:mrow><mml:mspace width="thickmathspace" /><mml:mi>r</mml:mi><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>m</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:mspace width="thinmathspace" /><mml:mspace width="thickmathspace" /><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>f</mml:mi></mml:mrow></mml:msub><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>I</mml:mi><mml:mo>&#x00D7;</mml:mo><mml:mn>2</mml:mn><mml:mi>&#x03F1;</mml:mi></mml:mrow></mml:msup><mml:mspace width="thickmathspace" /><mml:mi>a</mml:mi><mml:mi>n</mml:mi><mml:mi>d</mml:mi><mml:mspace width="thickmathspace" /><mml:msub><mml:mi>d</mml:mi><mml:mrow><mml:mi>e</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>T</mml:mi><mml:mrow><mml:mi>m</mml:mi></mml:mrow></mml:msup></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<p>In the above equation, a probability distribution is utilized for polarities; <inline-formula id="ieqn-58"><mml:math id="mml-ieqn-58"><mml:msub><mml:mi>Y</mml:mi><mml:mrow><mml:mi>f</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> and <inline-formula id="ieqn-59"><mml:math id="mml-ieqn-59"><mml:msub><mml:mi>d</mml:mi><mml:mrow><mml:mi>e</mml:mi><mml:mi>n</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> indicates weight matrix and bias. Further, sentiment polarities are denoted through <inline-formula id="ieqn-60"><mml:math id="mml-ieqn-60"><mml:mi>m</mml:mi></mml:math></inline-formula>.</p>
</sec>
<sec id="s3_7"><label>3.7</label><title>DSCNet Model Training</title>
<p>To distinguish among different aspect embedding, regularization parameter <inline-formula id="ieqn-61"><mml:math id="mml-ieqn-61"><mml:mspace width="thickmathspace" /><mml:mi>T</mml:mi></mml:math></inline-formula> is used to compute loss function with <inline-formula id="ieqn-62"><mml:math id="mml-ieqn-62"><mml:mi>I</mml:mi><mml:mi>D</mml:mi><mml:mi>M</mml:mi></mml:math></inline-formula> as identity matrix.
<disp-formula id="eqn-22"><label>(22)</label><mml:math id="mml-eqn-22" display="block"><mml:mi>T</mml:mi><mml:mo>=</mml:mo><mml:mo stretchy="false">[</mml:mo><mml:msubsup><mml:mi>G</mml:mi><mml:mi>p</mml:mi><mml:mi>c</mml:mi></mml:msubsup><mml:mo>.</mml:mo><mml:mspace width="thickmathspace" /><mml:msubsup><mml:mi>G</mml:mi><mml:mi>p</mml:mi><mml:mrow><mml:mi>c</mml:mi><mml:mi>V</mml:mi></mml:mrow></mml:msubsup><mml:mo>&#x2212;</mml:mo><mml:mi>I</mml:mi><mml:mi>D</mml:mi><mml:mi>M</mml:mi><mml:msubsup><mml:mo stretchy="false">]</mml:mo><mml:mrow><mml:mi>H</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:math></disp-formula></p>
<p>In the above equation, <inline-formula id="ieqn-63"><mml:math id="mml-ieqn-63"><mml:msubsup><mml:mrow><mml:mi>G</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mi>c</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> indicates normalization of L2 indicates Euclidian normalization; dot product is carried out among different embedding of a matrix. Log loss <inline-formula id="ieqn-64"><mml:math id="mml-ieqn-64"><mml:mi>L</mml:mi><mml:mspace width="thickmathspace" /></mml:math></inline-formula> is utilized for training learnable parameter <inline-formula id="ieqn-65"><mml:math id="mml-ieqn-65"><mml:mi>&#x03C6;</mml:mi></mml:math></inline-formula>, vector <italic>p</italic> is computed through aspect and sentence in the proposed model.
<disp-formula id="eqn-23"><label>(23)</label><mml:math id="mml-eqn-23" display="block"><mml:mi>L</mml:mi><mml:mo>=</mml:mo><mml:mo>&#x2212;</mml:mo><mml:msubsup><mml:mrow><mml:mo>&#x2211;</mml:mo></mml:mrow><mml:mrow><mml:mi>m</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>m</mml:mi></mml:mrow></mml:msubsup><mml:msub><mml:mi>A</mml:mi><mml:mrow><mml:mi>m</mml:mi></mml:mrow></mml:msub><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mrow><mml:mi>m</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>Simultaneously, Ground truth denoted as <italic>z</italic> is considered as a vector, thus loss function <italic>n</italic> is minimized through the below equation.
<disp-formula id="eqn-24"><label>(24)</label><mml:math id="mml-eqn-24" display="block"><mml:mi>n</mml:mi><mml:mo>=</mml:mo><mml:mi>L</mml:mi><mml:mo>+</mml:mo><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mi>T</mml:mi><mml:mo>+</mml:mo><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msub><mml:msubsup><mml:mrow><mml:mrow><mml:mo>[</mml:mo><mml:mi>&#x03C6;</mml:mi><mml:mo>]</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup></mml:math></disp-formula></p>
<p>In above equation <inline-formula id="ieqn-66"><mml:math id="mml-ieqn-66"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula> and <inline-formula id="ieqn-67"><mml:math id="mml-ieqn-67"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula> are hyper parameters, these are adjusted for regularization terms.</p>
</sec>
</sec>
<sec id="s4"><label>4</label><title>Performance Evaluation</title>
<p>Sentiment analysis, also known as opinion mining is one of the fundamental tasks in natural language processing; it offers to infer opinions from given text. This research work develops a proposed methodology for Aspect Based Sentiment Analysis. In this section of the research, we evaluate the proposed methodology considering different parameters; evaluation is carried out considering deep learning library PyTorch. The proposed model is designed using python as a programming language and spyder as IDE on the Windows 10 platform packed with 8GB of RAM and 2GB of NVIDIA graphics. DSCNet is evaluated considering precision, recall, accuracy and macro-f1 (these metrics have been discussed later in same section), also comparative analysis is carried out on accuracy and Macro-F1 with other model.</p>
<sec id="s4_1"><label>4.1</label><title>Dataset Details</title>
<p>To prove the model efficeincy, three distinctive datasets from SemEVAL 2014 task 4 [<xref ref-type="bibr" rid="ref-22">22</xref>], the dataset comprises Twitter [<xref ref-type="bibr" rid="ref-23">23</xref>], Restaurant and Laptop dataset; dataset statistics are presented in <xref ref-type="table" rid="table-1">Tab. 1</xref>. Each dataset is split into training and dataset into three categories i.e., positive, negative, and neutral. In total there are 2966 reviews of the laptop dataset, 4728 reviews for the Restaurant dataset, and 6728 reviews for the Twitter dataset.</p>
<table-wrap id="table-1"><label>Table 1</label><caption><title>Dataset statistics</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left" rowspan="2"/>
<th align="center" colspan="2">Restaurant</th>
<th align="center" colspan="2">Laptop</th>
<th align="center" colspan="2">Twitter</th>
</tr>
<tr>
<th align="left">Train</th>
<th align="left">Test</th>
<th align="left">Train</th>
<th align="left">Test</th>
<th align="left">Train</th>
<th align="left">Test</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Positive</td>
<td align="left">2164</td>
<td align="left">727</td>
<td align="left">976</td>
<td align="left">337</td>
<td align="left">1507</td>
<td align="left">172</td>
</tr>
<tr>
<td align="left">Negative</td>
<td align="left">807</td>
<td align="left">196</td>
<td align="left">851</td>
<td align="left">128</td>
<td align="left">1528</td>
<td align="left">169</td>
</tr>
<tr>
<td align="left">Neutral</td>
<td align="left">637</td>
<td align="left">196</td>
<td align="left">455</td>
<td align="left">167</td>
<td align="left">3016</td>
<td align="left">336</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="s4_2"><label>4.2</label><title>Metrics Evaluation</title>
<p>Precision is measure of correct identification of sentiment polarity out of all actual sentiment polarity, it is computed as ratio of true positive to the sum of false positive and true positive.
<disp-formula id="eqn-25"><label>(25)</label><mml:math id="mml-eqn-25" display="block"><mml:mi>p</mml:mi><mml:mi>r</mml:mi><mml:mi>e</mml:mi><mml:mi>c</mml:mi><mml:mo>=</mml:mo><mml:mstyle displaystyle="true" scriptlevel="0"><mml:mfrac><mml:mrow><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mi>u</mml:mi><mml:mi>e</mml:mi><mml:mi mathvariant="normal">&#x005F;</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mi>u</mml:mi><mml:mi>e</mml:mi><mml:mi mathvariant="normal">&#x005F;</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>s</mml:mi><mml:mo>+</mml:mo><mml:mrow><mml:mtext mathvariant="italic">false</mml:mtext></mml:mrow><mml:mi mathvariant="normal">&#x005F;</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>s</mml:mi></mml:mrow></mml:mfrac></mml:mstyle></mml:math></disp-formula></p>
<p>Recall is measure of all correct identification of sentiments; it can be computed as the correct identification of true positives. Mathematically, it is the ratio of True positive to the sum of false negative and true positive.<disp-formula id="eqn-26"><label>(26)</label><mml:math id="mml-eqn-26" display="block"><mml:mrow><mml:mtext mathvariant="italic">recall</mml:mtext></mml:mrow><mml:mo>=</mml:mo><mml:mstyle displaystyle="true" scriptlevel="0"><mml:mfrac><mml:mrow><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mi>u</mml:mi><mml:mi>e</mml:mi><mml:mi mathvariant="normal">&#x005F;</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mi>t</mml:mi><mml:mi>r</mml:mi><mml:mi>u</mml:mi><mml:mi>e</mml:mi><mml:mi mathvariant="normal">&#x005F;</mml:mi><mml:mi>p</mml:mi><mml:mi>o</mml:mi><mml:mi>s</mml:mi><mml:mo>+</mml:mo><mml:mrow><mml:mtext mathvariant="italic">false</mml:mtext></mml:mrow><mml:mi mathvariant="normal">&#x005F;</mml:mi><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mi>g</mml:mi></mml:mrow></mml:mfrac></mml:mstyle></mml:math></disp-formula></p>
<p><xref ref-type="table" rid="table-2">Tab. 2</xref> shows the precision and recall value of all the three dataset through DSCNet mechanism. In case of Restaurant dataset 90.90&#x0025; of precision was observed and 88.70&#x0025; of recall value; for Laptop dataset, 78.13&#x0025; of precision and 79.09&#x0025; of recall was observed. Considering twitter dataset, 79.11&#x0025; and 76.20&#x0025; of precision and recall is achieved.</p>
<table-wrap id="table-2"><label>Table 2</label><caption><title>Precision and recall evaluation</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Dataset</th>
<th colspan="2" align="center">Metrics</th>
</tr>
<tr>
<th/>
<th align="left">Precision</th>
<th align="left">Recall</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Restaurant</td>
<td align="left">90.90</td>
<td align="left">88.70</td>
</tr>
<tr>
<td align="left">Laptop</td>
<td align="left">78.13</td>
<td align="left">79.09</td>
</tr>
<tr>
<td align="left">Twitter</td>
<td align="left">79.11</td>
<td align="left">76.20</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="s4_3"><label>4.3</label><title>Comparison Method</title>
<p>To prove the model efficiency, the different existing technique is considered. Most of the state-of-art technique is categorized in two categories i.e., with syntactic knowledge and without syntactic knowledge.</p>
<sec id="s4_3_1"><label>4.3.1</label><title>Syntax Free Approach</title>
<p><bold>SVM</bold> [<xref ref-type="bibr" rid="ref-24">24</xref>]: It uses a support vector machine over large features for classification.</p>
<p><bold>IAN</bold> [<xref ref-type="bibr" rid="ref-25">25</xref>]: It represents the context and task interactively through an attention mechanism and two LSTMS.</p>
<p><bold>TNet</bold> [<xref ref-type="bibr" rid="ref-26">26</xref>]: It transforms BiLSTM to the target-specific embeddings and utilizes Convolutional neural network for encoding.</p>
<p><bold>MGAN</bold> [<xref ref-type="bibr" rid="ref-27">27</xref>]: This mechanism utilizes BiLSTM for using multi-grained attention and contextual information to exploit the relationship among context and aspects.</p>
<p><bold>AOA</bold> [<xref ref-type="bibr" rid="ref-28">28</xref>]: The attention over attention model is introduced for interaction among context and aspects combined.</p>
<p><bold>AEN</bold> [<xref ref-type="bibr" rid="ref-29">29</xref>]: This mechanism adopts the encoder-based attention network for feature modeling and representation to achieve the semantic relations among target and context.</p>
<p><bold>CapsNet</bold> [<xref ref-type="bibr" rid="ref-30">30</xref>]: It uses a capsule network for modeling the complicated relationship between the context and target.</p>
<p><bold>BERT-PT</bold> [<xref ref-type="bibr" rid="ref-31">31</xref>]: It uses a pre-trained BERT model on post-training for improvisation in target aspect sentiment and reading comprehension.</p>
<p><bold>BERT-SPC</bold> [<xref ref-type="bibr" rid="ref-29">29</xref>]: It gives &#x201C;[SEP]&#x201D; &#x002B; sentence &#x002B; &#x201C;[CLS]&#x201D; &#x002B; target mention into the pre-trained model and further uses pooling-based embedding for classification.</p>
<p><bold>CapsNet-BERT</bold> [<xref ref-type="bibr" rid="ref-30">30</xref>]: It utilizes the BERT model and on top of that capsule network is used for sentiment polarities.</p>
</sec>
<sec id="s4_3_2"><label>4.3.2</label><title>Syntax Based Approach</title>
<p><bold>AdaRNN</bold> [<xref ref-type="bibr" rid="ref-32">32</xref>]: This mechanism learns the sentence representation towards the target through a composition of RNN based semantics over the designed dependency tree.</p>
<p><bold>PhraseRNN</bold> [<xref ref-type="bibr" rid="ref-33">33</xref>]: It is an extended version of AdaRNN and improvisation is carried out through integrating two composition functions. This method takes the constituent tree and dependency tree as input.</p>
<p><bold>Synattn</bold> [<xref ref-type="bibr" rid="ref-34">34</xref>]: This method integrates syntactic distance to the attention mechanism for modeling the interaction among context and target mention.</p>
<p><bold>CDT</bold> [<xref ref-type="bibr" rid="ref-35">35</xref>] and <bold>ASGCN</bold> [<xref ref-type="bibr" rid="ref-36">36</xref>]: Both mechanisms incorporate dependency with GCN for aspect representation learning; ASGCN applies the attention mechanism for obtaining the final representation.</p>
<p><bold>TD-GAT</bold> [<xref ref-type="bibr" rid="ref-37">37</xref>] and <bold>TD-GAT-BERT</bold> [<xref ref-type="bibr" rid="ref-22">22</xref>] utilize GAT to exploit the syntax structure and improvise with LSTM.</p>
<p>All these methods are compared in existing model RGAT-BERT [<xref ref-type="bibr" rid="ref-38">38</xref>], thus considering RGAT-BERT model, this paper performs comparison on accuracy and macro-f1 in next subsection.</p>
</sec>
<sec id="s4_3_3"><label>4.3.3</label><title>Accuracy</title>
<p>In deep learning, accuracy is major metrics available for classification, it represents the correctly classified model. In general, it is computed through finding ratio of samples correctly identified with total number of samples. <xref ref-type="table" rid="table-3">Tab. 3</xref> shows the accuracy comparison of different methodologies on restaurant datasets, to achieve more accuracy BERT model has been utilized extensively such as BERT-PT, BERT-SPC, CapsNet-BERT, AEN-BERT, TD-GAT-BERT, out of these methods, BERT-PT achieves the highest accuracy of 84.95. Moreover, the Existing model also adopts BERT based model and achieves an accuracy of 86.68. However, in comparison with these techniques, DSCNet achieves an accuracy of 92.59.</p>
<table-wrap id="table-3"><label>Table 3</label><caption><title>Accuracy comparison on Restaurant</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Methodologies</th>
<th align="left">Accuracy</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">SVM</td>
<td align="left">80.16</td>
</tr>
<tr>
<td align="left">IAN</td>
<td align="left">78.60</td>
</tr>
<tr>
<td align="left">Tnet</td>
<td align="left">80.69</td>
</tr>
<tr>
<td align="left">MGAN</td>
<td align="left">81.25</td>
</tr>
<tr>
<td align="left">AOA</td>
<td align="left">81.20</td>
</tr>
<tr>
<td align="left">AEN</td>
<td align="left">80.98</td>
</tr>
<tr>
<td align="left">CapsNet</td>
<td align="left">80.79</td>
</tr>
<tr>
<td align="left">BERT-PT</td>
<td align="left">84.95</td>
</tr>
<tr>
<td align="left">BERT-SPC</td>
<td align="left">84.46</td>
</tr>
<tr>
<td align="left">AEN-BERT</td>
<td align="left">83.12</td>
</tr>
<tr>
<td align="left">CapsNET-BERT</td>
<td align="left">85.93</td>
</tr>
<tr>
<td align="left">PhraseRNN</td>
<td align="left">66.20</td>
</tr>
<tr>
<td align="left">synAttn</td>
<td align="left">80.45</td>
</tr>
<tr>
<td align="left">ASGCN</td>
<td align="left">80.77</td>
</tr>
<tr>
<td align="left">CDT</td>
<td align="left">82.30</td>
</tr>
<tr>
<td align="left">TD-GAT</td>
<td align="left">81.20</td>
</tr>
<tr>
<td align="left">TD-GAT-BERT</td>
<td align="left">83.00</td>
</tr>
<tr>
<td align="left">Transformer</td>
<td align="left">80.78</td>
</tr>
<tr>
<td align="left">RGAT</td>
<td align="left">83.55</td>
</tr>
<tr>
<td align="left">RGAT-BERT (existing) [<xref ref-type="bibr" rid="ref-38">38</xref>]</td>
<td align="left">86.68</td>
</tr>
<tr>
<td align="left"><bold>PS-DSCNet</bold></td>
<td align="left"><bold>92.59</bold></td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Similarly, on laptop datasets shown in <xref ref-type="table" rid="table-4">Tab. 4</xref> initial BERT model like BERT-PT and BERT-SPC achieves an accuracy of 78.07 and 78.99 respectively; AEN-BERT achieves an accuracy of 79.93. Other BERT model like TD-GAT-BERT achieves 80.10 and the recent existing BERT model achieves 80.94, in comparison with all these models, DSCNet achieves an accuracy of 86.99.</p>
<table-wrap id="table-4"><label>Table 4</label><caption><title>Accuracy comparison on laptop dataset</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Methodologies</th>
<th align="left">Accuracy</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">SVM</td>
<td align="left">70.49</td>
</tr>
<tr>
<td align="left">IAN</td>
<td align="left">72.10</td>
</tr>
<tr>
<td align="left">Tnet</td>
<td align="left">76.54</td>
</tr>
<tr>
<td align="left">MGAN</td>
<td align="left">75.39</td>
</tr>
<tr>
<td align="left">AOA</td>
<td align="left">74.5</td>
</tr>
<tr>
<td align="left">AEN</td>
<td align="left">73.51</td>
</tr>
<tr>
<td align="left">BERT-PT</td>
<td align="left">78.07</td>
</tr>
<tr>
<td align="left">BERT-SPC</td>
<td align="left">78.99</td>
</tr>
<tr>
<td align="left">AEN-BERT</td>
<td align="left">79.93</td>
</tr>
<tr>
<td align="left">synAttn</td>
<td align="left">72.57</td>
</tr>
<tr>
<td align="left">ASGCN</td>
<td align="left">75.55</td>
</tr>
<tr>
<td align="left">CDT</td>
<td align="left">77.19</td>
</tr>
<tr>
<td align="left">TD-GAT</td>
<td align="left">74.00</td>
</tr>
<tr>
<td align="left">TD-GAT-BERT</td>
<td align="left">80.10</td>
</tr>
<tr>
<td align="left">Transformer</td>
<td align="left">74.09</td>
</tr>
<tr>
<td align="left">RGAT</td>
<td align="left">78.02</td>
</tr>
<tr>
<td align="left">RGAT-BERT (existing)</td>
<td align="left">80.94</td>
</tr>
<tr>
<td align="left"><bold>PS-DSCNet</bold></td>
<td align="left"><bold>86.99</bold></td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Further, considering the Twitter dataset comparison in <xref ref-type="table" rid="table-5">Tab. 5</xref>, BERT-based model BERT-SPC and AEN-BERT achieve 73.55 and 74.71 respectively; the existing BERT-based model achieves an accuracy of 76.28. In comparison with these methods, DSCNet achieves 78.76&#x0025; of accuracy.</p>
<table-wrap id="table-5"><label>Table 5</label><caption><title>Accuracy comparison on Twitter</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Methodologies</th>
<th align="left">Accuracy</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">SVM</td>
<td align="left">63.40</td>
</tr>
<tr>
<td align="left">Tnet</td>
<td align="left">74.97</td>
</tr>
<tr>
<td align="left">MGAN</td>
<td align="left">72.54</td>
</tr>
<tr>
<td align="left">AEN</td>
<td align="left">72.83</td>
</tr>
<tr>
<td align="left">BERT-SPC</td>
<td align="left">73.55</td>
</tr>
<tr>
<td align="left">AEN-BERT</td>
<td align="left">74.71</td>
</tr>
<tr>
<td align="left">ADARNN</td>
<td align="left">66.30</td>
</tr>
<tr>
<td align="left">ASGCN</td>
<td align="left">72.15</td>
</tr>
<tr>
<td align="left">CDT</td>
<td align="left">74.66</td>
</tr>
<tr>
<td align="left">Transformer</td>
<td align="left">72.78</td>
</tr>
<tr>
<td align="left">RGAT</td>
<td align="left">75.36</td>
</tr>
<tr>
<td align="left">RGAT-BERT (existing)</td>
<td align="left">76.28</td>
</tr>
<tr>
<td align="left"><bold>DSCNet-PS</bold></td>
<td align="left"><bold>78.76</bold></td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
<sec id="s4_3_4"><label>4.3.4</label><title>Macro-F1</title>
<p>In classification, the F1 score is a measurement of the test&#x2019;s accuracy; further F1 measure is computed using precision and recall. In general, we can say that the F1 is harmony mean of recall and precision. <xref ref-type="table" rid="table-6">Tabs. 6</xref>, <xref ref-type="table" rid="table-7">7</xref>, and <xref ref-type="table" rid="table-8">8</xref> shows the Macro-F1 comparison of a restaurant, laptop, and Twitter dataset respectively. Several traditional approaches did not consider macro-F1, however, the BERT model has been considering it as one of the important parameters. considering the restaurant dataset, BERT-PT achieves Macro-F1 of 76.96 and BERT-SPC achieves a Macro-F1 score of 76.98. Furthermore, the existing BERT model i.e., RGAT-BERT achieves a Macro-F1score of 80.92 whereas DSCNet achieves 89.68 of Macro-F1.</p>
<table-wrap id="table-6"><label>Table 6</label><caption><title>Macro-F1 comparison on restaurant dataset</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Methodologies</th>
<th align="left">Macro-F1</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Tnet</td>
<td align="left">71.27</td>
</tr>
<tr>
<td align="left">MGAN</td>
<td align="left">71.94</td>
</tr>
<tr>
<td align="left">AEN</td>
<td align="left">72.14</td>
</tr>
<tr>
<td align="left">BERT-PT</td>
<td align="left">76.96</td>
</tr>
<tr>
<td align="left">BERT-SPC</td>
<td align="left">76.98</td>
</tr>
<tr>
<td align="left">AEN-BERT</td>
<td align="left">73.76</td>
</tr>
<tr>
<td align="left">PhraseRNN</td>
<td align="left">59.32</td>
</tr>
<tr>
<td align="left">synAttn</td>
<td align="left">71.26</td>
</tr>
<tr>
<td align="left">ASGCN</td>
<td align="left">72.02</td>
</tr>
<tr>
<td align="left">CDT</td>
<td align="left">74.02</td>
</tr>
<tr>
<td align="left">Transformer</td>
<td align="left">72.10</td>
</tr>
<tr>
<td align="left">RGAT</td>
<td align="left">75.99</td>
</tr>
<tr>
<td align="left">RGAT-BERT (existing)</td>
<td align="left">80.92</td>
</tr>
<tr>
<td align="left"><bold>PS-DSCNet</bold></td>
<td align="left"><bold>89.68</bold></td>
</tr>
</tbody>
</table>
</table-wrap>
<table-wrap id="table-7"><label>Table 7</label><caption><title>Macro-F1 on laptop dataset</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Methodologies</th>
<th align="left">Macro-F1</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Tnet</td>
<td align="left">71.75</td>
</tr>
<tr>
<td align="left">MGAN</td>
<td align="left">72.47</td>
</tr>
<tr>
<td align="left">AEN</td>
<td align="left">69.04</td>
</tr>
<tr>
<td align="left">BERT-PT</td>
<td align="left">75.08</td>
</tr>
<tr>
<td align="left">BERT-SPC</td>
<td align="left">75.03</td>
</tr>
<tr>
<td align="left">AEN-BERT</td>
<td align="left">76.31</td>
</tr>
<tr>
<td align="left">synAttn</td>
<td align="left">69.13</td>
</tr>
<tr>
<td align="left">ASGCN</td>
<td align="left">71.05</td>
</tr>
<tr>
<td align="left">CDT</td>
<td align="left">72.99</td>
</tr>
<tr>
<td align="left">Transformer</td>
<td align="left">69.42</td>
</tr>
<tr>
<td align="left">RGAT</td>
<td align="left">74.00</td>
</tr>
<tr>
<td align="left">RGAT-BERT (existing)</td>
<td align="left">78.20</td>
</tr>
<tr>
<td align="left"><bold>PS-DSCNet</bold></td>
<td align="left"><bold>84.04</bold></td>
</tr>
</tbody>
</table>
</table-wrap>
<table-wrap id="table-8"><label>Table 8</label><caption><title>Macro-F1 comparison on a Twitter dataset</title></caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Methodologies</th>
<th align="left">Macro-F1</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">SVM</td>
<td align="left">63.30</td>
</tr>
<tr>
<td align="left">Tnet</td>
<td align="left">73.60</td>
</tr>
<tr>
<td align="left">MGAN</td>
<td align="left">70.81</td>
</tr>
<tr>
<td align="left">AEN</td>
<td align="left">69.81</td>
</tr>
<tr>
<td align="left">BERT-SPC</td>
<td align="left">72.14</td>
</tr>
<tr>
<td align="left">AEN-BERT</td>
<td align="left">73.13</td>
</tr>
<tr>
<td align="left">ADARNN</td>
<td align="left">65.90</td>
</tr>
<tr>
<td align="left">ASGCN</td>
<td align="left">70.40</td>
</tr>
<tr>
<td align="left">CDT</td>
<td align="left">73.66</td>
</tr>
<tr>
<td align="left">Transformer</td>
<td align="left">70.23</td>
</tr>
<tr>
<td align="left">RGAT</td>
<td align="left">74.15</td>
</tr>
<tr>
<td align="left">RGAT-BERT (existing)</td>
<td align="left">75.25</td>
</tr>
<tr>
<td align="left"><bold>PS-DSCNet</bold></td>
<td align="left"><bold>77.42</bold></td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Similarly considering the laptop dataset, the existing BERT model of BERT-PT, BERT-SPC, and AEN-BERT achieve a Macro-F1score of 75.08, 75.03, and 76.31 respectively. Furthermore, the recent existing model i.e., RGAT-BERT model achieves 80.92&#x0025; and the proposed DSCNet achieves 84.04 of Macro-F1.</p>
<p><xref ref-type="table" rid="table-7">Tab. 7</xref> shows the comparison of Macro-F1on a Twitter dataset, existing BERT models like BERT-SPC and AEN-BERT achieve a Macro-F1score of 72.14 and 73.13 respectively. Improvised BERT model i.e., RGAT-BERT model achieves a Macro-F1score of 77.42</p>
</sec>
</sec>
<sec id="s4_4"><label>4.4</label><title>Comparative Analysis and Discussion</title>
<p>Different deep learning architecture has been adopted to enhance the sentiment analysis, this research work tends to exploit the features in deep to understand different contexts of aspect, this section discuss discusses improvisation over the existing model in terms of accuracy and Macro-F1. Considering the laptop dataset, DSCNet improves accuracy by 6.05&#x0025; and Macro-F1by 5.84&#x0025;; similarly considering the restaurant dataset DSCNet improves accuracy by 5.91&#x0025; and a Macro-F1score of 8.76&#x0025;. At last, considering the Twitter dataset DSCNet improves accuracy by 2.48&#x0025; and Macro-F1 by 2.17. Further, <xref ref-type="fig" rid="fig-6">Figs. 6</xref> and <xref ref-type="fig" rid="fig-7">7</xref> presents the graphical comparison on all the three dataset (Laptop, restaurant and Twitter) accuracy and Macro-F1score.</p>
<fig id="fig-6"><label>Figure 6</label><caption><title>Accuracy comparison on three different dataset</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_30262-fig-6.png"/></fig>
<fig id="fig-7"><label>Figure 7</label><caption><title>Macro-F1 score comparison on the three dataset</title></caption><graphic mimetype="image" mime-subtype="png" xlink:href="CMC_30262-fig-7.png"/></fig>
<p>Moreover, considering the deep learning architecture, each metrics plays important role, through analysis it is observed that most of the comparing model ignored precision and recall value and this research did not find suitable mechanism to compare with respect to these two metrics. comparative analysis, it is observed that all these mechanisms did not consider all three datasets, most of the method carried out only on laptop and restaurant; few of them considered Twitter dataset but omitted laptop and restaurant. Furthermore, in past BERT model has been proven to be adopted widely for sentiment analysis; also, existing BERT model like RGAT-BERT has not only considered all three datasets but also achieved better results than other BERT model. However, as discussed earlier BERT model leaves research gap, also most of successful model adopted LSTM and attention based approach, this provided us flexibility to utilize Bi-LSTM in optimal manner. Thus, incorporating with the developed architecture i.e., DSCNet, proposed model achieves better metrics.</p>
</sec>
</sec>
<sec id="s5"><label>5</label><title>Conclusion</title>
<p>Aspect-SA (aspect based sentiment analysis) is an indispensable task in sentiment analysis and subtask in natural language processing. with the development of NLP, research on sentiment analysis has been processed mainly on deep learning from past few years as deep learning architecture provides deep feature extraction, however considering the vulnerabilities of sentiment polarity, existing deep learning architecture fails to understand the context of an aspect which could be solved through deep context knowledge. This research work introduces DSCNet (deep semantic and deep contextual knowledge) architecture, DSCNet architecture incorporates Semantic information and contextual knowledge together in deep to enhance the aspect extraction and understanding. DSCNet have three modules i.e., deep extraction, updation and deep integration which aims for optimal aspect based feautures. DSCNet is evaluated on three customer reviews dataset, two reviews dataset of laptop and restaurant from semeval 2014 task B and ascl Twitter dataset. DSCNet model efficiency is proven through computing deep learning metrics like precision, recall, accuracy and Macro-F1. Furthermore, comparative analysis is carried out with syntax based and syntax free technique along with recent deep learning model and our model is proven to achieve better metrics than these model. Although DSCNet achieves marginal improvisation over the other deep learning architecture, several research areas need to be looked into such as considering dynamic dataset which includes sarcastic comments.</p>
</sec>
</body>
<back>
<fn-group>
<fn fn-type="other"><p><bold>Funding Statement:</bold> This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (NRF-2022R1A2C2012243).</p></fn>
<fn fn-type="conflict"><p><bold>Conflicts of Interest:</bold> The authors declare that they have no conflicts of interest to report regarding the present study.</p></fn>
</fn-group>
<ref-list content-type="authoryear">
<title>References</title>
<ref id="ref-1"><label>[1]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Liu</surname></string-name></person-group>, &#x201C;<article-title>Sentiment analysis and opinion mining</article-title>,&#x201D; <source>Synthesis Lectures on Human Language Technologies</source>, vol. <volume>5</volume>, no. <issue>1</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>167</lpage>, <year>2012</year>.</mixed-citation></ref>
<ref id="ref-2"><label>[2]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name>E. Cambria</string-name></person-group>, &#x201C;<article-title>Affective computing and sentiment analysis</article-title>,&#x201D; <source> IEEE Intelligent System</source>, vol. <volume>31</volume>, no. <issue>2</issue>, pp. <fpage>102</fpage>&#x2013;<lpage>107</lpage>, <year>2016</year>.</mixed-citation></ref>
<ref id="ref-3"><label>[3]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J.</given-names> <surname>Zhou</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Jin</surname></string-name> and <string-name><given-names>X.</given-names> <surname>Huang</surname></string-name></person-group>, &#x201C;<article-title>ADeCNN: An improved model for aspect-level sentiment analysis based on deformable CNN and attention</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>8</volume>, pp. <fpage>132970</fpage>&#x2013;<lpage>132979</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-4"><label>[4]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Poria</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Hazarika</surname></string-name>, <string-name><given-names>N.</given-names> <surname>Majumder</surname></string-name> and <string-name><given-names>R.</given-names> <surname>Mihalcea</surname></string-name></person-group>, &#x201C;<article-title>Beneath the tip of the iceberg: Current challenges and new directions in sentiment analysis research</article-title>,&#x201D; <source>IEEE Transactions on Affective Computing</source>, pp. 1--1, 2020. <uri xlink:href="https://doi.org/10.1109/TAFFC.2020.3038167">https://doi.org/10.1109/TAFFC.2020.3038167</uri>.</mixed-citation></ref>
<ref id="ref-5"><label>[5]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>P.</given-names> <surname>Xiang</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>F.</given-names> <surname>Wu</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Cheng</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Zhou</surname></string-name></person-group>, &#x201C;<article-title>Single-image deraining with feature-supervised generative adversarial network</article-title>,&#x201D; <source>IEEE Signal Process. Letters</source>, vol. <volume>26</volume>, no. <issue>5</issue>, pp. <fpage>650</fpage>&#x2013;<lpage>654</lpage>, <year>May 2019</year>.</mixed-citation></ref>
<ref id="ref-6"><label>[6]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>T.</given-names> <surname>Mikolov</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Karafi&#x00E1;t</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Burget</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Cernock&#x00FD;</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Khudanpur</surname></string-name></person-group>, &#x201C;<article-title>Recurrent neural network based language model</article-title>,&#x201D; in <conf-name>Proc. of the 11th Annual Conf. of the Int. Speech Communication Association</conf-name>, <conf-loc>Makuhari, Japan</conf-loc>, pp. <fpage>1045</fpage>&#x2013;<lpage>1048</lpage>, <year>2010</year>.</mixed-citation></ref>
<ref id="ref-7"><label>[7]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>R.</given-names> <surname>Socher</surname></string-name>, <string-name><given-names>C. C. -Y.</given-names> <surname>Lin</surname></string-name>, <string-name><given-names>A. Y.</given-names> <surname>Ng</surname></string-name> and <string-name><given-names>C. D.</given-names> <surname>Manning</surname></string-name></person-group>, &#x201C;<article-title>Parsing natural scenes and natural language with recursive neural networks</article-title>,&#x201D; in <conf-name>Proc. of the 28th Int. Conf. on Machine Learning. Bellevue Washington</conf-name>, <conf-loc>USA</conf-loc>, pp. <fpage>129</fpage>&#x2013;<lpage>136</lpage>, <year>2011</year>.</mixed-citation></ref>
<ref id="ref-8"><label>[8]</label><mixed-citation publication-type="other"><person-group person-group-type="author"><string-name><given-names>J.</given-names> <surname>Weston</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Chopra</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Bordes</surname></string-name></person-group>, &#x201C;<article-title>Memory networks</article-title>,&#x201D; <year>2014</year>, arXiv:1410.3916. [Online]. Available: <uri xlink:href="http://arxiv.org/abs/1410.3916">http://arxiv.org/abs/1410.3916</uri>.</mixed-citation></ref>
<ref id="ref-9"><label>[9]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>H.</given-names> <surname>Liu</surname></string-name>, <string-name><given-names>I.</given-names> <surname>Chatterjee</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Zhou</surname></string-name>, <string-name><given-names>X. S.</given-names> <surname>Lu</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Abusorrah</surname></string-name></person-group>, &#x201C;<article-title>Aspect-based sentiment analysis: A survey of deep learning methods</article-title>,&#x201D; <source>IEEE Transactions on Computational Social Systems</source>, vol. <volume>7</volume>, no. <issue>6</issue>, pp. <fpage>1358</fpage>&#x2013;<lpage>1375</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-10"><label>[10]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>E.</given-names> <surname>Cambria</surname></string-name></person-group>, &#x201C;<article-title>Affective computing and sentiment analysis</article-title>,&#x201D; <source>IEEE Intelligent Systems</source>, vol. <volume>31</volume>, no. <issue>2</issue>, pp. <fpage>102</fpage>&#x2013;<lpage>107</lpage>, <year>2016</year>.</mixed-citation></ref>
<ref id="ref-11"><label>[11]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Xu</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Zhang</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Li</surname></string-name></person-group>, &#x201C;<article-title>STCS lexicon: Spectralclustering-based topic-specific Chinese sentiment lexicon construction for social networks</article-title>,&#x201D; <source>IEEE Transactions on Computational Social Systems</source>, vol. <volume>6</volume>, no. <issue>6</issue>, pp. <fpage>1180</fpage>&#x2013;<lpage>1189</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-12"><label>[12]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Kiritchenko</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Zhu</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Cherry</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Mohammad</surname></string-name></person-group>, &#x201C;<article-title>NRC-Canada2014: Detecting aspects and sentiment in customer reviews</article-title>,&#x201D; in <conf-name>Proc. of the 8th Int. Workshop on Semantic Evaluation (SemEval)</conf-name>, <conf-loc>Dublin, Ireland</conf-loc>, pp. <fpage>437</fpage>&#x2013;<lpage>442</lpage>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-13"><label>[13]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>L.</given-names> <surname>M&#x00E0;rquez</surname></string-name> and <string-name><given-names>H.</given-names> <surname>Rodr&#x00ED;guez</surname></string-name></person-group>, &#x201C;<article-title>Part-of-speech tagging using decision trees</article-title>,&#x201D; <source>European Conference on Machine Learning (ECML</source><italic>)</italic>, Chemnitz, Germany, vol. 1398, pp. <fpage>25</fpage>&#x2013;<lpage>36</lpage>, <year>1998</year>.</mixed-citation></ref>
<ref id="ref-14"><label>[14]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Y.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Jin</surname></string-name> and <string-name><given-names>Z. -H.</given-names> <surname>Zhou</surname></string-name></person-group>, &#x201C;<article-title>Understanding bag-of-words model: A statistical framework</article-title>,&#x201D; <source>International Journal of Machine Learning and Cybernetics</source>, vol. <volume>1</volume>, no. <issue>1&#x2013;4</issue>, pp. <fpage>43</fpage>&#x2013;<lpage>52</lpage>, <year>2010</year>.</mixed-citation></ref>
<ref id="ref-15"><label>[15]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>W.</given-names> <surname>Ali</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Yang</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Qiu</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Ke</surname></string-name> and <string-name><given-names>Y.</given-names> <surname>Wang</surname></string-name></person-group>, &#x201C;<article-title>Aspect-level sentiment analysis based on bidirectional-GRU in SIoT</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>9</volume>, pp. <fpage>69938</fpage>&#x2013;<lpage>69950</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-16"><label>[16]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>N.</given-names> <surname>Zhao</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Gao</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Wen</surname></string-name> and <string-name><given-names>H.</given-names> <surname>Li</surname></string-name></person-group>, &#x201C;<article-title>Combination of convolutional neural network and gated recurrent unit for aspect-based sentiment analysis</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>9</volume>, pp. <fpage>15561</fpage>&#x2013;<lpage>15569</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-17"><label>[17]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>X.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Xu</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Cai</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Tan</surname></string-name> and <string-name><given-names>C.</given-names> <surname>Zhu</surname></string-name></person-group>, &#x201C;<chapter-title>Detecting dependency-related sentiment features for aspect-level sentiment classification</chapter-title>,&#x201D; in <source>IEEE Transactions on Affective Computing</source>, pp. <fpage>1</fpage>&#x2013;<lpage>1</lpage>, <year>2021</year>. <uri xlink:href="https://doi.org/10.1109/TAFFC.2021.3063259">https://doi.org/10.1109/TAFFC.2021.3063259</uri>.</mixed-citation></ref>
<ref id="ref-18"><label>[18]</label><mixed-citation publication-type="other"><person-group person-group-type="author"><string-name><given-names>K.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>Q.</given-names> <surname>Liu</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Qian</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Xiang</surname></string-name>, <string-name><given-names>Q.</given-names> <surname>Cui</surname></string-name> <etal>et al.,</etal></person-group> &#x201C;<article-title>EATN: An efficient adaptive transfer network for aspect-level sentiment analysis</article-title>,&#x201D; <source>IEEE Transactions on Knowledge and Data Engineering</source>, pp. <fpage>1</fpage>&#x2013;<lpage>1</lpage>, <year>2021</year>. <uri xlink:href="https://doi.org/10.1109/TKDE.2021.3075238">https://doi.org/10.1109/TKDE.2021.3075238</uri>.</mixed-citation></ref>
<ref id="ref-19"><label>[19]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>L.</given-names> <surname>Canales</surname></string-name>, <string-name><given-names>W.</given-names> <surname>Daelemans</surname></string-name>, <string-name><given-names>E.</given-names> <surname>Boldrini</surname></string-name> and <string-name><given-names>P.</given-names> <surname>Mart&#x00ED;nez-Barco</surname></string-name></person-group>, &#x201C;<article-title>EmoLabel: Semi-automatic methodology for emotion annotation of social media text</article-title>,&#x201D; <source>IEEE Transactions on Affective Computing</source>, vol. <volume>13</volume>, no. <issue>2</issue>, pp. <fpage>579</fpage>&#x2013;<lpage>591</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-20"><label>[20]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Huang</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Xie</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Rao</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Liu</surname></string-name>, <string-name><given-names>L. K. M.</given-names> <surname>Poon</surname></string-name> <etal>et al.,</etal></person-group> &#x201C;<article-title>Lexicon-based sentiment convolutional neural networks for online review analysis</article-title>,&#x201D; <source>IEEE Transactions on Affective Computing</source>, pp. <fpage>1</fpage>&#x2013;<lpage>1</lpage>, <year>2020</year>. <uri xlink:href="https://doi.org/10.1109/TAFFC.2020.2997769">https://doi.org/10.1109/TAFFC.2020.2997769</uri>.</mixed-citation></ref>
<ref id="ref-21"><label>[21]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Chiu</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Shang</surname></string-name> and <string-name><given-names>L.</given-names> <surname>Shao</surname></string-name></person-group>, &#x201C;<article-title>Neural text segmentation and its application to sentiment analysis</article-title>,&#x201D; <source>IEEE Transactions on Knowledge and Data Engineering</source>, vol. <volume>34</volume>, no. <issue>2</issue>, pp. <fpage>828</fpage>&#x2013;<lpage>842</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-22"><label>[22]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Pontiki</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Galanis</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Pavlopoulos</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Papageorgiou</surname></string-name>, <string-name><given-names>I.</given-names> <surname>Androutsopoulos</surname></string-name> <etal>et al.,</etal></person-group> &#x201C;<article-title>Semeval-2014 task 4: Aspect based sentiment analysis</article-title>,&#x201D; in <conf-name>Proc. of the 8th Int. Workshop on Semantic Evaluation</conf-name>, <conf-loc>Dublin, Ireland</conf-loc>, pp. <fpage>27</fpage>&#x2013;<lpage>35</lpage>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-23"><label>[23]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>L.</given-names> <surname>Dong</surname></string-name>, <string-name><given-names>F.</given-names> <surname>Wei</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Tan</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Tang</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Zhou</surname></string-name> <etal>et al.,</etal></person-group> &#x201C;<article-title>Adaptive recursive neural network for target-dependent twitter sentiment classification</article-title>,&#x201D; in <conf-name>Proc. of the 52nd Annual Meeting of the Association for Computational Linguistics</conf-name>, <conf-loc>Baltimore, Maryland, USA</conf-loc>, pp. <fpage>49</fpage>&#x2013;<lpage>54</lpage>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-24"><label>[24]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>C.</given-names> <surname>Cortes</surname></string-name> and <string-name><given-names>V.</given-names> <surname>Vapnik</surname></string-name></person-group>, &#x201C;<article-title>Support-vector networks</article-title>,&#x201D; <source>Machine Learning</source>, vol. <volume>20</volume>, pp. <fpage>273</fpage>&#x2013;<lpage>297</lpage>, <year>1995</year>.</mixed-citation></ref>
<ref id="ref-25"><label>[25]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>D.</given-names> <surname>Ma</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Zhang</surname></string-name> and <string-name><given-names>H.</given-names> <surname>Wang</surname></string-name></person-group>, &#x201C;<article-title>Interactive attention networks for aspect-level sentiment classification</article-title>,&#x201D; in <conf-name>Proc. of the 26 Int. Joint Conf. on Artificial Intelligence (IJCAI)</conf-name>, <conf-loc>Melbourne, Australia</conf-loc>, pp. <fpage>4068</fpage>&#x2013;<lpage>4074</lpage>, <year>2017</year>.</mixed-citation></ref>
<ref id="ref-26"><label>[26]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>X.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Bing</surname></string-name>, <string-name><given-names>W.</given-names> <surname>Lam</surname></string-name> and <string-name><given-names>B.</given-names> <surname>Shi</surname></string-name></person-group>, &#x201C;<article-title>Transformation networks for targetoriented sentiment classification</article-title>,&#x201D; in <conf-name>Proc. of the 56th Annual Meeting of the Association for Computational Linguistics</conf-name>, <conf-loc>Melbourne, Australia</conf-loc>, pp. <fpage>946</fpage>&#x2013;9, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-27"><label>[27]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>F.</given-names> <surname>Fan</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Feng</surname></string-name> and <string-name><given-names>D.</given-names> <surname>Zhao</surname></string-name></person-group>, &#x201C;<article-title>Multi-grained attention network for aspectlevel sentiment classification</article-title>,&#x201D; in <conf-name>Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing (EMNLP)</conf-name>, <conf-loc>Brussels, Belgium</conf-loc>, pp. <fpage>3433</fpage>&#x2013;<lpage>3442</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-28"><label>[28]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Huang</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Ou</surname></string-name> and <string-name><given-names>K. M.</given-names> <surname>Carley</surname></string-name></person-group>, &#x201C;<article-title>Aspect level sentiment classification with attention-over-attention neural networks</article-title>,&#x201D; in <conf-name>Int. Conf. on Social Computing, Behavioral-Cultural Modeling, &#x0026; Prediction and Behavior Representation in Modeling and Simulation</conf-name>, <conf-loc>Washington DC, USA</conf-loc>, pp. <fpage>197</fpage>&#x2013;<lpage>206</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-29"><label>[29]</label><mixed-citation publication-type="other"><person-group person-group-type="author"><string-name><given-names>Y.</given-names> <surname>Song</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>T.</given-names> <surname>Jiang</surname></string-name>, <string-name><given-names>Z.</given-names> <surname>Liu</surname></string-name> and <string-name><given-names>Y.</given-names> <surname>Rao</surname></string-name></person-group>, &#x201C;<article-title>Attentional encoder network for targeted sentiment classification</article-title>,&#x201D; arXiv, vol.abs/1902.09314, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-30"><label>[30]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>Q.</given-names> <surname>Jiang</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Chen</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Xu</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Ao</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Yang</surname></string-name></person-group>, &#x201C;<article-title>A challenge dataset and effective models for aspect-based sentiment analysis</article-title>,&#x201D; in <conf-name>Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNLP)</conf-name>, <conf-loc>Hong Kong, China</conf-loc>, pp. <fpage>6280</fpage>&#x2013;<lpage>6285</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-31"><label>[31]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>H.</given-names> <surname>Xu</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Liu</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Shu</surname></string-name> and <string-name><given-names>P.</given-names> <surname>Yu</surname></string-name></person-group>, &#x201C;<article-title>BERT Post-training for review reading comprehension and aspect-based sentiment analysis</article-title>,&#x201D; in <conf-name>Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)</conf-name>, <conf-loc>Minneapolis, Minnesota</conf-loc>, pp. <fpage>2324</fpage>&#x2013;<lpage>2335</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-32"><label>[32]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>Y.</given-names> <surname>Du</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>W.</given-names> <surname>Feng</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Pan</surname></string-name>, <string-name><given-names>T.</given-names> <surname>Qin</surname></string-name> <etal>et al.,</etal></person-group> &#x201C;<article-title>AdaRNN: Adaptive learning and forecasting of time series</article-title>, &#x201D; in <conf-name>Proc. of the 30th ACM Int. Conf. on Information &#x0026; Knowledge Management (CIKM)</conf-name>, <conf-loc>Glod Coast, Australia</conf-loc>, pp. <fpage>402</fpage>&#x2013;<lpage>411</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-33"><label>[33]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>T. H.</given-names> <surname>Nguyen</surname></string-name> and <string-name><given-names>K.</given-names> <surname>Shirai</surname></string-name></person-group>, &#x201C;<article-title>PhraseRNN: Phrase recursive neural network for aspect-based sentiment analysis</article-title>,&#x201D; in <conf-name>Proc. of the 2015 Conf. on Empirical Methods in Natural Language Processing (EMNLP)</conf-name>, <conf-loc>Lisbon, Portugal</conf-loc>, pp. <fpage>2509</fpage>&#x2013;<lpage>2514</lpage>, <year>2015</year>.</mixed-citation></ref>
<ref id="ref-34"><label>[34]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>R.</given-names> <surname>He</surname></string-name>, <string-name><given-names>W. S.</given-names> <surname>Lee</surname></string-name>, <string-name><given-names>H. T.</given-names> <surname>Ng</surname></string-name> and <string-name><given-names>D.</given-names> <surname>Dahlmeier</surname></string-name></person-group>, &#x201C;<article-title>Effective attention modeling for aspect-level sentiment classification</article-title>,&#x201D; in <conf-name>Proc. of the 27th Int. Conf. on Computational Linguistics</conf-name>, <conf-loc>Santa Fe, New Mexico, USA</conf-loc>, pp. <fpage>1121</fpage>&#x2013;<lpage>1131</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-35"><label>[35]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>K.</given-names> <surname>Sun</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Mensah</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Mao</surname></string-name> and <string-name><given-names>X.</given-names> <surname>Liu</surname></string-name></person-group>, &#x201C;<article-title>Aspect-level sentiment analysis via convolution over dependency tree</article-title>,&#x201D; in <conf-name>Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNLP)</conf-name>, <conf-loc>Hong Kong, China</conf-loc>, pp. <fpage>5678</fpage>&#x2013;<lpage>5687</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-36"><label>[36]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>C.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>Q.</given-names> <surname>Li</surname></string-name> and <string-name><given-names>D.</given-names> <surname>Song</surname></string-name></person-group>, &#x201C;<article-title>Aspect-based sentiment classification with aspect-specific graph convolutional networks</article-title>,&#x201D; in <conf-name>Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNLP)</conf-name>, <conf-loc>Hong Kong, China</conf-loc>, pp. <fpage>4567</fpage>&#x2013;<lpage>4577</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-37"><label>[37]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Huang</surname></string-name> and <string-name><given-names>K.</given-names> <surname>Carley</surname></string-name></person-group>, &#x201C;<article-title>Syntax-aware aspect level sentiment classification with graph attention networks</article-title>,&#x201D; in <conf-name>Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNLP)</conf-name>, <conf-loc>Hong Kong, China</conf-loc>, pp. <fpage>5468</fpage>&#x2013;<lpage>5476</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-38"><label>[38]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>X.</given-names> <surname>Bai</surname></string-name>, <string-name><given-names>P.</given-names> <surname>Liu</surname></string-name> and <string-name><given-names>Y.</given-names> <surname>Zhang</surname></string-name></person-group>, &#x201C;<article-title>Investigating typed syntactic dependencies for targeted sentiment classification using graph attention neural network</article-title>,&#x201D; <source>IEEE/ACM Transactions on Audio, Speech, and Language Processing</source>, vol. <volume>29</volume>, pp. <fpage>503</fpage>&#x2013;<lpage>514</lpage>, <year>2021</year>.</mixed-citation></ref>
</ref-list>
</back>
</article>