This paper aims to build a graph-based representation method to improve the efficiency of quantity relation extraction for solving algebra story problems stated in Chinese. As discussed above, the algebra story problem is a sub-category of the math word problem which is much more complex than other math word problems such as arithmetic word problems [3]. Hence, in this section, we discuss the related work of solving math word problems, not just algebra story problems. The math word problem solvers can be divided into three categories: rule-based, semantic parsing-based and end-to-end method.

Rule-Based Method. Early solvers used rule-based methods to solve math word problems, which leverage predefined rules or templates to extract quantity relation from the problem text. A computer program, WORDPRO [4] was proposed in the 1980s. By manually defining rules, the program converts text in a particular format into propositions, and then makes simple inferences about these propositions to solve one-step problems. Bakman [24] proposed another system ROBUST which is an improvement on WORDPRO by defining a wider range of rules that enable it to solve free-format multi-step problems. Yuhui et al. [5] expanded this method to solve multi-step addition and subtraction algebra story problems by using a fixed frame. All these methods leverage fine defined rules to represent the input problems and can only work on small and closed data set of problems. Besides, the rules could be inefficient in a complicated context of problems.

Semantic Parsing-Based Method. With the development of natural language processing (NLP) technologies, researchers use semantic parsing to extract quantity relations from the problem text to avoid the huge work of manual rule construction. Roy et al. [6] made a first attempt to map text segments into quantity-value representation (QVR) by using text features and part of speech tags. The method of tracking the changes of quantity relation state with verbs was described in [7] which can be used to solve some numerical state change problems, but only in addition and subtraction. Another approach to semantic analysis described in [8] which is to extract relations by defining expressions named schema that match text. A tag-based algebra story problems solver was proposed in [9] which analyzes and transforms the text into predefined tag-based logic forms to achieve relation extraction. Mandal et al. [10] proposed an approach to extract the key entities and their attributes and quantities for solving addition-subtraction type arithmetic MWP. The problem of these semantic parsing methods is that they are too sensitive to interference information, and the performance of quantity relations extraction is significantly reduced when complex text is encountered.

End-to-End Method. Recent years, a number of researchers try to solve algebra story problems by generating a mathematical expression directly from the problem text. In [11], tree model is proposed to construct an expression tree. Then, the concept of unit dependency graph [11] is proposed to optimize the construction process of the tree. Inspired by this work, Wang et al. [12] attempted to use deep reinforcement learning to generate expression trees. Slightly different from expression trees, the concept of equation trees is proposed in [25]. Expression trees differ from equation trees in whether they contain unknowns in tree nodes, but are essentially equivalent.

Other researchers adopt the deep learning approach and obtain an expression from the problem text through training on the big data set without manual intervention, so as to calculate and solve the target. A method to align the quantities in the text with the template by pre-defining the equation template and using the deep learning is proposed in [26]. Wang et al. [15] made the first attempt to use a deep neural network to generate expressions for solving algebra story problems. In [27], a neural math solver based on an encoder-decoder framework was proposed to generate expressions. Although the number of researchers studying the end-to-end method is increasing, solving algebra story problems is not only about obtaining a result, because it is the retention of intermediate process that makes sense for the intelligent tutoring system, and this is the biggest problem of these methods that do not extract numeric relations.

The quantity relations and solution goals in algebra story problems are presented associated with math objects and recognizing the math object and identifying the relationship between the math objects is the fundamental work before quantity relation extraction. In the field of natural language processing (NLP), objects are usually named entities and a fair amount of work has been done on entity relation representation. This section introduces the general approaches of entity relation representation in NLP: semantic relation-based method, semantic framework-based method, and commonsense network-based method.

The semantic relation-based method classifies words according to the lexical meaning to obtain the relations between words, such as WordNet [28]. The WordNet is a database of English words developed by Princeton University in 1985. First, the words are divided according to the four parts of speech: noun, verb, adjective, and adverb, and then words are grouped according to the connections between different lexical meanings. WordNet defined four semantic relations: synonyms, hyponymy, antonymy, and meronymy. WordNet is an inductive lexicon not associated in meaning [29] but simultaneously present in the context.

The method based on semantic framework takes the semantic context of natural language into constructing lexical associations, such as FrameNet database [30]. which are organized by lexical meanings, the core of the FrameNet is to link words through semantic frameworks. The FrameNet consists of lexicon, annotated example sentences and the FrameNet database. The lexicon contains the definitions of word meaning, the annotated example sentences store the application examples of words, and the FrameNet database represents numerous patterns of the semantic framework [31]. The semantic framework is the core of FrameNet, which defines the framework’s name, meaning, and relation [29], and the lexical units. By defining a semantic framework, FrameNet can represent relationships when they exist simultaneously in a natural language context but where the entities are not lexically related. However, if the entities to be described do not have an associated semantic context frame defined in FrameNet, it is impossible to relate them.

The commonsense network-based method represents general commonsense information, such as ConceptNet [32]. ConceptNet is a dataset to track commonsense associations between entities. ConceptNet can construct associations between entities not directly related lexically or semantically. The combination and association of ConceptNet entities neglect the linguistic relevance of entities, making the relationships between entities dependent on commonsense. ConceptNet does not have the linguistic discipline for the existence of regularisations and formalization in algebra story problems. Therefore, ConceptNet is not adapted to modeling entity relations in algebra story problems.

As described in [23], a qualia role is proposed and designed to indicate a word’s meaning by a set of relations, called qualia, between the concept of the word and another concept that the word evokes. In [22], a more elaborated set of semantic roles (e.g., formal role, constitutive role, agentive role and telic role) is proposed to represent the meaning of nominals as well as the hidden information that is not overtly expressed in the syntax. The qualia role framework is described as follows: [α,QUALIA=[F,C,T,A]], where F=Formal denotes what kind of thing is it, what is its nature; C=Constitutive denotes what it make of, what are its constituents; T=Telic denotes what is it for, how does it function; A=Agentive denotes how did it come into being, what brought it about.

Later, Yuan [23] extended the semantic roles to 10 categories to form a more complete qualia structure of nouns, verbs and adjectives in Chinese.

The qualia role explores subjective evaluation and objective attribute of the math objects within an individual framework, which coincides with the expectation of the S2 method but is more powerful in understanding concept relations of linguistic expressions. In this paper, benefiting from this framework, the qualia role is chosen to modeling the quantity relations of math objects in sophisticated scenarios for solving algebra story problems stated in Chinese.

In an algebra story problem, quantities are mostly associated with the attributes that belong to math objects and the attributes are related to each other according to the interconnections of the math objects. Quantity relations are formed from the source formula of the attribute(s) of an individual math object or the attribute relation(s) between the interconnected math objects.

For example, in the case of “A 2000 lbs car with a speed of 50 km/h”, the quantities of “2000 lbs” and “50 km/h” are both associated with the entity “car”, but each refers a specific attribute (e.g., “2000 lbs” refers to the weight of the car, “50 km/h” refers to the speed of the car). To distinguish the quantities of various attributes is essential to prevent solvers from invalid operations that against the elemental principles of mathematics. For example, it is meaningless to plus “2000 lbs” and “50 km/h”. However, in most current MWP solvers, attributes of “weight”, “speed” and “car” are modeled as equal objects and it is hard to distinguish the entities and the associated attributes.

In this paper, the detected objects are divided into two categories: math entity and math attribute, from the perspective of semantic roles.

Math Entity (ME): Math entity is a noun object with math attribute(s).

Both MA and ME are terms generated from math objects, which are annotated as nouns but different for mathematical property presentation. Generally, MAs represent numeric values associated with the math attributes of math entities. While MEs are used to distinguish different math entities. As a result, the quantity relations can be a hierarchical structure of “ME−MA−Value”.

To note that not all the math attributes are presented explicitly in the problem text. For example, in statement of “a 2 m long wire”, the quantity “2 m long” describes the attribute length of the wire which is not given. In such cases, an unit parsing is applied to complete the missing MEs.

Another issue is that the boundary between math attributes and math entities is not static, i.e., there is a variable semantic role. For example, in the case of “the leg of the table is 5 inches height”, here “the leg” is an attribute of the entity “the table”, but in some time, it is also an entity with an attribute “height”. In such case, the algorithm needs to determine the specific role of the object according to the context. Besides, entities may be missed as the idiomatic usage in a language. For example, “A storybook consists of 50 pages. Ming reads ten pages on the first day and five pages more on the second day”. The problem does not explicitly state the mathematical entity underlying the ten and five more pages. However, it requires an algorithm to construct the implicit entity of “sub-page”, which forms the Constitute and Unit semantic roles with the storybook and the numerical value.

In the qualia role description system, entity relationships are modeled by the qualia roles of entities. Specifically, the materiality roles of entities are determined by the type and syntactic structure, and we use ten qualia roles [23] to describe the math entity relationships in Chinese math word problems. As a result, the math entity relationship can be described as a qualia structure denoted as QS and each element qs in QS is structured as a tuple of <esrc,edst,r,f,p>, where esrc and edst are two math entities, r denotes the semantic role of edst associated with esrc, f denotes the source formulas related to r, p denotes the syntax pattern related to r.

Formal (FOR): the taxonomic information about the math entity (the is-a relation).

Recalling the example of wire-square-circle problem above, the dependency between “wire” and “square” can be represented by a constitutive role (CON). The dependency between of “square” and “side” can be represented by an agentive role (AGE).

Entity dependency presents the relationship that exists between different entities. The relationship can be described as a graph called entity-dependency graph (EDG). We define the EDG as a tuple (N,A) consisting of nodes N and edges A. The nodes represent entities and the links represent the dependencies. Entity dependency in problem text exists in the following two situations. One is when entities are used to present the attributes of others. Take “weight of a tiger” for example, the entity “weight” is used to specify the attribute of another entity “tiger”, which means that it can not be treated as the principal subject in presenting numeric values. Similar examples such as the “length”, “width” and “area” of a rectangle, the “radius” or “diameter” of a circle. In this situation, entity dependency is implied by domain knowledge, and it can be specified either explicit or implicit. Another situation is when entities are used as syntactic modifiers. Examples such as “tigers in the zoo”, where the entities “zoo” and “tiger” constitute an adjunct relation. In this situation, numeric values are used to present the attributes of either a single entity (e.g., “tiger”) or the compound entity of “zoo−tiger”, which depends on the problem context. This section presents a detailed discussion of the types of entity dependency in algebra story problems.

As shown in Fig. 2, entity relationships can be classified into three fundamental types, each type refers to a sub-graph.

Attribute Relationship Sub-graph: describes the attributes of a math entity, such as the quantity, length, weight, speed, etc., which usually associates with qualia roles of FOR and UNI.

Coordinate Relationship Sub-graph: describes a concurrent relationship between two or more entities, e.g., a comparative relationship, a multiplicative/proportional relationship, etc., which usually associates with qualia roles of EVA, MAT and ORI.

Convergence Relationship Sub-graph: describes the convergent relationship between two or more math entities, e.g., the summation relation, which usually associates with qualia roles of CON, TEL, ACT, HAN, MAT, and ORI.

As a result, entity relationships could be organized as a network named entity dependency graph (EDG in short) which is composed by the above three relationship networks. In the field of math word problem solving, we can build a static EDG that contains domain knowledge and commonsense knowledge to store and represent the attribute relations of entities. Besides, EDG can be also used to represent the dynamic relations of entities given by the problem text. In the end, all the static relations and dynamic relations are integrated into a single EDG. Next, we give an introduction to how the quantity relations are represented and stored in an EDG.

According to the definition of the categories of entity relationships, quantity relations in algebra story problems could be divided into the following three categories:

Attribute Relation. Attribute relation usually presents a numeric value for an attribute (as shown in formula (1)) of a math entity. The numeric value is given or known by the input problem. Another frequently existed attribute relation is the quantity transformation between multiple attributes (as shown in formula (2)). For example, the relation between the radius and the circumference of a circle. These attribute relations are not directly presented by the input problem, but could be inferred out following certain knowledge. Therefore, in this case, a domain knowledge base is needed and a tiny inference engine should be designed to obtain attribute relations. (1)r1,i=c (2)r1,i=domain_formula(r1,j,r1,k,…)where r1,i,r1,j,r1,k,… denote the attributes of the math entity e1o as shown in Fig. 2a, c is a constant extracted from the input problem, domain_formula() is a source formula on attributes r1,i,r1,j,r1,k,…

Comparable Relation. The comparable relation describes a comparative or multiplicative relationship between two math entities with the same type of attribute, such as a relation of “more or less than”, “times”, etc. (3)r1,i=c⊕r2,i (4)c=r1,i⊙r2,iwhere ⊕ denotes operator + or ×, ⊙ is an operator from − or ÷. Comparable relations could be translated from the coordinate relationship sub-graph as shown in Fig. 2b.

Convergence Relation. The aggregate relation in a convergence relationship describes a total relation of the same type attribute between multiple math entities as expressed in formula (5), which usually refers to “total”, “sum”, etc. Convergence relations could be translated from the convergence relationship sub-graph as shown in Fig. 2c. (5)r0,i=r1,i+r2,i+⋯where i=1,…,n, n represents the entity number involved in the convergence operation. By using the above formulas, the entity relationship sub-graphs defined in Fig. 2 could be translated into quantity expressions for solving the input problem.

The above types of quantity relations exist everywhere in algebra story problems. The difference between each category is that the attribute relation is used to represent the quantities of the attributes within an attribute sub-graph Ga, which could be written as a constant expression or an expression with two or more attributes of the math entity in Ga. The comparable relation and the convergence relation are used to represent quantity relations of the same attribute between different math entities but are fit for different typologies of sub-graphs. For a coordinate sub-graph, expressions can be obtained from the source formulas of the comparable relations, and for a convergence sub-graph, expressions can be obtained from the source formulas of the convergence relations. Therefore, the task of quantity relation extraction can be divided into two sub-tasks of EDG construction and sub-graph identification.

To build the EDG for each input problem, we first create a node set N based on the result of part-of-speech annotation. Then, edges aij are added according to the syntactic dependency and the qualia role of the nodes. The complete process of EDG generation is described in Algorithm 1 which contains two main processes: node generation and edge generation.

Node Generation A graph EDG has two types of nodes: ME nodes and MA nodes, which correspond to math entities and math attributes separately. To generate the nodes, part-of-speech (POS) analyzing [33] is adopted to obtain all the math objects from the original text. Then, attribute checking is implemented to identify entity objects and attribute objects and ME nodes and MA nodes are created correspondingly. Implicit MA nodes as discussed in Section 3.1 are added before the edge generation.

Edge Generation An edge could to added to connect either two ME nodes, or between a ME node and a MA node if there exists a qualia relationship between the two nodes. Ten types of qualia relationships as described in Section 3.2 determine to add which edges.

Take the problem shown in Fig. 1 as an example, to generate the EDG nodes, nouns of “wire”, “square”, “side”, “circle” and “radius” annotated by a POS analyzer could perceive as the original math objects. Then, an attribute analyzing is implemented according to the qualia roles described by the knowledge base QRKB, to distinguish ME nodes “wire”, “square” and “circle” from MA nodes “side” and “radius”. In the end, a qualia relationship analysis determines when to add which edges. For example, when a constitutive relation is detected between the MA node “side” and the ME node “square”, an edge between nodes “side” and “square” is added. Similar operations can be done to add a new edge between the ME node “wire” and “square” because of an agentive relation.

As discussed in Section 3, entity relationships are categorized into three categories and quantity relations can be obtained by using formulas (1)–(5). Therefore, the task of quantity relation extraction is to identify the categories of the sub-graph in the generated EDG. The process of quantity relation extraction is described in Algorithm 2.

The core task of Algorithm 2 is to search out all the sub-graphs Gi in directly graph G, and let each Gi refers to one category of entity relationship as discussed in Section 3. To identify the category of a sub-graph composed by (pi,pj), we only need to test the node type of pj. If pj is an attribute node, then (pi,pj) belongs to an attribute sub-graph associated with entity node pi. Otherwise, (pi,pj) belongs to a concurrent sub-graph. Therefore, the algorithm first travels all the nodes to find out attribute relation sub-graphs. Then, the concurrency relation sub-graphs are searched out, denoted as G3. To identify the convergence sub-graph and the compound sub-graph, we implement a further analysis on G3. Specifically, if the sub-graph G3 only has two nodes, then it can be identified as a convergence sub-graph G2. Finally, the left sub-graphs in G3 are considered as convergence sub-graphs. The output relations are translated from the identified sub-graphs according to formulas (1)–(5).

For example in Fig. 1, when the sub-graph G1=(e2:square,attr:side,attr:length) is identified, an expression of “e2.length=4×e2.side” could be extracted from G1. Similarly, an area calculation expression “e2.area=e2.side×e2.side” could also be generated if keyword “area” is involved in the input problem.

Though some AWP solvers can directly use the generated expressions, most of the neural solvers cannot accept these expressions directly. To address the incompatible issue, the expressions need to be transformed into natural language statements to input to the solvers.

As the performance of quantity relation extraction is not evaluated by all the neural solvers, we only compare the result to the S2 method. The test uses accuracy (Acc), recall (R) and F1-score as metrics to evaluate the performance of quantity relation extraction. A big challenge is that there is a lack of large scale ground-truth for quantity relation evaluation. Therefore, we randomly choose 1421 problems from the new dataset and add the quantity relation labels for each problem. Comparison with S2 method, Table 2 summarizes the comparison of the answer accuracy on the test set with regard to problem types. The proposed EDG model significantly outperforms the S2 method nearly 7.7% in terms of the overall accuracy. More specifically, EDG outperforms the neural models by 15.7% and 10.6% on the Unitary and Motion problems.

Inspired by [34], a prompt learning mechanism is implemented to evaluate the performance of problem solving. In our experiment, the original problem text, as well as the output of Algorithm 2, were encoded together and were taken as input of the MWP solvers, e.g., GTS, Graph2Tree. The accuracy of problem solving is then evaluated as shown in Table 3.

From Table 3, we can observe that benefiting from the EDG tasks, the average accuracy is up to 71.9% and 68.4% by injecting the extracted quantity relations into Graph2Tree and GTS models separately. Significant improvements (5%, 3.3%) were achieved on questions of Unitary and Proportion after integrating EDG method into Grahp2Tree model, which shows our method is more feasible for solving WMPs in case of more implicit relations are needed.

For most MWP solvers, problem texts are treated as an unstructured word sequence. However, the relationships among the math objects involved in the text, such as math entities and attributes, are structured information with corresponding hidden domain knowledge. This structured information can be organized as a tree or graph [11,14,35] to improve the final performance. As shown in Fig. 3a, Roy et al. [11] used unit dependency graph (UDG) to represent the relationship between the numbers and the question being asked. Here, “66” and “10” are connected via a “SAME UNIT” edge, hence they can be added or subtracted, “8” is connected to the goal question node with an “DEN UNIT” edge, indicating that some expression will be divided by “8” to get the answer’s unit. To enrich the representation of a quantity, Zhang et al. [14] (as shown in Fig. 3b) leverage two graphs, including a quantity comparison graph and a quantity cell graph, to model the relationships between the descriptive words associated with a quantity. To associate the phrase structure information, the dependency parsing [35] (as shown in Fig. 3c) is implemented to link with two word nodes using syntactic constituency information to construct quantity graphs.

Compared with the above three types of graphs, the proposed entity-dependency graph (EDG) can be used to represent the relationship among the quantities and math entities in sophisticated scenarios. In each constructed EDG (as shown in Fig. 3d), math entities (e.g., “Isabel”, “flower”, “bouquets”) and quantities, as well as the associated nouns, verbs, adjectives, units, and rates that describe a quantity, are listed as nodes. Entity relationships are modeled according to the qualia roles of entities. For example, the edge <Isabel, flower, ACT> denotes an Action relationship between entities “Isabel” and “flower”, and <flower, bouquets, AGE> denotes an Agentive relationship between entities “flower” and “bouquets”. Quantity nodes associated to math entities denote the attribute values assigned to the connected math entities. Possible advantages of EDG are summarized below: 1)

It is a first attempt to integrate nodes of math entities, attributes and quantities into a single graph which provides more indicative information for solution tree construction.