In addition to the one-hop neighborhood information, the non-directly related high-order neighborhood information is also very important for the representation of the central entity [16]. Therefore, MSM first constructed a multi-neighborhood network representation learning method (MNE), using this representation method learns the structure embedding of KG to gather the high-order neighborhood structure information of the entity. The overall architecture and processing pipeline of MNE is shown in Fig. 2. MSM uses pre-trained word vectors to initialize MNE.

We use G=(E,R,T) to represent a knowledge graph, in which E,R,T represents entities, relationships, and sets of triples. Without loss of generality, we consider the entity alignment task between two knowledge graphs G1 and G2 based on a set of pre-aligned equivalent entities. Our goal is to find an equivalent entity pair between G1 and G2 . We put G1 and G2 as a large input graph into MSM, each MNE layer consists of two layers of GCN aggregated entities of one-hop neighborhood information, a layer of attention-network (ATT-Net) aggregated entities two-hop neighborhood information and each GNN updates the node representation as:

(1) hi,1(l)=ReLU(∑j∈N1(i)∪{i}1εiW1lhj(l−1)) where hi,1(l) is the output node features of l -th GCN layer. εi is the normalization constant. Ni is the set of neighbor indices of entity i . W1(l)∈Rd(l)×d(l−1) is the attention weight of l layer neighborhood of node i obtained by GNN training. To aggregate the two-hop neighborhood information of entity i , we have adopted ATT-Net. The hidden representation of entity i , denoted as hi,2l , is computed as follows:

(2) hi,2(l)=ReLU(∑j∈N2(i)∪{i}αij(l)W2lhj,2(l−1)) where hi,2(l) is the output node features of l -th ATT-Net layer. αij(l) is a learnable attention weight for entities i and its neighbor j . W2(l)∈Rd(l)×d(l−1) is the wight matrix. Finally, for entity i , we use the gating mechanism to combine its one-hop and two-hop neighborhood information. The node representations are given as below:

(3) hi(l)=g(hi,1(l))(g(hi,2(l))⋅hi,1(l)+(1−g(hi,2(l)))⋅hi,2(l)+(1−g(hi,1(l)))⋅hi,1(l) where g(hi,1(l))=σ(M1hi,1(l)+b1) , g(hi,2(l))=σ(M2hi,2(l)+b2) and M1 , M2 and b1 , b2 are the weight matrix and bias vector, respectively. In order to control the accumulated noise, we also introduce the highway network [15] into the MNE layer, which can effectively control the propagation of noise in the MNE layer.

The first-order and second-order neighborhoods of an entity are the key to determining whether the entity should be aligned with other entities. However, as we discussed before, not all first-order and second-order neighborhoods make a positive contribution to entity alignment. In order to select the correct neighbor, we use a down-sampling process to select the entity that provides the most information to the central target entity from its first-order and second-order neighborhoods. Previously, many random sampling methods were used, but the method have some randomness. The neighbor nodes sampled may not necessarily contribute the most. Therefore, we give a sampling method based on probability. Formally, given an entity ei , the calculation formula for sampling its first-hop domain ei−j and second-hop domain ei−j′ are [16]:

(4) p(hi−j|hi)=softmax(hiW1hi−jT)=exp⁡(hiW1hi−jT)∑k∈Ni1exp⁡(hiW1hi−kT)p(hi−j′|hi)=softmax(hiW2hi−j′T)=exp⁡(hiW2hi−jT)∑k∈Ni2exp⁡(hiW2hi−kT)

where Ni1 and Ni2 are the first-hop and second-hop neighborhood index set of the central entity ei , respectively. hi , hi−j and hi−j′ are the entity representation vectors of the entities ei , ei−j and ei−j′ respectively, and W1 , W2 are shared weight matrix.

By selectively sampling first-hop and second-hop neighborhoods, MSM gets the most valuable neighborhood information of each entity. In the end, MSM achieves the alignment of G1 and G2 through neighborhood matching and neighborhood aggregation.

In order to judge whether the two entities should be aligned, we need a similarity calculation to its neighbor nodes. MSM builds hence a neighborhood subgraph generated by the sampling process for each entity. Then MSM will only operate neighbors within the subgraph, achieving the neighborhood matching.

Inspired by the graph matching method [5], our specific matching method is as follows.

We need to compare the subgraphs of each candidate entity in its sampling neighborhood subgraph E2 to select the best aligned entity of the entity ei in E1 . But if each entity neighborhood in E2 is compared and calculated with ei , this requires too much calculation. Therefore, MSM adopts an approximate alternative method. Inspired by the candidate selection method [16], MSM samples an entity alignment set Ci={ci1,ci2,...,cit|cit∈E2} of ei , and then calculates the subgraph similarity between ei and these candidate entities. Therefore, for the entity ej in E2 , the calculation formula for it to be a candidate entity for ei is:

(5) p(hj|hi)=exp⁡(hi⋅hj)∑k∈E2exp⁡(hi⋅hk) where hi and hj are the node vector representations of ei and ej output by MNE.

We propose the following graph matching network, which changes the node update module in each propagation layer. It not only considers the aggregated messages on the edge of each graph as before, but also considers a cross-graph matching vector, which measures the degree of matching between a node in one graph and one or more nodes in another graph. Define (ei,ej) as the entity pair to be measured, where ei∈E1 , ej∈E2 . The cross-matching vector of ei is calculated as follows:

aj→i=cos⁡ine(hi,hj)

hi−1=∑j=1Niaj→i⋅hj∑j=1Niaj−>i

(6) mi=hi−hi−1

Here aj→i is the cosine similarities of entity ei in E1 with the entity ej in E2 , mi is the matching vector of node ei , Ni is the sampled neighbor set of ei , hi and hj are the node vector representations of ei and ej output by MNE, respectively. hi−1 is an attentive vector for the entire graph E2 . Then, we combine hi and mi to get the sampled neighbor representation of ei :

(7) h⌢i=hi⊕β*mi where ⊕ indicates vector concatenation.

We give the loss function of MNE:

(8) L=∑(i,j)∈X∑(i′,j′)∈X′max{0,d(i,j)−d(i′−j′)+γ} where d(i,j)=‖hi−hj‖L1 , ‖⋅‖L2 means L2−norm ; γ>0 is a margin hyper-parameter; X is the alignment seeds and X′ is the set of negative aligned entity pairs generated by nearest neighbor sampling [15].

The loss function of MSM after the pre-training phase, defined as:

(9) L=∑(i,j)∈T∑(i′,j′)∈T′max{0, f(i,j)−f(i′−j′)+γ}

where f(i,j)=‖h^i−h⌢j‖L1; T is the positive alignments set and T′ the negative alignments set.

To compares the various methods more objectively and comprehensively, in the experiment, the commonly used model evaluation indicators Hit@k in the field of entity alignment are used, and the calculation formula is as follows:

(11) Hits@k=nrkN where nrk represents the number of times the target node appears in the k closest candidate target nodes, and N is the number of tests. We select Hits@1 and Hits@10 indicators according to the commonly used experimental methods.

(1) The experimental results of each method on different data subsets of the data set DBP15k are shown in Tab. 2. From Tab. 2, we can see that under the evaluation index of Hits@1 , MSM has an improvement of 3% to 5% relative to GMNN, and also has an improvement of about 3% under the evaluation index of Hits@10 . The result meets 8% improvement on zh-en data subset. Compared with JAPE, AliNet and BoostEA, MSM has a greater improvement.

To evaluate the sampling method on the accuracy of the model, we implement a model variant (referred as MSM _(s-two)) that samples two-hop neighbors. From Tab. 2, we find that the sampling two-hop neighbors leads MSM a gain of 0.4%~0.8% by Hits@1 and 0.1%~0.3% by Hits@10 relative to the sampling one-hop neighbors (referred as MSM _(s-one)) on DBP15K. The result confirms the importance of the two-hop neighborhoods in KG structure embedding and neighborhood sampling.

(2) To explore the impact of using the pre-trained word embeddings to initialize the MSM. We remove the initialization part of the MNE, choosing the different sampling size for comparative analysis. From Fig. 3, for DBP15K_ZH-EN, we observe that the initialization of MNE is very important of MSM. More exactly, removing the initialization from MSM, leads to around a 43% drop in Hits@1 and a 40% drop in Hits@10 on average. On the other hand, MSM is sensitive to the size of sampling and the model works better when the size of sampling is 5.

(3) To verify the superiority of our cross-graph neighborhood matching method, we compare it with the common direct aggregate matching method. As shown in Fig. 4, when we choose different candidate set sizes, the cosine cross-neighborhood matching method performs better. It increases 0.5%~1% relative to the overall the cross-neighborhood matching method under the evaluation index of Hits@10 and about 0.2% under the evaluation index of Hits@1 . At the same time, the different matching candidate size also has some influence on MSM performance. When the size of candidate selection is 18, the performance of the MSM is better on DBP15K_ZH-EN.

(4) To show the effect of entity alignment more intuitively, this part visually shows the node distribution consistency before and after entity alignment. We reduce the 300-dimensional vector to two-dimensional space through t-sne. Fig. 5 show the visualization of 4000 pairs of nodes randomly selected from two KG on DBP15K_JA-EN.

These result show that our pre-trained vectors, sampling and matching modules are particularly important, when the neighborhood sizes of equivalent entities greatly differ and especially there may be few common neighbors in their neighborhoods.