摘要: Knowledge Distillation (KD) as a model compression technique has been widely used in Graph Neural Networks (GNNs). Recently research has demonstrated that GNNs with shallow or deep hidden layers have different representational abi... 展开 Knowledge Distillation (KD) as a model compression technique has been widely used in Graph Neural Networks (GNNs). Recently research has demonstrated that GNNs with shallow or deep hidden layers have different representational abilities on nodes with low or high degrees. The use of multi-teacher models can provide diverse knowledge sources for the student model. However, a significant amount of information is missing when the knowledge is transferred through the deep hidden layers in the multi-teacher models. It is difficult for multi-teacher models to transfer the information of each hidden layer to the student model while preserving the local semantics of their respective models. To solve this problem, we propose a novel multi-teacher knowledge distillation method. Multi-teacher Local Semantic distillation from Graph Neural Networks (MLS-GNN). MLS-GNN is able to agglomerate the representation capabilities of shallow and deep GNN teacher models on nodes with different degrees. Not only that, the local semantics of each hidden layer in multi-teacher models is preserved through nodes and edges. Finally, the adaptive weights are assigned to different teacher models to stabilize the knowledge acquisition process and summarize the local semantics. We conducted experiments on five public datasets for node classification task, and the results show that MLS-GNN has excellent performance and better than other baseline methods. 收起