[:bell: News! :bell: ]
We have released a new survey paper:“Generative Knowledge Graph Construction: A Review” based on this repository, with a perspective of existing Generative Knowledge Graph Construction! We are looking forward to any comments or discussions on this topic :)
🕵️ Introduction
Generative Knowledge Graph Construction (KGC) refers to those methods that leverage the sequence-to-sequence framework for building knowledge graphs, which is flexible and can be adapted to widespread tasks.
In this study, we summarize the recent compelling progress in generative knowledge graph construction.
We present the advantages and weaknesses of each paradigm in terms of different generation targets and provide theoretical insight and empirical analysis.
Based on the review, we suggest promising research directions for the future. Our contributions are threefold: (1) We present a detailed, complete taxonomy for the generative KGC methods;
(2) We provide a theoretical and empirical analysis of the generative KGC methods;
(3) We propose several research directions that can be developed in the future.
For more resources about knowledge graph construction, please check our tookit DeepKE.
👋 News!
- We release EasyEdit, an easy-to-use framework to edit Large Language Models.
- We release DeepKE-LLM to support knowledge extraction with KnowLM, ChatGLM, LLaMA-series, GPT-series etc.
- We release a survey and paper-list for reasoning with language model prompting.
- We release a prompt&KG paper-list at PromptKG.
- Congratulations! Our work has been accepted by the EMNLP2022 main conference.
- Due to the rise of generative extraction methods in the NLP community，we summarize recent progress in generative KGC and release our paper on arivx.
- We release Eng/Cn slides at Silde file.
🚩Citation
If you find this survey useful for your research, please consider citing
@article{DBLP:journals/corr/abs-2210-12714,
author = {Hongbin Ye and
Ningyu Zhang and
Hui Chen and
Huajun Chen},
title = {Generative Knowledge Graph Construction: {A} Review},
journal = {CoRR},
volume = {abs/2210.12714},
year = {2022},
url = {https://doi.org/10.48550/arXiv.2210.12714},
doi = {10.48550/arXiv.2210.12714},
eprinttype = {arXiv},
eprint = {2210.12714},
timestamp = {Fri, 28 Oct 2022 14:21:57 +0200},
biburl = {https://dblp.org/rec/journals/corr/abs-2210-12714.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
⚓️ Preliminary on Knowledge Graph Construction
Knowledge Graph Construction mainly aims to extract structural information from unstructured texts,
such as Named Entity Recognition (NER), Relation Extraction (RE), Event Extraction (EE), Entity Linking (EL), and Knowledge Graph Completion (KGC).
Generally, KGC can be regarded as structure prediction tasks, where a model is trained to approximate a target function $F(x) \rightarrow y$, where $x \in \mathcal{X}$ denotes the input data and $y \in \mathcal{Y}$ denotes the output structure sequence.
For instance, given a sentence, “Steve Jobs and Steve Wozniak co-founded Apple in 1977.”:
-
Named Entity Recognition aims to identify the types of entities, i.e., ‘Steve Job’, ‘Steve Wozniak’ $\Rightarrow$ PERSON, ‘Apple’ $\Rightarrow$ ORG;
-
Relation Extraction aims to identify the relationship of the given entity pair $\langle$Steve Job, Apple$\rangle$ as founder;
-
Event Extraction aims to identify the event type as Business Start-Org where ‘co-founded’ triggers the event and (Steve Jobs, Steve Wozniak) are participants in the event as AGENT and Apple as ORG respectively.
-
Entity Linking aims to link the mention Steve Job to Steven Jobs (Q19837) on Wikidata, and Apple to Apple (Q312) as well.
-
Knowledge Graph Completion aims to complete incomplete triples $\langle$Steve Job, create, ?$\rangle$ for blank entities Apple, NeXT Inc. and Pixar.
🏳🌈 A Taxonomy of Current Methods
In this Survey, we summarize recent progress in generative KGC. We propose to organize relevant work by the generation target of models and also present the axis of the task level.
1. Copy-based Sequence.
This paradigm refers to developing more robust models to copy the corresponding entity directly from the input sentence during the generation process.
As shown in figure, the model copies the head entity from the input sentence and then the tail entity.
2. Structure-linearized Sequence.
This paradigm refers to utilizing structural knowledge and label semantics, making it prone to handling a unified output format.
As shown in figure, the output is a linearization of the extracted knowledge structure.
-
Per-token tag encoding
-
“Exploring Sequence-to-Sequence Learning in Aspect Term Extraction”, ACL 2019
- Dehong Ma, Sujian Li, Fangzhao Wu, Xing Xie, Houfeng Wang
- [Paper]
-
“Neural Architectures for Nested NER through Linearization”, ACL 2019
- Jana Straková, Milan Straka, Jan Hajic
- [Paper]
-
Faithful contrastive learning
-
“Contrastive Triple Extraction with Generative Transformer”, AAAI 2021
- Hongbin Ye, Ningyu Zhang, Shumin Deng, Mosha Chen, Chuanqi Tan, Fei Huang, Huajun Chen
- [Paper]
-
“Contrastive Information Extraction with Generative Transformer”, IEEE ACM Trans. Audio Speech Lang. Process
- Ningyu Zhang, Hongbin Ye, Shumin Deng, Chuanqi Tan, Mosha Chen, Songfang Huang, Fei Huang, Huajun Chen
- [Paper]
-
“Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning”, ACL 2022
- Swarnadeep Saha, Prateek Yadav, Mohit Bansal
- [Paper]
-
Prefix tree constraint decoding
-
“Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction”, ACL 2021
- Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, Jialong Tang, Annan Li, Le Sun, Meng Liao, Shaoyi Chen
- [Paper]
-
“GenIE: Generative Information Extraction”, NAACL 2022
- Martin Josifoski, Nicola De Cao, Maxime Peyrard, Fabio Petroni, Robert West
- [Paper]
-
Triplet linearization
-
“REBEL: Relation Extraction By End-to-end Language generation”, EMNLP 2021
- Pere-Lluís Huguet Cabot, Roberto Navigli
- [Paper]
-
“De-Bias for Generative Extraction in Unified NER Task”, ACL 2022
- Shuai Zhang, Yongliang Shen, Zeqi Tan, Yiquan Wu, Weiming Lu
- [Paper]
-
Entity-aware hierarchical decoding
- “From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer”, WWW 2022
- Xin Xie, Ningyu Zhang, Zhoubo Li, Shumin Deng, Hui Chen, Feiyu Xiong, Mosha Chen, Huajun Chen
- [Paper]
-
Unified structure generation
-
“Unified Structure Generation for Universal Information Extraction”, ACL 2022
- Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, Hua Wu
- [Paper]
-
“DeepStruct: Pretraining of Language Models for Structure Prediction”, ACL 2022
- Chenguang Wang, Xiao Liu, Zui Chen, Haoyun Hong, Jie Tang, Dawn Song
- [Paper]
-
“Text-to-Text Extraction and Verbalization of Biomedical Event Graphs”, COLING 2022
- Giacomo Frisoni, Gianluca Moro, Lorenzo Balzani
- [Paper]
-
Reformulating triple prediction
-
“Intent Classification and Slot Filling for Privacy Policies”, ACL 2021
- Wasi Uddin Ahmad, Jianfeng Chi, Tu Le, Thomas Norton, Yuan Tian, Kai-Wei Chang
- [Paper]
-
“HySPA: Hybrid Span Generation for Scalable Text-to-Graph Extraction”, ACL 2021
- Liliang Ren, Chenkai Sun, Heng Ji, Julia Hockenmaier
- [Paper]
-
“SQUIRE: A Sequence-to-sequence Framework for Multi-hop Knowledge Graph Reasoning”, EMNLP 2022
- Yushi Bai, Xin Lv, Juanzi Li, Lei Hou, Yincen Qu, Zelin Dai, Feiyu Xiong
- [Paper]
-
Query Verbalization
-
“Improving Candidate Retrieval with Entity Profile Generation for Wikidata Entity Linking”, ACL 2022
- Tuan Lai, Heng Ji, ChengXiang Zhai
- [Paper]
-
“Sequence-to-Sequence Knowledge Graph Completion and Question Answering”, ACL 2022
- Apoorv Saxena, Adrian Kochsiek, Rainer Gemulla
- [Paper]
-
“Knowledge Is Flat: A Seq2Seq Generative Framework for Various Knowledge Graph Completion”, COLING 2022
- Chen Chen, Yufei Wang, Bing Li, Kwok-Yan Lam
- [Paper]
3. Label-based Sequence.
This paradigm refers to utilizing the extra markers to indicate specific entities or relationships.
As shown in figure, the output sequence copies all words in the input sentence, as it helps to reduce ambiguity.
In addition, this paradigm uses square brackets or other identifiers to specify the tagging sequence for the entity of interest.
The relevant labels are separated by the separator “$|$” within enclosed brackets.
Meanwhile, the labeled words are described with natural words so that the potential knowledge of the pre-trained model can be leveraged.
4. Indice-based Sequence.
This paradigm generates the indices of the words in the input text of interest directly, and encodes class labels as label indices.
As the output is strictly restricted, it will not generate indices that corresponding entities do not exist in the input text, except for relation labels.
-
Pointer mechanism
-
“Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction Authors”, AAAI 2020
-
“Don’t Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing”, WWW 2020
- Subendhu Rongali, Luca Soldaini, Emilio Monti, Wael Hamza
- [Paper]
-
“A Unified Generative Framework for Various NER Subtasks”, ACL 2021
- Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, Xipeng Qiu
- [Paper]
-
“A Unified Generative Framework for Aspect-based Sentiment Analysis”, ACL 2021
- Hang Yan, Junqi Dai, Tuo Ji, Xipeng Qiu, Zheng Zhang
- [Paper]
-
Pointer selection
- “GRIT: Generative Role-filler Transformers for Document-level Event Entity Extraction”, EACL 2021
- Xinya Du, Alexander Rush, Claire Cardie
- [Paper]
5. Blank-based Sequence.
This paradigm refers to utilizing templates to define the appropriate order and relationship for the generated spans.
As shown in figure, the template refers to a text describing an event type, which adds blank argument role placeholders.
The output sequences are sentences where the blank placeholders are replaced by specific event arguments.
🏆 A List of Survey Papers
🕚 A Timeline of generative KGC.
The time for each paper is based on its first arXiv version (if exists) or estimated submission time.
🌟 TIPS
If you find this repository useful to your research or work, it is really appreciate to star this repository.
评论