benchmarking-gnns

Benchmarking Graph Neural Networks

Updates

May 10, 2022

• Project based on DGL 0.6.1 and higher. See the relevant dependencies defined in the environment yml files (CPU, GPU).
• Updated technical report of the framework on ArXiv.
• Added AQSOL dataset, which is similar to ZINC for graph regression task, but has a real-world measured chemical target.
• Added mathematical datasets – GraphTheoryProp and CYCLES which are useful to test GNNs on specific theoretical graph properties.
• Fixed issue #57.

Oct 7, 2020

• Repo updated to DGL 0.5.2 and PyTorch 1.6.0. Please update your environment using yml files (CPU, GPU).
• Added ZINC-full dataset (249K molecular graphs) with scripts.

Jun 11, 2020

• Second release of the project. Major updates :
  • Added experimental pipeline for Weisfeiler-Lehman-GNNs operating on dense rank-2 tensors.
  • Added a leaderboard for all datasets.
  • Updated PATTERN dataset.
  • Fixed bug for PATTERN and CLUSTER accuracy.
  • Moved first release to this branch.
• New ArXiv’s version of the paper.

Mar 3, 2020

• First release of the project.

1. Benchmark installation

Follow these instructions to install the benchmark and setup the environment.

2. Download datasets

Proceed as follows to download the benchmark datasets.

3. Reproducibility

Use this page to run the codes and reproduce the published results.

4. Adding a new dataset

Instructions to add a dataset to the benchmark.

5. Adding a Message-passing GCN

Step-by-step directions to add a MP-GCN to the benchmark.

6. Adding a Weisfeiler-Lehman GNN

Step-by-step directions to add a WL-GNN to the benchmark.

7. Leaderboards

Full leaderboards coming soon on paperswithcode.com.

8. Reference

ArXiv’s paper

@article{dwivedi2020benchmarkgnns,
  title={Benchmarking Graph Neural Networks},
  author={Dwivedi, Vijay Prakash and Joshi, Chaitanya K and Luu, Anh Tuan and Laurent, Thomas and Bengio, Yoshua and Bresson, Xavier},
  journal={arXiv preprint arXiv:2003.00982},
  year={2020}
}