arXiv (Cornell University) / 2017

Graph Attention Networks

Veli\v{c}kovi\'c, Petar, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Li\`o, Pietro, Yoshua Bengio

Graph LearningPopular and Landmark Papers

We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset (wherein test graphs remain unseen during training).

8,306 citations0 influential

Full paper

Read the original paper

Open PDF Source page

Learning resources

arXiv PDFPDF arXiv abstract pagearXiv Google Scholar referencesGoogle Scholar Paper pageOpenAlex Papers with Code searchPapers with Code YouTube explanationsYouTube

Reading state

Discuss in ChatGPT

Uses your own ChatGPT account. The paper context is copied into a tutor prompt before ChatGPT opens.

Preview prompt

You are my AI/ML research paper instructor. I want to deeply understand the paper below.

First, teach it in layers:
1. One-paragraph intuition.
2. Problem statement and why it mattered.
3. Key method, architecture, or algorithm.
4. Important equations or mechanisms, explained intuitively.
5. Experiments and evidence.
6. Limitations, assumptions, and failure modes.
7. How this paper influenced later AI/ML/Deep Learning/GenAI work.
8. A 30-minute study plan with checkpoints.
9. Quiz me with 5 questions and wait for my answers.

When something is not available in the attached context, say what is missing and infer carefully.

### Paper attached as context
Title: Graph Attention Networks
Authors: Veli\v{c}kovi\'c, Petar, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Li\`o, Pietro, Yoshua Bengio
Year: 2017
Venue: arXiv (Cornell University)
Categories: Graph Learning, Popular and Landmark Papers
Citations: 8,306
Paper URL: https://arxiv.org/abs/1710.10903
Open PDF: https://arxiv.org/pdf/1710.10903

Abstract:
We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over their neighborhoods' features, we enable (implicitly) specifying different weights to different nodes in a neighborhood, without requiring any kind of costly matrix operation (such as inversion) or depending on knowing the graph structure upfront. In this way, we address several key challenges of spectral-based graph neural networks simultaneously, and make our model readily applicable to inductive as well as transductive problems. Our GAT models have achieved or matched state-of-the-art results across four established transductive and inductive graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as a protein-protein interaction dataset (wherein test graphs remain unseen during training).

Learning resources:
- PDF: arXiv PDF (https://arxiv.org/pdf/1710.10903)
- arXiv: arXiv abstract page (https://arxiv.org/abs/1710.10903)
- Google Scholar: Google Scholar references (https://scholar.google.com/scholar?q=Graph%20Attention%20Networks)
- OpenAlex: Paper page (http://arxiv.org/abs/1710.10903)
- Papers with Code: Papers with Code search (https://paperswithcode.com/search?q=Graph%20Attention%20Networks)
- YouTube: YouTube explanations (https://www.youtube.com/results?search_query=Graph%20Attention%20Networks+paper+explained)