Transformers with Knowledge Graph | K-BERT: Enabling Language Representation with Knowledge Graph

preview_player
Показать описание
In this video, we review the paper K-BERT: Enabling Language Representation with Knowledge Graph.
This paper proposes an approach to connect BERT models to external knowledge graphs like Wikipedia knowledge graphs, medical knowledge graphs, or any other knowledge graph.

If you want to refresh your understanding of transformers and self-attention check out these videos :
.
.
Some of my other popular videos on Deep learning :

.
.

Music from #Uppbeat (free for Creators!):
License code: HMQG1YGQY83EWIPU
  1. Dingu Sagar
  2. Transformers with knowledge base
  3. transformers
  4. bert
  5. neural networks
  6. ai
Рекомендации по теме
Transformers with Knowledge 0:11:26

Transformers with Knowledge Graph | K-BERT: Enabling Language Representation with Knowledge Graph

Representation Learning on 0:17:20

Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers (KDD2023)

Knowledge Graph Construction 0:06:42

Knowledge Graph Construction Demo from raw text using an LLM

Knowledge graph embedding 0:00:16

Knowledge graph embedding llm #transformers #coding

Graphormer - Do 0:38:41

Graphormer - Do Transformers Really Perform Bad for Graph Representation? | Paper Explained

Knowledge Graph Transformers 0:07:47

Knowledge Graph Transformers and FinDKG: A Guide for Investors

Graphs and Transformers: 0:45:19

Graphs and Transformers: Exploring the Next AI Frontiers

Langchain & Neo4j: 0:12:06

Langchain & Neo4j: Create Knowledge Graphs from Text

Building a GraphRAG 0:22:53

Building a GraphRAG Agent with Neo4j and Milvus

KDD 2023 -Representation 0:01:59

KDD 2023 -Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers

Easily build quality 0:06:47

Easily build quality Knowledge Graphs From Text

How large language 0:27:14

How large language models work, a visual intro to transformers | Chapter 5, Deep Learning

CVPR2023: TextKG - 0:04:14

CVPR2023: TextKG - Text with Knowledge Graph Augmented Transformer for Video Captioning

093 Keyword Disambiguation 0:35:11

093 Keyword Disambiguation Using Transformers and Clustering to Build Cleaner Knowledge - NODES2022

GraphGPT: Transform Text 0:06:31

GraphGPT: Transform Text into Knowledge Graphs with GPT-3

GraphGPT: Instant Text 0:04:21

GraphGPT: Instant Text to Knowledge Graph App

Rik Koncel-Kedziorski: 'Text 0:50:47

Rik Koncel-Kedziorski: 'Text Generation from Knowledge Graphs'

Neural Networks vs 0:00:36

Neural Networks vs Transformer Models!

Transformers are Graph 0:08:42

Transformers are Graph Attention Networks !? - Oxford Geometric Deep Learning

Graph.Build Platform Architecture 0:02:57

Graph.Build Platform Architecture - No-code Automation of your Knowledge Graph Production

Building Knowledge Graphs 0:29:27

Building Knowledge Graphs with Rebel: Step By Step Guide for Extracting Entities & Enriching Inf...

Convert Any Text 0:30:52

Convert Any Text into a Knowledge Graph

Neo4j: How to 0:02:39

Neo4j: How to create knowledge graphs with gen AI

Step-by-step Code for 0:31:12

Step-by-step Code for Knowledge Graph Construction

Комментарии
Автор

Thanks for your explanation. Good job!

ApusApus-
Автор

I want to share my experience with this approach. I am currently working with this model for my thesis project. It seems that the author used their own dataset, which, as far as I can see, is in Chinese. The paper shows promising results with this approach. However, when I implement this with my own data, as well as my own custom knowledge graph, it fails. During training, it consistently shows an "Error: Divided by Zero, " indicating that the model fails to classify one or more labels. Due to this, I am currently exploring alternative ways of implementing a knowledge graph into a transformer-based architecture, in this context, is BERT.

ariefpurnamamuharram