Transformers with knowledge base