knowledge-graph-embeddings
Python Implementations of Embedding-based methods for Knowledge Base Completion tasks, mainly inspired by scikit-kge and complex.
List of methods
- RESCAL [Nickel+. 2011]
- TransE [Bordes+. 2013]
- DistMult [Yang+. 2015]
- HolE [Nicklel+. 2016]
- This model is equivalent to ComplEx[Hayashi and Shimbo. 2018], and the computation cost of ComplEx is lower than of HolE.
- ComplEx [Trouillon+. 2016]
- ANALOGY [Liu+. 2017]
- This model can be regarded as a hybrid between DistMult and ComplEx.
Run to train and test
For training...
â–¶ python train.py -h
usage: Link prediction models [-h] [--mode MODE] [--ent ENT] [--rel REL]
[--train TRAIN] [--valid VALID]
[--method METHOD] [--epoch EPOCH]
[--batch BATCH] [--lr LR] [--dim DIM]
[--margin MARGIN] [--negative NEGATIVE]
[--opt OPT] [--l2_reg L2_REG]
[--gradclip GRADCLIP] [--save_step SAVE_STEP]
[--cp_ratio CP_RATIO] [--metric METRIC]
[--nbest NBEST] [--filtered]
[--graphall GRAPHALL] [--log LOG]
optional arguments:
-h, --help show this help message and exit
--mode MODE training mode ["pairwise", "single"]
--ent ENT entity list
--rel REL relation list
--train TRAIN training data
--valid VALID validation data
--method METHOD method ["complex", "distmult", "transe", "hole",
"rescal", "analogy"]
--epoch EPOCH number of epochs
--batch BATCH batch size
--lr LR learning rate
--dim DIM dimension of embeddings
--margin MARGIN margin in max-margin loss for pairwise training
--negative NEGATIVE number of negative samples for pairwise training
--opt OPT optimizer ["sgd", "adagrad"]
--l2_reg L2_REG L2 regularization
--gradclip GRADCLIP gradient clipping
--save_step SAVE_STEP
epoch step for saving model
--cp_ratio CP_RATIO ratio of complex's dimention in ANALOGY
--metric METRIC evaluation metrics ["mrr", "hits"]
--nbest NBEST n-best for hits metric
--filtered use filtered metric
--graphall GRAPHALL all graph file for filtered evaluation
--log LOG output log dir
For testing...
â–¶ python test.py -h
usage: Link prediction models [-h] [--ent ENT] [--rel REL] [--data DATA]
[--filtered] [--graphall GRAPHALL]
[--method METHOD] [--model MODEL]
optional arguments:
-h, --help show this help message and exit
--ent ENT entity list
--rel REL relation list
--data DATA test data
--filtered use filtered metric
--graphall GRAPHALL all graph file for filtered evaluation
--method METHOD method ["complex", "distmult", "transe", "hole",
"rescal", "analogy"]
--model MODEL trained model path
Experiments
WordNet (WN18)
| Models | MRR (flt) | MRR (raw) | Hits@1 (flt) | Hits@3 (flt) | Hits@10 (flt) |
|---|---|---|---|---|---|
| ComplEx* | 94.1 | 58.7 | 93.6 | 94.5 | 94.7 |
| ComplEx | 94.3 | 58.2 | 94.0 | 94.6 | 94.8 |
hyper parameters
- mode : single
- epoch : 500
- batch : 128
- lr : 0.05
- dim : 200
- negative : 5
- opt : adagrad
- l2_reg : 0.001
- gradclip : 5
FreeBase (FB15k)
| Models | MRR (flt) | MRR (raw) | Hits@1 (flt) | Hits@3 (flt) | Hits@10 (flt) |
|---|---|---|---|---|---|
| ComplEx* | 69.2 | 24.2 | 59.9 | 75.9 | 84.0 |
| ComplEx | 69.5 | 24.2 | 59.8 | 76.9 | 85.0 |
hyper parameters
- mode : single
- epoch : 500
- batch : 128
- lr : 0.05
- dim : 200
- negative : 10
- opt : adagrad
- l2_reg : 0.0001
- gradclip : 5
* means the results reported from the original papers
Dependencies
- numpy
- scipy
References
-
Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; and Yakhnenko, O. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems (NIPS).
-
Liu, H.; Wu, Y.; and Yang, Y. 2017. Analogical inference for multi-relational embeddings. In Proceedings of the 34th International Conference on Machine Learning (ICML).
-
Nickel, M.; Rosasco, L.; and Poggio, T. 2016. Holographic embeddings of knowledge graphs. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16.
-
Nickel, M.; Tresp, V.; and Kriegel, H.-P. 2011. A threeway model for collective learning on multi-relational data. In International Conference on Machine Learning (ICML-11), ICML ’11,
-
Trouillon, T.; Welbl, J.; Riedel, S.; Gaussier, E.; and Bouchard, G. 2016. Complex embeddings for simple link prediction. In International Conference on Machine Learning (ICML).
-
Yang, B.; Yih, W.; He, X.; Gao, J.; and Deng, L. 2015. Embedding entities and relations for learning and inference in knowledge bases. International Conference on Learning Representations 2015.