cair/TsetlinMachine cair/TsetlinMachine

  • Stars
    star
    444
  • Rank 98,300 (Top 2 %)
  • Language Cython
  • License
    MIT License
  • Created over 6 years ago
  • Updated 10 months ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
cair/TsetlinMachine
github

cair

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Code and datasets for the Tsetlin Machine

Tsetlin Machine

License Maintenance

Code and datasets for the Tsetlin Machine. Implements the Tsetlin Machine from https://arxiv.org/abs/1804.01508, including the multiclass version. The Tsetlin Machine solves complex pattern recognition problems with easy-to-interpret propositional formulas, composed by a collective of Tsetlin Automata.

Contents

Basics

Classification

A basic Tsetlin Machine takes a vector of Boolean features as input, to be classified into one of two classes, or . Together with their negated counterparts, , the features form a literal set .

A Tsetlin Machine pattern is formulated as a conjunctive clause , formed by ANDing a subset of the literal set:

.

For example, the clause consists of the literals and outputs iff and .

The number of clauses employed is a user-configurable parameter . Half of the clauses are assigned positive polarity. The other half is assigned negative polarity. The clause outputs, in turn, are combined into a classification decision through summation and thresholding using the unit step function :

.

In other words, classification is based on a majority vote, with the positive clauses voting for and the negative for . The classifier

,

for instance, captures the XOR-relation.

Learning

A clause is composed by a team of Tsetlin Automata, each Tsetlin Automaton deciding to Include or Exclude a specific literal in the clause (see figure above). Learning which literals to include is based on reinforcement: Type I feedback produces frequent patterns, while Type II feedback increases the discrimination power of the patterns.

A Tsetlin Machine learns on-line, processing one training example at a time.

Type I feedback is given stochastically to clauses with positive polarity when and to clauses with negative polarity when . An afflicted clause, in turn, reinforces each of its Tsetlin Automata based on: (i) the clause output ; (ii) the action of the targeted Tsetlin Automaton - Include or Exclude; and (iii) the value of the literal assigned to the automaton. As shown in Table 1, two rules govern Type I feedback, given independently to each Tsetlin Automaton of the clause:

  • Include is rewarded and Exclude is penalized with probability whenever and . This reinforcement is strong (triggers with high probability) and makes the clause remember and refine the pattern it recognizes in .
  • Include is penalized and Exclude is rewarded with probability if or . This reinforcement is weak (triggers with low probability) and coarsens infrequent patterns, making them frequent.

Above, is a hyperparameter that controls the frequency of the patterns produced.

Type II feedback is given stochastically to clauses with positive polarity when and to clauses with negative polarity when . Again, an affected clause reinforces each of its Tsetlin Automata based on: (i) the clause output ; (ii) the action of the targeted Tsetlin Automaton - Include or Exclude; and (iii) the value of the literal assigned to the automaton. As captured by Table 2, Type II feedback penalizes Exclude whenever and . This feedback is strong and produces candidate literals for discriminating between and .

Resource allocation dynamics ensure that clauses distribute themselves across the frequent patterns, rather than missing some and overconcentrating on others. That is, for any input , the probability of reinforcing a clause gradually drops to zero as the clause output sum

approaches a user-set target for ( for ). To exemplify, the below plot shows the probability of reinforcing a clause when and for different clause output sums :

If a clause is not reinforced, it does not give feedback to its Tsetlin Automata, and these are thus left unchanged. In the extreme, when the voting sum equals or exceeds the target (the Tsetlin Machine has successfully recognized the input ), no clauses are reinforced. Accordingly, they are free to learn new patterns, naturally balancing the pattern representation resources.

See https://arxiv.org/abs/1804.01508 for details.

Learning Behaviour

The below figure depicts average learning progress (over 50 runs) of the Tsetlin Machine on a binarized, but otherwise unenhanced version of the MNIST dataset (https://en.wikipedia.org/wiki/MNIST_database). See also https://github.com/cair/fast-tsetlin-machine-with-mnist-demo.

As seen in the figure, both test and training accuracy increase almost monotonically across the epochs. Even while accuracy on the training data approaches 99.9%, accuracy on the test data continues to increase as well, hitting 98.2% after 400 epochs. This is quite different from what occurs with backpropagation on a neural network, where accuracy on test data starts to drop at some point due to overfitting, without proper regularization mechanisms.

Noisy XOR Demo

./NoisyXORDemo.py

Accuracy on test data (no noise): 1.0
Accuracy on training data (40% noise): 0.603

Prediction: x1 = 1, x2 = 0, ... -> y =  1
Prediction: x1 = 0, x2 = 1, ... -> y =  1
Prediction: x1 = 0, x2 = 0, ... -> y =  0
Prediction: x1 = 1, x2 = 1, ... -> y =  0

Requirements

Other Implementations

Other Architectures

Hardware

Books

Conferences

Videos

  • Watching the state transitions of Tsetlin Automata. Demo by Jie Lei, Microsystems Research Group, Newcastle University. https://youtu.be/wXyiLtlpwHI
  • Keyword Spotting using Tsetlin Machines. Demo by Jie Lei, Microsystems Research Group, Newcastle University. https://youtu.be/JW0tztpjX8k
  • Mignon AI Presentation at Arm Summit 2020 by Adrian Wheeldon and Jie Lei, Microsystems Research Group, Newcastle University. https://youtu.be/N-wkgibJAZE
  • Explainability and Dependability Analysis of Learning Automata based AI Hardware. IOLTS presentation by Rishad Shafik, Microsystems Research Group, Newcastle University. https://youtu.be/IjzZY0fDYiA
  • Tsetlin Machine - A new paradigm for pervasive AI. DATE SCONA Workshop presentation by Adrian Wheeldon, Microsystems Research Group, Newcastle University. https://youtu.be/TaspuovmSR8
  • Quick Guide to the Tsetlin Machine using Sequential Logic in Logisim. Presentations by Jie Lei, Microsystems Research Group, Newcastle University.
  • Tsetlin Machine on Iris Data Set Demo with Handheld MignonAI (http://www.mignon.ai). Presentation by Jie Lei, Microsystems Research Group, Newcastle University. https://youtu.be/BzaPGByX-hg
  • The-Ruler-of-Tsetlin-Automaton. Presentation by Jie Lei, Microsystems Research Group, Newcastle University. https://youtu.be/LltDhg4ZuWo
  • Interpretable clustering and dimension reduction with Tsetlin automata machine learning. Presentation by Christian D. Blakely, PwC Switzerland. https://youtu.be/5-09LOGLcV8
  • Predicting and explaining economic growth using real-time interpretable learning. Presentation by Christian D. Blakely, PwC Switzerland. https://youtu.be/J6K7V7V7ayo
  • Early detection of breast cancer from a simple blood test. Presentation by Christian D. Blakely, PwC Switzerland. https://youtu.be/FrHN_aRLRug
  • Recent advances in Tsetlin Machines. NORA.ai Webinar presentation by Ole-Christoffer Granmo, CAIR, University of Agder. https://youtu.be/GHelDh3bN00

Tutorials

Convolutional Tsetlin Machine tutorial, https://github.com/cair/convolutional-tsetlin-machine-tutorial

Acknowledgements

I thank my colleagues from the Centre for Artificial Intelligence Research (CAIR), Lei Jiao, Xuan Zhang, Geir Thore Berge, Darshana Abeyrathna, Saeed Rahimi Gorji, Sondre Glimsdal, Rupsa Saha, Bimal Bhattarai, Rohan K. Yadav, Bernt Viggo Matheussen, Morten Goodwin, Christian Omlin, Vladimir Zadorozhny (University of Pittsburgh), Jivitesh Sharma, Ahmed Abouzeid, and Charul Giri, for their contributions to the development of the Tsetlin machine family of techniques. I would also like to thank our House of CAIR partners, Alex Yakovlev, Rishad Shafik, Ashur Rafiev, Sidharth Maheshwari, Adrian Wheeldon, Jie Lei, Tousif Rahman, (Newcastle University), Jonny Edwards (Temporal Computing), Marco Wiering (University of Groningen), Christian D. Blakely (PwC Switzerland), Adrian Phoulady, Anders Refsdal Olsen, Halvor Smørvik, and Erik Mathisen for their many contributions.

Tsetlin Machine Papers

@InProceedings{yadav2022robust,
  title     = {Robust Interpretable Text Classification against Spurious Correlations Using AND-rules with Negation},
  author    = {Yadav, Rohan Kumar and Jiao, Lei and Granmo, Ole-Christoffer and Goodwin, Morten},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Lud De Raedt},
  pages     = {4439--4446},
  year      = {2022},
  month     = {7},
  note      = {Main Track}
  doi       = {10.24963/ijcai.2022/616},
  url       = {https://doi.org/10.24963/ijcai.2022/616},
}
@InProceedings{bhattarai2022fakenews,
  author    = {Bhattarai, Bimal  and  Granmo, Ole-Christoffer  and  Jiao, Lei},
  title     = {Explainable Tsetlin Machine Framework for Fake News Detection with Credibility Score Assessment},
  booktitle      = {Proceedings of the Language Resources and Evaluation Conference},
  month          = {June},
  year           = {2022},
  address        = {Marseille, France},
  publisher      = {European Language Resources Association},
  pages     = {4894--4903},
  url       = {https://aclanthology.org/2022.lrec-1.523}
}
@InProceedings{bhattarai2022convtext,
  author    = {Bhattarai, Bimal  and  Granmo, Ole-Christoffer  and  Jiao, Lei},
  title     = {ConvTextTM: An Explainable Convolutional Tsetlin Machine Framework for Text Classification},
  booktitle      = {Proceedings of the Language Resources and Evaluation Conference},
  month          = {June},
  year           = {2022},
  address        = {Marseille, France},
  publisher      = {European Language Resources Association},
  pages     = {3761--3770},
  url       = {https://aclanthology.org/2022.lrec-1.401}
}
@article{saha2021disc,
  author = {Saha, Rupsa and Granmo, Ole-Christoffer and Goodwin, Morten},
  title = {Using Tsetlin Machine to discover interpretable rules in natural language processing applications},
  journal = {Expert Systems},
  url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.12873},
  year={2021}
}
@article{abeyrathna2021multistep,
  author = {Abeyrathna, Kuruge Darshana and Granmo, Ole-Christoffer and Shafik, Rishad and Jiao, Lei and Wheeldon, Adrian and Yakovlev, Alex and Lei, Jie and Goodwin, Morten},
  title = {A multi-step finite-state automaton for arbitrarily deterministic Tsetlin Machine learning},
  journal = {Expert Systems},
  url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.12836},
  year={2021}
}
@InProceedings{yadav2021dwr,
  title={Enhancing Interpretable Clauses Semantically using Pretrained Word Representation},
  author={Rohan Kumar Yadav and Lei Jiao and Ole-Christoffer Granmo and Morten Goodwin},
  booktitle={BLACKBOXNLP},
  url = {https://aclanthology.org/2021.blackboxnlp-1.19.pdf},
  year={2021}
}
@article{jiao2021andorconvergence,
  title={On the Convergence of Tsetlin Machines for the AND and the OR Operators},
  author={Lei Jiao and Xuan Zhang and Ole-Christoffer Granmo},
  journal = {arXiv preprint arXiv:2109.09488}, year = {2021},
  url = {https://arxiv.org/abs/2109.09488}
}
@InProceedings{wheeldon2021self,
  title="{Self-Timed Reinforcement Learning using Tsetlin Machine}",
  author={Adrian Wheeldon and Alex Yakovlev and Rishad Shafik},
  booktitle={27th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC 2021)},
  year={2021},
  organization={IEEE},
  url={https://arxiv.org/abs/2109.00846}
}
@article{glimsdal2021coalesced,
  title={Coalesced Multi-Output Tsetlin Machines with Clause Sharing},
  author={Sondre Glimsdal and Ole-Christoffer Granmo},
  journal = {arXiv preprint arXiv:2108.07594}, year = {2021},
  url = {https://arxiv.org/abs/2108.07594}
}
@article{Abeyrathna2021adaptivesparse,
  title="{Adaptive Sparse Representation of Continuous Input for Tsetlin Machines Based on Stochastic Searching on the Line}",
  volume={10},
  ISSN={2079-9292},
  url={http://dx.doi.org/10.3390/electronics10172107},
  DOI={10.3390/electronics10172107},
  number={17},
  journal={Electronics},
  publisher={MDPI AG},
  author={Abeyrathna, Kuruge Darshana and Granmo, Ole-Christoffer and Goodwin, Morten},
  year={2021},
  month={Aug},
  pages={2107}}
@article{zhang2021convergence,
  title = {On the {{Convergence}} of {{Tsetlin Machines}} for the {{IDENTITY}}- and {{NOT Operators}}},
  author = {Zhang, Xuan and Jiao, Lei and Granmo, Ole-Christoffer and Goodwin, Morten},
  year = {2021},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence}
}
@InProceedings{abeyrathna2021parallel,
  title="{Massively Parallel and Asynchronous Tsetlin Machine Architecture Supporting Almost Constant-Time Scaling}",
  author={K. Darshana Abeyrathna and Bimal Bhattarai and Morten Goodwin and Saeed Gorji and Ole-Christoffer Granmo and Lei Jiao and Rupsa Saha and Rohan K. Yadav},
  booktitle={The Thirty-eighth International Conference on Machine Learning (ICML 2021)},
  year={2021},
  organization={ICML}
}
@article{sharma2021dropclause,
  title="{Human Interpretable AI: Enhancing Tsetlin Machine Stochasticity with Drop Clause}",
  author={Jivitesh Sharma and Rohan Yadav and Ole-Christoffer Granmo and Lei Jiao},
  journal = {arXiv preprint arXiv:2105.14506}, year = {2021},
  url = {https://arxiv.org/abs/2105.14506}
}
@article{bhattarai2021fakenews,
  title="{Explainable Tsetlin Machine framework for fake news detection with credibility score assessment}",
  author={Bimal Bhattarai and Ole-Christoffer Granmo and Lei Jiao},
  journal = {arXiv preprint arXiv:2105.09114}, year = {2021},
  url = {https://arxiv.org/abs/2105.09114}
}
@article{bhattarai2021wordlevel,
  title="{Word-level Human Interpretable Scoring Mechanism for Novel Text Detection Using Tsetlin Machines}",
  author={Bimal Bhattarai and Ole-Christoffer Granmo and Lei Jiao},
  journal = {arXiv preprint arXiv:2105.04708}, year = {2021},
  url = {https://arxiv.org/abs/2105.04708}
}
@article{lei2021kws,
  title="{Low-Power Audio Keyword Spotting Using Tsetlin Machines}",
  author={Jie Lei and Tousif Rahman and Rishad Shafik and Adrian Wheeldon and Alex Yakovlev and Ole-Christoffer Granmo and Fahim Kawsar and Akhil Mathur},
  journal = {Journal of Low Power Electronics and Applications}, year = {2021},
  volume=11,
  issue=18,
  url = {https://www.mdpi.com/2079-9268/11/2/18},
  organization={MDPI}
}
@InProceedings{blakely2021closed,
  title="{Closed-Form Expressions for Global and Local Interpretation of Tsetlin Machines}",
  author={Christian D. {Blakely} and Ole-Christoffer {Granmo}},
  booktitle={34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems (IEA/AIE 2021)},
  year={2021},
  organization={Springer}
}
@InProceedings{gorji2021rl,
  title="{Explainable Reinforcement Learning with the Tsetlin Machine}",
  author={Saeed {Gorji} and Ole Christoffer {Granmo} and Marco {Wiering}},
  booktitle={34th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems (IEA/AIE 2021)},
  year={2021},
  organization={Springer}
}
@InProceedings{yadav2021sentiment,
  title="{Human-Level Interpretable Learning for Aspect-Based Sentiment Analysis}",
  author={Rohan Kumar {Yadav} and Lei {Jiao} and Ole-Christoffer {Granmo} and Morten {Goodwin}},
  booktitle={The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21)},
  year={2021},
  organization={AAAI}
}
@InProceedings{nicolae2021question,
  title="{Question Classification using Interpretable Tsetlin Machine}",
  author={Dragos Constantin {Nicolae}},
  booktitle={The 1st International Workshop on Machine Reasoning (MRC 2021)},
  year={2021}
}
@article{saha2021relational,
  title="{A Relational Tsetlin Machine with Applications to Natural Language Understanding}",
  author={Rupsa Saha and Ole-Christoffer Granmo and Vladimir I. Zadorozhny and Morten Goodwin},
  journal = {arXiv preprint arXiv:2102.10952}, year = {2021},
  url = {https://arxiv.org/abs/2102.10952}
}
@InProceedings{yadav2021wordsense,
  title="{Interpretability in Word Sense Disambiguation using Tsetlin Machine}",
  author={Rohan Kumar {Yadav} and Lei {Jiao} and Ole-Christoffer {Granmo} and Morten {Goodwin}},
  booktitle={13th International Conference on Agents and Artificial Intelligence (ICAART 2021)},
  year={2021},
  organization={INSTICC}
}
@InProceedings{bhattarai2021novelty,
  title="{Measuring the Novelty of Natural Language Text Using the Conjunctive Clauses of a Tsetlin Machine Text Classifier}",
  author={Bimal Bhattarai and Lei Jiao and Ole-Christoffer Granmo},
  booktitle={13th International Conference on Agents and Artificial Intelligence (ICAART 2021)},
  year={2021},
  organization={INSTICC}
}
@InProceedings{abeyrathna2021convreg,
  title="{Convolutional Regression Tsetlin Machine}",
  author={Abeyrathna, Kuruge Darshana and Granmo, Ole-Christoffer and Goodwin, Morten},
  booktitle={6th International Conference on Machine Learning Technologies (ICMLT 2021)},
  year={2021},
  organization={ACM}
}
@article{abeyrathna2021integer,
  author = {Abeyrathna, Kuruge Darshana and Granmo, Ole-Christoffer and Goodwin, Morten},
  title = "{Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability}",
  journal = {IEEE Access},
  volume = 9,
  year = {2021}
}
@article{jiao2021xor,
  title="{On the Convergence of Tsetlin Machines for the XOR Operator}",
  author={Lei Jiao and Xuan Zhang and Ole-Christoffer Granmo and K. Darshana Abeyrathna},
  journal = {arXiv preprint arXiv:2101.02547}, year = {2021},
  url = {https://arxiv.org/abs/2101.02547}
}
@InProceedings{wheeldon2021low,
  title="{Low-Latency Asynchronous Logic Design for Inference at the Edge}",
  author={Adrian Wheeldon and Alex Yakovlev and Rishad Shafik and Jordan Morris},
  booktitle={2021 Design, Automation and Test in Europe Conference (DATE21)},
  year={2021},
  pages="370-373"
}
@InProceedings{lei2020arithmetic,
  title="{From Arithmetic to Logic Based AI: A Comparative Analysis of Neural Networks and Tsetlin Machine}",
  author={Jie {Lei} and Adrian {Wheeldon} and Rishad {Shafik} and Alex {Yakovlev} and Ole-Christoffer {Granmo}},
  booktitle={27th IEEE International Conference on Electronics Circuits and Systems (ICECS2020)},
  year={2020},
  organization={IEEE}
}
@InProceedings{abeyrathna2020auc,
  title="{On Obtaining Classification Confidence, Ranked Predictions and AUC with Tsetlin Machines}",
  author={K. Darshana Abeyrathna and Ole-Christoffer Granmo and Morten Goodwin},
  booktitle={2020 IEEE Symposium Series on Computational Intelligence (SSCI)},
  year={2020},
  organization={IEEE}
}
@InProceedings{abeyrathna2020intrusion,
  title="{Intrusion Detection with Interpretable Rules Generated Using the Tsetlin Machine}",
  author={K. Darshana Abeyrathna and Harsha S. Gardiyawasam Pussewalage and Sasanka N. Ranasinghea and Vladimir A. Oleshchuk and Ole-Christoffer Granmo},
  booktitle={2020 IEEE Symposium Series on Computational Intelligence (SSCI)},
  year={2020},
  organization={IEEE}
}
@InProceedings{abeyrathna2020adaptive,
  title="{Adaptive Continuous Feature Binarization for Tsetlin Machines Applied to Forecasting Dengue Incidences in the Philippines}",
  author={K. Darshana Abeyrathna and Ole-Christoffer Granmo and Xuan Zhang and Morten Goodwin},
  booktitle={2020 IEEE Symposium Series on Computational Intelligence (SSCI)},
  year={2020},
  organization={IEEE}
}
@InProceedings{saha2020causal,
  author = {Rupsa {Saha} and Ole-Christoffer {Granmo} and Morten {Goodwin}},
  title = "{Mining Interpretable Rules for Sentiment and Semantic Relation Analysis using Tsetlin Machines}",
  booktitle="Lecture Notes in Computer Science: Proceedings of the 40th International Conference on Innovative Techniques and Applications of Artificial Intelligence (SGAI-2020)", year="2020",
  publisher="Springer International Publishing"
}
@InProceedings{abeyrathna2020deterministic,
  title="{A Novel Multi-Step Finite-State Automaton for Arbitrarily Deterministic Tsetlin Machine Learning}",
  author={K. Darshana Abeyrathna and Ole-Christoffer Granmo and Rishad Shafik and Alex Yakovlev and Adrian Wheeldon and Jie Lei and Morten Goodwin},
  booktitle="Lecture Notes in Computer Science: Proceedings of the 40th International Conference on Innovative Techniques and Applications of Artificial Intelligence (SGAI-2020)", year="2020",
  publisher="Springer International Publishing"
}
@article{wheeldon2020learning, 
  author={Adrian {Wheeldon} and Rishad {Shafik} and Tousif {Rahman} and Jie {Lei} and Alex {Yakovlev} and Ole-Christoffer {Granmo}}, 
  journal={Philosophical Transactions of the Royal Society A},
  title="{Learning Automata based Energy-efficient AI Hardware Design for IoT}",
  year={2020}
}
@InProceedings{shafik2020explainability,
  title="{Explainability and Dependability Analysis of Learning Automata based AI Hardware}",
  author={Rishad {Shafik} and Adrian {Wheeldon} and Alex {Yakovlev}},
  booktitle={IEEE 26th International Symposium on On-Line Testing and Robust System Design (IOLTS)},
  year={2020},
  organization={IEEE}
}
@article{lavrova2020,
  author = {D. S. {Lavrova} and N. N. {Eliseev}},
  title = "{Network Attacks Detection based on Tsetlin Machine}",
  pages = {17-23},
  journal = {Information Security Problems. Computer Systems.}, year = {2020}
}
@InProceedings{gorji2020indexing,
  title="{Increasing the Inference and Learning Speed of Tsetlin Machines with Clause Indexing}",
  author={Saeed {Gorji} and Ole Christoffer {Granmo} and Sondre {Glimsdal} and Jonathan {Edwards} and Morten {Goodwin}},
  booktitle={33rd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems (IEA/AIE 2020)},
  year={2020},
  organization={Springer}
}
@InProceedings{abeyrathna2020integerregression,
  title="{A Regression Tsetlin Machine with Integer Weighted Clauses for Compact Pattern Representation}",
  author={Abeyrathna, Kuruge Darshana and Granmo, Ole-Christoffer and Goodwin, Morten},
  booktitle={33rd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems (IEA/AIE 2020)},
  year={2020},
  organization={Springer}
}
@InProceedings{phoulady2020weighted, 
  author={Adrian {Phoulady} and Ole-Christoffer {Granmo} and Saeed Rahimi {Gorji} and Hady Ahmady {Phoulady}}, 
  booktitle={Proceedings of the Ninth International Workshop on Statistical Relational AI (StarAI 2020)}, 
  title="{The Weighted Tsetlin Machine: Compressed Representations with Clause Weighting}",
  year={2020}
}
@InProceedings{wheeldon2020pervasive, 
  author={Adrian {Wheeldon} and Rishad {Shafik} and Alex {Yakovlev} and Jonathan {Edwards} and Ibrahim {Haddadi} and Ole-Christoffer {Granmo}}, 
  booktitle={SCONA Workshop at Design, Automation and Test in Europe (DATE 2020)}, 
  title="{Tsetlin Machine: A New Paradigm for Pervasive AI}",
  year={2020}
}
@article{abeyrathna2020nonlinear, 
  author={K. Darshana {Abeyrathna} and Ole-Christoffer {Granmo} and Xuan {Zhang} and Lei {Jiao} and Morten {Goodwin}}, 
  journal={Philosophical Transactions of the Royal Society A},
  title="{The Regression Tsetlin Machine - A Novel Approach to Interpretable Non-Linear Regression}",
  volume={378}, issue={2164},
  year={2020}
}
@InProceedings{gorji2019multigranular,
  author = {Saeed Rahimi {Gorji} and Ole-Christoffer {Granmo} and Adrian {Phoulady} and Morten {Goodwin}},
  title = "{A Tsetlin Machine with Multigranular Clauses}",
  booktitle="Lecture Notes in Computer Science: Proceedings of the Thirty-ninth International Conference on Innovative Techniques and Applications of Artificial Intelligence (SGAI-2019)", year="2019",
  volume = {11927},
  publisher="Springer International Publishing"
}
@article{berge2019text, 
  author={Geir Thore {Berge} and Ole-Christoffer {Granmo} and Tor Oddbjørn {Tveit} and Morten {Goodwin} and Lei {Jiao} and Bernt Viggo {Matheussen}}, 
  journal={IEEE Access}, 
  title="{Using the Tsetlin Machine to Learn Human-Interpretable Rules for High-Accuracy Text Categorization with Medical Applications}",
  volume={7},
  pages={115134-115146}, 
  year={2019}, 
  doi={10.1109/ACCESS.2019.2935416}, 
  ISSN={2169-3536}
}
@article{granmo2019convtsetlin,
  author = {{Granmo}, Ole-Christoffer and {Glimsdal}, Sondre and {Jiao}, Lei and {Goodwin}, Morten and {Omlin}, Christian W. and {Berge}, Geir Thore},
  title = "{The Convolutional Tsetlin Machine}",
  journal = {arXiv preprint arXiv:1905.09688}, year = {2019},
  url={https://arxiv.org/abs/1905.09688}
}
@InProceedings{abeyrathna2019regressiontsetlin,
  author = {{Abeyrathna}, Kuruge Darshana and {Granmo}, Ole-Christoffer and {Jiao}, Lei and {Goodwin}, Morten},
  title = "{The Regression Tsetlin Machine: A Tsetlin Machine for Continuous Output Problems}",
  editor="Moura Oliveira, Paulo and Novais, Paulo and Reis, Lu{\'i}s Paulo ",
  booktitle="Progress in Artificial Intelligence", year="2019",
  publisher="Springer International Publishing",
  pages="268--280"
}
@InProceedings{abeyrathna2019continuousinput,
  author = {{Abeyrathna}, Kuruge Darshana and {Granmo}, Ole-Christoffer and {Zhang}, Xuan and {Goodwin}, Morten},
  title = "{A Scheme for Continuous Input to the Tsetlin Machine with Applications to Forecasting Disease Outbreaks}",
  booktitle = "{Advances and Trends in Artificial Intelligence. From Theory to Practice}", year = "2019",
  editor = "Wotawa, Franz and Friedrich, Gerhard and Pill, Ingo and Koitz-Hristov, Roxane and Ali, Moonis",
  publisher = "Springer International Publishing",
  pages = "564--578"
}
@article{granmo2018tsetlin,
  author = {{Granmo}, Ole-Christoffer},
  title = "{The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic}",
  journal = {arXiv preprint arXiv:1804.01508}, year = {2018},
  url={https://arxiv.org/abs/1804.01508}
}

Licence

Copyright (c) 2023 Ole-Christoffer Granmo

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

More Repositories

1

Fire-Detection-Image-Dataset

This dataset contains normal images and images with fire. It is highly unbalanced to reciprocate real world situations. It consists of a variety of scenarios and different fire situations (intensity, luminosity, size, environment etc).
208
star
2

deep-rts

A Real-Time-Strategy game for Deep Learning research
C++
202
star
3

pyTsetlinMachine

Implements the Tsetlin Machine, Convolutional Tsetlin Machine, Regression Tsetlin Machine, Weighted Tsetlin Machine, and Embedding Tsetlin Machine, with support for continuous features, multigranularity, clause indexing, and literal budget
C
120
star
4

tmu

Implements the Tsetlin Machine, Coalesced Tsetlin Machine, Convolutional Tsetlin Machine, Regression Tsetlin Machine, and Weighted Tsetlin Machine, with support for continuous features, drop clause, Type III Feedback, focused negative sampling, multi-task classifier, autoencoder, literal budget, and one-vs-one multi-class classifier. TMU is written in Python with wrappers for C and CUDA-based clause evaluation and updating.
Python
106
star
5

pyVNC

VNC Client Library for Python
Python
82
star
6

fast-tsetlin-machine-with-mnist-demo

A fast Tsetlin Machine implementation employing bit-wise operators, with MNIST demo.
C
61
star
7

convolutional-tsetlin-machine-tutorial

Tutorial on the Convolutional Tsetlin Machine
Python
51
star
8

TextUnderstandingTsetlinMachine

Using the Tsetlin Machine to learn human-interpretable rules for high-accuracy text categorization with medical applications
Cuda
48
star
9

PyTsetlinMachineCUDA

Massively Parallel and Asynchronous Architecture for Logic-based AI
Python
41
star
10

pyTsetlinMachineParallel

Multi-threaded implementation of the Tsetlin Machine, Convolutional Tsetlin Machine, Regression Tsetlin Machine, and Weighted Tsetlin Machine, with support for continuous features and multigranularity.
C
39
star
11

TsetlinMachineBook

Python code accompanying the book "An Introduction to Tsetlin Machines".
Jupyter Notebook
31
star
12

FlashRL

Python
26
star
13

fast-tsetlin-machine-in-cuda-with-imdb-demo

A CUDA implementation of the Tsetlin Machine based on bitwise operators
Cuda
26
star
14

deep_maze

Python
22
star
15

open-tsetlin-machine

Open Source Tsetlin Machine framework
17
star
16

TsetlinMachineC

A C implementation of the Tsetlin Machine
C
14
star
17

rl

C++
10
star
18

awesome-tsetlin-machine

A curated list of Tsetlin Machine research
10
star
19

regression-tsetlin-machine

Implementation of the Regression Tsetlin Machine
Python
9
star
20

deep-warehouse

A Simulator for complex logistic environments
Python
7
star
21

TM-XOR-proof

#tsetlin-machine #machine-learning #game-theory #propositional-logic #pattern-recognition #bandit-learning #frequent-pattern-mining #learning-automata
Cython
5
star
22

Axis_and_Allies

A simple Axis & Allies engine.
Python
5
star
23

python-fast-tsetlin-machine

Python wrapper for https://github.com/cair/fast-tsetlin-machine-with-mnist-demo
C
3
star
24

tmu-datasets

A dataset repository for datasets in tmu
Python
3
star
25

ICML-Massively-Parallel-and-Asynchronous-Tsetlin-Machine-Architecture

Code repository for ICML 21 for Paper titled Massively Parallel and Asynchronous Tsetlin Machine Architecture
Python
3
star
26

ikt111

Python
3
star
27

notebooks

A collection of jupyter notebooks
Jupyter Notebook
2
star
28

Fire-Scene-Parsing

2
star
29

py_image_stitcher

A small library for stitching together images, from Numpy or PIL Sources
Python
1
star
30

deep-line-wars

Python
1
star
31

Docker-Tutorial

A docker tutorial for cair-gpu's
Python
1
star
32

ray-bugfix

A workaround to issues with Rllib, given it does not work for your current gym environment. CarRacing-v0 is one of these.
Python
1
star
33

fire

Python
1
star
34

deep-line-wars-2

C++
1
star
35

Deterministic-Tsetlin-Machine

Due to the high energy consumption and scalability challenges of deep learning, there is a critical need to shift research focus towards dealing with energy consumption constraints. Tsetlin Machines (TMs) are a recent approach to machine learning that has demonstrated significantly reduced energy usage compared to neural networks alike, while performing competitively accuracy-wise on several benchmarks. However, TMs rely heavily on energy-costly random number generation to stochastically guide a team of Tsetlin Automata to a Nash Equilibrium of the TM game. In this paper, we propose a novel finite-state learning automaton that can replace the Tsetlin Automata in TM learning, for increased determinism. The new automaton uses multi-step deterministic state jumps to reinforce sub-patterns. Simultaneously, flipping a coin to skip every d'th state update ensures diversification by randomization. The d-parameter thus allows the degree of randomization to be finely controlled. E.g., d=1 makes every update random and d=infinity makes the automaton completely deterministic. Our empirical results show that, overall, only substantial degrees of determinism reduces accuracy. Energy-wise, random number generation constitutes switching energy consumption of the TM, saving up to 11 mW power for larger datasets with high d values. We can thus use the new d-parameter to trade off accuracy against energy consumption, to facilitate low-energy machine learning.
Python
1
star
36

DeepAxie

Implementation of a simplified Axie Infinity Environment in C++ that is used to train an agent with the reinforcement learning algorithm DQN to play the game.
C++
1
star
37

Tsetlin-Machine-Deep-Neural-Network-Recommendation-System-Comparison

Jupyter Notebook
1
star
38

LogicalTransformerWithTsetlinMachine

Python
1
star