# Getting starting with transformers

If you want to know more about the transformers architecture, have a look at:
- [The illustrated transformers](http://jalammar.github.io/illustrated-transformer/)
- [The annotated transformers](https://nlp.seas.harvard.edu/2018/04/03/attention.html?s=09)

## Bio-transformers

If you want to understand the secret of how to train deep-learning model on protein, the two best repo are:
- [ProtTrans repo](https://github.com/agemagician/ProtTrans)
- [ESM repo](https://github.com/facebookresearch/esm)