An open collection of methodologies to help with successful training of large language models.

This is technical material suitable for LLM training engineers and operators. That is the content here contains lots of scripts and copy-n-paste commands to enable you to quickly solve your problems.

If you are not interested in technical details but want more of a detailed overview and concepts please refer to the sister The Large Language Model Training Playbook instead.

note: The list of topics will expand over time - at the moment filling in only a subset

The content of this site is distributed under Attribution-ShareAlike 4.0 International.

Unless specified otherwise the code in this repo is licensed under Apache License, Version 2.0.