As more open-source LLM models are released, we want to test out these models for our use cases. The hardware requirements to run and finetune such a large model are often a limiting factor.
We will see how to build prompts for training and different options like distillation, quantization, and low-rank adaption to reduce memory usage during training and prediction.
These are the keys to loading these models on consumer hardware. Even Finetuning free Google Colab-Notebooks is possible.
This session will explain how this works and how you can finetune your LLM.