We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic.
By clicking "I accept", you consent to our use of cookies.

Creating your own LLM from opensource models

Riproduci il video

Evento:
AI Conf 2024
Lingua:
Inglese

Tag

  • AI Generativa
  • Artificial Intelligence
  • LLM

Speaker

Creating your own LLM from opensource models

From a "simple" finetuning to your own Mixture of Expert model using opensource models.
Nowadays training from scratch an LLM is a so huge effort also for very big company. Starting from pre-trained models to create your own model is no more a way for resourceless companies, but a often a must starting point.

  • Lora
  • Quantization and QLora
  • Injecting embeddinds model into Lora to manage multiple Lora adapters.
  • Mixing models
  • Creating your MoE (Mixture of experts) model using several finetuned (Your own) models