# Mastering LoRA 📽 🎥\ \ [LoRA](https://www.linkedin.com/feed/hashtag/?keywords=lora\&highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7205578116985929728): Low-Rank Adaptation of Large Language Models is a groundbreaking technique developed by Microsoft researchers to address the challenge of fine-tuning large language models.\ \ Models with billions of parameters, like [GPT](https://www.linkedin.com/feed/hashtag/?keywords=gpt\&highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7205578116985929728)-3, are incredibly powerful but come with a hefty price tag when it comes to fine-tuning for specific tasks or domains.\ \ LoRA offers an elegant solution by freezing the pre-trained model weights and introducing trainable layers, known as rank-decomposition matrices, into each transformer block.\ \ Speed and Efficiency Advantages\ \ Reduced Training Time:\ LoRA significantly decreases the number of trainable parameters by introducing low-rank decomposition matrices into the layers of the model. This reduction allows for faster training compared to standard fine-tuning, which requires updating all the parameters in the model. For instance, LoRA was found to outperform full fine-tuning while using a fraction of the training time and memory resources​ ()​.\ \ Memory Efficiency:\ By freezing the original pre-trained model weights and only updating the low-rank matrices, LoRA drastically reduces the memory requirements. This approach allows training on smaller, less expensive hardware configurations, making it more accessible and cost-effective​​.\ \ Watch this video to find out all about LoRA. How it works and how you can use it to dramatically reduce the cost and time of developing your own Fine Tuned [LLMs](https://www.linkedin.com/feed/hashtag/?keywords=llms\&highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7205578116985929728) for [GenAI](https://www.linkedin.com/feed/hashtag/?keywords=genai\&highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7205578116985929728) {% embed url="" %} {% endembed %} --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://metaverse-imagen.gitbook.io/ai-tools-research/ai-technology/generative-ai-architectures-and-models/lora-low-rank-adaptation-of-large-language-models/mastering-lora.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.