# How to Build a LLM from Scratch

Open AI’s release of ChatGPT in late 2002 ushered in the ‘Era of Generative Artificial Intelligence”. ChatGPT introducing the amazing powers of AI to the public. Now we have an environment where many Businesses enterprises and other Organizations are seeking to adopt Gen-AI and train their their own Large Language Models in order to remain competitive.

One of the most notable examples of this trend is Bloomberg GPT, which is a Large Language Model that was specifically built by Bloomberg to handle tasks in the Finance domain.

Even though building a Large Language Model from scratch is often not necessary for a vast majority of use cases since fine-tuning an existing LLM is relatively quick and inexpensive, it is still valuable to understand what it takes to build one since all use cases are not addressed by existing LLM’s. Moreover, some organization may have strategies to for competitive and proprietary assets.

In this section, we’ll discuss the key aspects, considerations and pro’s and con’s for building a Large Language Model from scratch.

Here are the steps we will cover:

1. [**Estimate of Cost (Rent Compute Hardware vs. Purchase Compute Hardware)**](/ai-tools-research/about-ai-tools-research/articles-and-transcripts/how-to-build-a-llm-from-scratch/1.-evaluate-financial-costs-involved.md)
2. [**Data Curation**](/ai-tools-research/about-ai-tools-research/articles-and-transcripts/how-to-build-a-llm-from-scratch/2.-data-curation.md) 
3. [**Model Architecture**](/ai-tools-research/about-ai-tools-research/articles-and-transcripts/how-to-build-a-llm-from-scratch/3.-model-architecture.md) 
4. [**Training the Model at Scale** ](/ai-tools-research/about-ai-tools-research/articles-and-transcripts/how-to-build-a-llm-from-scratch/4.-training-the-model-at-scale.md)
5. [**Evaluating the Model**](/ai-tools-research/about-ai-tools-research/articles-and-transcripts/how-to-build-a-llm-from-scratch/5.-evaluating-the-model.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://metaverse-imagen.gitbook.io/ai-tools-research/about-ai-tools-research/articles-and-transcripts/how-to-build-a-llm-from-scratch.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.