# LLM Evaluation

## LLM IEvaluation

###  1. Model Details:

a.   Model Name: \[LLM full Name]

b.   Creators: \[Organization/Individual(s) Responsible]

c.   Architecture and Type: \[Generative Pre-trained Transformer (GPT), Masked Language Model (MLM), Encoder only, Decoder Only, Both Encoder and Decoder etc.]

d.   Number of Experts: \[If Mixture of Experts (MoE) used, example 8x etc.]

e.   Modality:  \[Multimodal, Audio, Video, Text etc.]

f.    Model Size: \[Number of Parameters, example 70B, 300B etc.]

g.   Year Launched: \[Year of Release]

h.   Access: \[Open-source, Closed-source, etc.]

i.    License: \[Specific License Type]

 

### 2. Hardware and Software:

a.   Training Hardware: \[How much is Used for Training (Number of TPUs, GPUs, etc.)]

b.   Inference Hardware: \[Hardware Recommended for Running the Model]

c.   Software Frameworks: \[Libraries or Frameworks Used (TensorFlow, PyTorch, etc.)]

 

### 3. Training Datasets:

a.      Dataset (s) Name: \[Names of the Datasets Used for Training]

b.     Type of Data: \[Text, Code, Video, Audio, etc.]

c.      Size of Dataset: \[In Terabytes (TB), Petabytes (PB), Exabytes (EB), Zettabytes (ZB) etc.]

d.     Number of Tokens: \[Number of Tokens it has been trained on]

 

### 4. Intended Use:

 

### 5. Number of Attention Heads: 

 

### 6. Size Context Window:  

 

### 7. Training Cost per Token:

 

### 8. Total Training Cost: \[No. of Tokens x Training Cost per Token]

 

### 9. Inference Cost per Token: 

 

### 10. Ethical Considerations and Limitations:

a.   Potential Biases: \[Known Biases in the Model]

b.   Limitations: \[Capabilities and Limitations of the Model]

c.   Safety Measures: \[Measures Implemented to Mitigate Risks]

 

### 11. Fine-tuned Models: \[List of Available Fine-tuned Models with Specific Use Cases]

 

### 12. Additional Information:

a.   Instructions on How to Report Issues with the Model

b.   Links to Relevant Resources \[Official Website, Documentation, Research Paper, etc.]


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://metaverse-imagen.gitbook.io/ai-tools-research/about-ai-tools-research/frequently-asked-questions-faqs/faqs-on-llm-training-and-data-labelling/llm-evaluation.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.