# LLM Evaluation

## LLM IEvaluation

###  1. Model Details:

a.   Model Name: \[LLM full Name]

b.   Creators: \[Organization/Individual(s) Responsible]

c.   Architecture and Type: \[Generative Pre-trained Transformer (GPT), Masked Language Model (MLM), Encoder only, Decoder Only, Both Encoder and Decoder etc.]

d.   Number of Experts: \[If Mixture of Experts (MoE) used, example 8x etc.]

e.   Modality:  \[Multimodal, Audio, Video, Text etc.]

f.    Model Size: \[Number of Parameters, example 70B, 300B etc.]

g.   Year Launched: \[Year of Release]

h.   Access: \[Open-source, Closed-source, etc.]

i.    License: \[Specific License Type]

 

### 2. Hardware and Software:

a.   Training Hardware: \[How much is Used for Training (Number of TPUs, GPUs, etc.)]

b.   Inference Hardware: \[Hardware Recommended for Running the Model]

c.   Software Frameworks: \[Libraries or Frameworks Used (TensorFlow, PyTorch, etc.)]

 

### 3. Training Datasets:

a.      Dataset (s) Name: \[Names of the Datasets Used for Training]

b.     Type of Data: \[Text, Code, Video, Audio, etc.]

c.      Size of Dataset: \[In Terabytes (TB), Petabytes (PB), Exabytes (EB), Zettabytes (ZB) etc.]

d.     Number of Tokens: \[Number of Tokens it has been trained on]

 

### 4. Intended Use:

 

### 5. Number of Attention Heads: 

 

### 6. Size Context Window:  

 

### 7. Training Cost per Token:

 

### 8. Total Training Cost: \[No. of Tokens x Training Cost per Token]

 

### 9. Inference Cost per Token: 

 

### 10. Ethical Considerations and Limitations:

a.   Potential Biases: \[Known Biases in the Model]

b.   Limitations: \[Capabilities and Limitations of the Model]

c.   Safety Measures: \[Measures Implemented to Mitigate Risks]

 

### 11. Fine-tuned Models: \[List of Available Fine-tuned Models with Specific Use Cases]

 

### 12. Additional Information:

a.   Instructions on How to Report Issues with the Model

b.   Links to Relevant Resources \[Official Website, Documentation, Research Paper, etc.]