# Codex P\@1 (0-Shot)

**Codex P\@1 (0-Shot)** 

Codex P\@1 (0-Shot) is a metric used to evaluate the performance of large language models (LLMs) in code generation. It measures the percentage of times that the LLM generates the correct code for a given prompt, without any prior training on that specific prompt.

The metric is calculated by evaluating the LLM on a set of code generation prompts. For each prompt, the LLM is given a natural language description of the desired code output. The LLM then generates the code, and the output is compared to the ground truth code. If the LLM's output matches the ground truth code, then the prompt is considered to be correct.

The P\@1 score is calculated by dividing the number of correctly generated code outputs by the total number of prompts.

Codex P\@1 (0-Shot) is a useful metric for evaluating the ability of LLMs to generate code without any prior training. It is a challenging task, as it requires the LLM to understand the natural language description of the code, and to generate the correct code syntax.

**Here is an example of how Codex P\@1 (0-Shot) might be used:**

**Prompt:** Write a function in Python that reverses a string.

**Ground truth code:**

```python
Copy codedef reverse_string(string):
  reversed_string = ""
  for i in range(len(string)-1, -1, -1):
    reversed_string += string[i]
  return reversed_string
```

**LLM output:**

```python
Copy codedef reverse_string(string):
  reversed_string = ""
  for i in range(len(string)-1, -1, -1):
    reversed_string += string[i]
  return reversed_string
```

**Codex P\@1 score:** 100% (correct)

In this example, the LLM generated the correct code for the given prompt, without any prior training. Therefore, the Codex P\@1 (0-Shot) score is 100%.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://metaverse-imagen.gitbook.io/ai-tools-research/about-ai-tools-research/llm-performance-benchmarks/llm-benchmarks-and-tasks/codex-p-1-0-shot.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.