# RACE-H (5-Shot)

RACE-H (5-Shot) is a benchmark dataset used to evaluate the performance of large language models (LLMs) on natural language reasoning tasks. It is a more challenging dataset than the original RACE dataset, as it requires the LLM to reason over multiple sentences and to learn from a smaller number of training examples (5 shots).

In LLM comparisons, RACE-H (5-Shot) is often used as a way to measure the LLM's ability to perform complex reasoning tasks. A higher score on RACE-H (5-Shot) indicates that the LLM is better at understanding and reasoning about language.

Here are some examples of tasks that a LLM might be asked to perform on RACE-H (5-Shot):

* Given a passage of text, identify the main idea or purpose of the text.
* Given two passages of text, determine whether they are compatible or contradictory.
* Given a passage of text and a question, answer the question correctly.

LLMs with high scores on RACE-H (5-Shot) are generally considered to be more capable and powerful than LLMs with lower scores. This is because the ability to perform complex reasoning tasks is a key requirement for many natural language processing applications, such as machine translation, question answering, and text summarization.

Here are some of the top performing LLMs on RACE-H (5-Shot):

* PaLM
* Megatron-Turing NLG
* Wu Dao 2.0
* GPT-3
* Jurassic-1 Jumbo

As you can see, RACE-H (5-Shot) is a challenging benchmark dataset that can be used to evaluate the performance of LLMs on complex reasoning tasks. It is an important tool for researchers and developers who are working on improving the capabilities of LLMs.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://metaverse-imagen.gitbook.io/ai-tools-research/about-ai-tools-research/llm-performance-benchmarks/llm-benchmarks-and-tasks/race-h-5-shot.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.