# GSM8k (0-Shot CoT) GSM8k (0-Shot CoT) in LLM comparisons refers to the performance of an LLM on the GSM8k benchmark using zero-shot **Chain of Thought (CoT)** prompting. GSM8k is a large-scale arithmetic reasoning benchmark that consists of 8,000 problems, including addition, subtraction, multiplication, division, and more complex operations. Zero-shot CoT prompting is a technique for eliciting complex multi-step reasoning from LLMs by simply adding the prompt "Let's think step by step" before each answer. GSM8k (0-Shot CoT) is a useful metric for comparing the zero-shot reasoning capabilities of different LLMs. It is also a useful metric for evaluating the effectiveness of different CoT prompting techniques. In the paper "Large Language Models are Zero-Shot Reasoners," the authors evaluated the performance of several LLMs on GSM8k (0-Shot CoT). They found that LLMs with zero-shot CoT prompting significantly outperformed LLMs without CoT prompting. For example, the InstructGPT model achieved an accuracy of 40.7% on GSM8k (0-Shot CoT), compared to 10.4% without CoT prompting. The authors also found that the performance of LLMs on GSM8k (0-Shot CoT) increased with their size. This suggests that larger LLMs have better zero-shot reasoning capabilities. Overall, GSM8k (0-Shot CoT) is a useful metric for comparing the zero-shot reasoning capabilities of different LLMs and for evaluating the effectiveness of different CoT prompting techniques. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://metaverse-imagen.gitbook.io/ai-tools-research/about-ai-tools-research/llm-performance-benchmarks/llm-benchmarks-and-tasks/gsm8k-0-shot-cot.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.