Instruction Induction

# Instruction Induction: Natural language hypothesis optimization

**Date**: May 22, 2022  
**Institution**: Tel Aviv University, NYU

## Algorithm

Instruction Induction tackled the problem of task inference from demonstrations, enabling language models to generate natural language task descriptions from input-output examples. The two-stage process involved instruction generation from demonstrations followed by instruction execution on test cases, using both forward and reverse instruction generation strategies.

Algorithmic innovation: Established instruction induction as a learning paradigm with systematic dataset creation (24 tasks) for evaluating instruction generation. InstructGPT achieved 65.7% of human performance vs. 9.8% for GPT-3, demonstrating the critical importance of instruction alignment in optimization.

Core methodology: LLMs prompted to generate natural language task descriptions with execution-based evaluation metrics, showing emergent instruction generation capability in large aligned models.

## Comparison with Other Work

Instruction Induction differs from AutoPrompt and PET which focus on optimizing model behavior for specific tasks, instead focusing on generating natural language instructions from examples. Unlike continuous methods like Prefix-Tuning and Prompt Tuning, it works with natural language rather than continuous embeddings. It's a precursor to APE and OPRO which also use LLMs as optimizers but focuses specifically on instruction generation rather than general prompt optimization. Compared to reinforcement learning methods like RLPrompt, it uses a more direct generation approach. The method differs from DSPy which uses a programming paradigm rather than direct instruction generation.