Toward understanding and preventing misalignment generalization

Company news OpenAI Source language: English

We study how training on incorrect responses can cause broader misalignment in language models and identify an internal feature driving this behavior—one that can be reversed with minimal fine-tuning.

Excerpt

We study how training on incorrect responses can cause broader misalignment in language models and identify an internal feature driving this behavior—one that can be reversed with minimal fine-tuning.

Short excerpt from the source — open the original for the full text.

Sources

← Back to the daily brief