Topic: memorisation

A recent study proposes pruning redundant "attention heads" in large language models can reduce their tendency to memorize and reproduce exact sequences from training data, addressing privacy and intellectual property concerns.

arxiv.org↗