Topic: inference time

Researchers have compared different methods of expert deactivation during inference to understand their impact on model performance, focusing on token routing and weight adjustments. This study is crucial for optimizing the efficiency and effectiveness of large language models without compromising accuracy.

arxiv.org↗