HSE Scientists Develop Method to Compress Large Language Models Without Losing Quality

© iStock

Researchers from the AI and Digital Science Institute at the HSE Faculty of Computer Science have developed a new compression method for large language models such as GPT and LLaMA that reduces their size by 25–36% without additional training or significant loss of accuracy. This is the first approach to use mathematical transformations—specifically, rotations of model weights—to make models more amenable to compression with structured matrices. The study results have been published in ACL Findings 2025. The code is available on GitHub.

Large language models such as ChatGPT and LLaMA demonstrate impressive results in text generation, translation, and other tasks, but their enormous size makes them costly to deploy and store. Traditional compression methods—such as reducing numerical precision, pruning redundant connections, or simplifying the architecture—often require time-consuming retraining and can degrade performance. Scientists sought a way to shrink model size quickly without compromising its intelligence.

Researchers from the Laboratory for Matrix and Tensor Methods in Machine Learning at the HSE FCS AI and Digital Science Institute proposed a method called ProcrustesGPT, based on the idea that a model’s output remains unchanged if special orthogonal transformations are applied to its internal weights—a kind of mathematical rotation. As the scientists explain, this is a transformation of space that can rotate or flip an image in any way but cannot stretch or compress any object. For example, if you take a piece of paper with a triangle drawn on it, you can flip or rotate it at any angle—the side lengths and the angles between them will remain exactly the same. In mathematics, such a transformation is called orthogonal. These transformations are chosen so that the model’s weights can be compressed more efficiently using structured matrices—mathematical constructions that require far less memory.  

Ekaterina Grishina

Ekaterina Grishina, Research Assistant at the Laboratory for Matrix and Tensor Methods in Machine Learning, explains, 'Our work is based on an elegant mathematical concept—the Procrustes problem. Like the mythical figure Procrustes, who forced travellers to fit his bed, this method helps identify the optimal orthogonal transformation that reshapes the model’s weights into a simpler structure without distorting their essence. This idea inspired the name of our method, ProcrustesGPT, and became the key to achieving compression without significant loss of quality.'

As part of the study, the researchers tested two types of such structures: sums of Kronecker products and GS matrices. The method does not require additional training, works quickly, and can be applied to existing models. The experiments were conducted on the open OPT and LLaMA 2 models.

The new ProcrustesGPT method has demonstrated strong effectiveness: it reduces the size of large language models by about a third—more precisely, by 25–36% of their original size—while preserving their capabilities. The compressed models deliver results close to those of the originals, retaining 90–95% of their initial performance in generating coherent text and solving logical tasks.

Compared with other modern compression methods, such as SliceGPT—which also does not require lengthy additional training—ProcrustesGPT proved more accurate in most tests. This advantage is particularly evident for models in the LLaMA 2 family, where the proposed approach outperforms its counterpart by 9–10%.

Maxim Rakhuba

According to Maxim Rakhuba, Head of the Laboratory for Matrix and Tensor Methods in Machine Learning at the HSE AI and Digital Science Institute, 'Compression methods help accelerate the deployment of large language models on resource-constrained devices, such as mobile devices and IoT gadgets, making AI more accessible and widely integrated into everyday life.'