Modern neural networks [1–4] demonstrate outstanding results in computer vision, natural language and speech processing [5], but this is accompanied by a significant increase in the number of parameters and computational complexity of the models. These characteristics significantly complicate the application of deep neural networks in environments with limited computational resources, such as mobile devices, embedded systems, and edge platforms. The increasing computational complexity of models and the need for rational use of resources are also emphasized in modern review papers on optimization and modeling, where the importance of choosing and combining optimization methods taking into account computational limitations is noted [6]. In this context, model compression methods [7–10], and quantization in particular, are one of the key tools for ensuring the practical applicability of modern architectures. Quantization [11–14] involves representing real-valued model parameters as numbers with reduced bit depth, which allows for a significant reduction in memory footprint and acceleration of computations through the use of integer operations. In practice, methods of uniform quantization of weights and activations with a fixed or per-channel scale are widely used. Despite their simplicity and computational efficiency, such approaches often lead to a noticeable degradation of model quality, especially when using low bit depth, for example, 4 or 3 bits. Most existing post-training quantization methods formulate the problem as minimizing some measure of distortion between the original and quantized weights, most often the mean squared error. However, this criterion only considers the magnitude of the error, completely ignoring its distribution. Meanwhile, the statistical structure of the quantization error can have a significant impact on how this error propagates through the network and is reflected in the final model output. From an information-theoretical perspective, quantization is naturally viewed as a tradeoff between the accuracy of approximation and the complexity of error representation, traditionally described within the rate-distortion paradigm. In this context, the entropy of the quantization error is an important characteristic, reflecting the degree of its uncertainty and the potential complexity of compensation. Nevertheless, the explicit use of entropy criteria in practical post-training quantization algorithms remains relatively understudied. This paper proposes the DARE-Q (Distribution-Aware Residual Entropy Quantization) method, which extends the standard quantization problem formulation by introducing entropy regularization of the quantization residual. The method is based on channel-by-channel selection of the quantization scale, optimized by a combined loss function that includes both the quadratic distortion of the weights and the entropy of the error distribution. This approach not only reduces the magnitude of the quantization error but also produces a more structured and predictable residual distribution. The main advantages of the proposed method are its simplicity, the absence of the need for additional model training, and compatibility with a wide range of neural network architectures.

The main contributions of this work are as follows. First, we propose DARE-Q, a fully data-free post-training quantization method that introduces entropy regularization of the quantization residual as an explicit optimization criterion, enabling principled control over the statistical structure of the quantization error. Second, we develop the DAREQuantLinear module—a plug-and-play implementation that integrates seamlessly into standard transformer pipelines without modifications to the inference logic or specialized kernels. Third, we provide experimental evidence that DARE-Q delivers practical 4-bit weight compression on OPT-class language models in scenarios where data-dependent methods such as GPTQ and AWQ are fundamentally inapplicable. The remainder of the paper is organized as follows: Section 2 reviews related work, Section 3 describes the method, Section 4 presents experiments, Section 5 discusses the results, and Section 6 concludes.

Neural network quantization is one of the most actively developing areas in the field of model compression and inference acceleration. Depending on the degree of integration into the training process, existing methods are typically divided into quantization-aware training (QAT) and post-training quantization (PTQ). QAT methods [15–17] integrate quantization operations directly into the training process, allowing model parameters to adapt to limited bit depth. Despite high quality at low bit depth, such approaches require retraining the model and, as a rule, access to the training data, which significantly limits their applicability to large pre-trained models. Post-training quantization methods [18–22] quantize an already trained model without additional training. Basic PTQ approaches use uniform symmetric or asymmetric quantization with scaling based on simple weight distribution statistics, such as maximum absolute value or quantile estimates. Despite their computational efficiency, such methods often exhibit significant quality degradation at low bit depths, especially in the presence of outliers. To reduce quantization error, methods optimizing quantization parameters using the mean square distortion criterion have been proposed. This category includes approaches with channel-by-channel quantization, as well as methods searching for the optimal scale to minimize the L2 error between the original and quantized weights. However, such methods still only consider the magnitude of the error, completely ignoring its distribution. In recent years, more advanced PTQ methods have been proposed that take into account the effect of weight quantization on the outputs of layers or the model as a whole. For example, the GPTQ method (and those based on it) [23–26] formulates the quantization problem as weight optimization taking into account the second-order approximation of the loss function, which allows for taking into account correlations between parameters and significantly reducing the error in the layer outputs. Similarly, a number of methods minimize the error in activations or output layer representations by using calibration data to fine-tune the quantization parameters. Another area of research involves methods that analyze activation and weight statistics to improve the robustness of quantization. For example, AWQ (activation-aware weight quantization) [27,28] uses information about the distribution of activations to identify the most sensitive weights and adapt the quantization parameters accordingly. Such approaches improve the quality of quantization without completely retraining the model; however, they require access to calibration data and complicate the preliminary analysis procedure. A separate line of research is concerned with the application of information theory ideas to neural network quantization and compression problems. In this work, quantization is considered as a tradeoff between approximation accuracy and representation complexity, often formalized through the rate–distortion paradigm.

This section presents the DARE-Q method in detail. We begin by formalizing the quantization problem and the desired properties of the quantization error (Section 3.1). We then describe the symmetric uniform quantization scheme and the motivation for choosing it (Section 3.2), followed by the per-channel quantization strategy (Section 3.3). The central contribution—the combined loss function based on L2 distortion and residual entropy—is introduced in Sections 3.4–3.6, and the full quantization algorithm is given in Section 3.7. Finally, Section 3.8 describes how the quantized weights are used during inference. Throughout, all optimization steps rely exclusively on the weight statistics of the already-trained model, without any access to calibration data or model outputs.

This section presents a preliminary experimental evaluation of the proposed DARE-Q method. The main goal of the experiments is to verify the feasibility of the approach and to analyze its behavior under low-bit post-training quantization without the use of calibration data.

The experimental results presented in the previous section establish the empirical baseline for DARE-Q under strict data-free constraints. This section situates those results within the broader landscape of quantization research, elaborates on the method’s design rationale, identifies its current limitations, and charts directions for future work.

The proposed DARE-Q method offers an alternative perspective on the problem of post-training weight quantization in neural networks. Unlike most existing approaches that focus exclusively on minimizing the magnitude of quantization error (e.g., L2 distortion) or preserving layer outputs, DARE-Q explicitly accounts for the statistical structure of the quantization error through the entropy of the residual.

One of the key advantages of the method is the absence of any requirement for calibration data or access to the training set. This makes DARE-Q practically applicable to the quantization of large pretrained models, including language models and transformers, where data access may be limited or infeasible. In this respect, the method favorably differs from activation- or output-oriented approaches such as GPTQ and AWQ.

Per-channel scale optimization enables adaptation to heterogeneous weight distributions across individual neurons, which is particularly important in modern models with pronounced parameter non-uniformity. The inclusion of an entropy term in the loss function encourages the formation of a more “structured” quantization error, potentially less destructive for subsequent network layers. This can be interpreted as an implicit regularization of the error, consistent with information-theoretic principles and the accuracy–complexity trade-off.

At the same time, the method has several limitations. First, scale selection is performed via grid search, which increases the computational complexity of the quantization procedure compared to simple heuristic methods. Although this step is executed only once in an offline setting, it may become a bottleneck for very large models. A potential direction for improvement is the use of more efficient scale optimization techniques, such as adaptive search or gradient-based approximations.

Second, the entropy of the residual is estimated using histograms, which introduces additional hyperparameters, including the number of bins and the weighting coefficients of the loss components. The choice of these parameters may affect quantization quality and requires further empirical investigation.

Finally, in its current formulation, the method is restricted to weight quantization of linear layers. Extending DARE-Q to joint quantization of weights and activations, as well as adapting it to other layer types, constitutes a natural direction for future work.

In this work, we proposed DARE-Q (Distribution-Aware Residual Entropy Quantization), a post-training per-channel weight quantization method based on the joint optimization of mean squared error and the entropy of the quantization residual. Unlike traditional approaches, DARE-Q accounts not only for the magnitude of weight distortion but also for the statistical properties of the resulting error.

The method does not require model retraining or calibration data and can be applied to already trained neural networks, making it particularly attractive for practical scenarios with limited resources. By leveraging per-channel scale selection and a combined loss function, DARE-Q provides a flexible trade-off between approximation accuracy and quantization error structure.

The proposed approach opens new opportunities for incorporating information-theoretic criteria into neural network quantization and can be viewed as a step toward more deliberate control of error distributions in low-bit parameter representations. Future work will focus on experimental evaluation across different architectures and on extending the method to more advanced quantization schemes.