Automated ML (AutoML) has been used widely to automate the entire process of building AI system pipelines, including collecting data, data preprocessing, feature engineering, feature selection, meta-learning, hyperparameter searching, and neural network architecture searching without requiring a deep knowledge of ML or any human intervention. During the hyperparameter and neural network architecture searching using meta-learning, it may use various parameters to find an appropriate model for the given task, likewise fine-tuning NN models [20,21]. Since it is so popular, many ML frameworks have developed their AutoML versions, including Autosklearn, AutoTensorflow, and AutoPyTorch. In the case of an instance-level assessment (the paper’s primary focus), we also leverage similar parameters to constitute the emergent behavior of AI systems with other related factors to this process.

The most significant difference between them is the assessor model aims to assess before employing an AI system on each task. Suppose we know how an AI system behaves for every job. In that case, we can apply this result to a range of AI problems such as combining models, explaining AI systems, fixing outputs (the main objective of this paper), and auditing or certifying [7]. Moreover, AutoML does not satisfy all properties of the assessor model listed above except distributional¹1

Nevertheless, AutoML searches several AI models and evaluates them based on some metrics according to the considered tasks. It usually suggests one of these models while the assessor models evaluate them based on each instance separately. If the task is selecting a system, the assessor selects a system for each instance separately based on success rate. In contrast, AutoML chooses only one system for generalization metrics, such as accuracy or loss on test sets.

One way to explain AI systems is to derive their IRT difficulties developed [13] supported by regression examples for a broad range of ML tasks, including adversarial examples and out-of-distribution samples. Authors acquire 10–20 responses for each item from models to estimate the difficulties by training a meta-learning model on the responses. In this case, the “difficulty estimator” model is not an assessor model that holds the above properties. Similarly, IRT responses are leveraged to build a weighted majority voting framework based on the hardness of instances in ensembles by measuring how close the objects are to the boundary [19,22]. Another analysis of AI models at the granular level based on IRT explored ML classifiers to estimate their behavior [13]. To explain these classifiers, the authors combined latent variables of items and responses by models. Most of this research has benefited from the two-parameter logistic model to identify instance abilities and difficulties as a function of probabilities as follows: Pr(Uij=1|θj)=11+e−ai(θj−bj),where Uij, ai, and bj are binary response of respondent i to question j, a slope, and a difficulty accordingly. These parameters facilitate explaining the model abilities and instance hardness (difficulty). Our main aim in this work is to train the assessor model to fix models’ outputs. However, we also used similar responses to construct a dataset to train the assessor model on it. These approaches also do not complete the properties of the assessor. Other work has been proposed to evaluate robustness testing of ML families [18], speech synthesis [23].

Several recommended fields of assessors are listed in a previous study [7] along with the proposal of the assessor and its properties, providing a simple classification example to demonstrate its ability. Moreover, one study proposed selecting and combining several systems to improve performance [24]. They developed a small random forest model to estimate how large language models can compile an input prompt properly. To construct the relationship between items and responses, they extract instance features from text and use the number of shots in training with systems’ characteristics as three-dimensional space. Instead, we build this relationship differently because we aim to reduce models’ errors.

Since many practical AI solutions are based on deep neural networks, their output mainly produces MCP; this raises the question of whether the outputs are correct. One particular work in [25] devoted to this is to depend on the True Class Probability instead of MCP by building an extra loss function based on the model confidence. Similar to [19], this work also benefits from the actual predictor. Therefore, it cannot be an assessor model.

We define a set of systems representing their emergent behaviors by S={s1,…,sN}, si∈Rn, and a set of objects by matrix X∈RM×m with corresponding target value vector y. Once we have both input data and systems to predict them, we can construct a set to train an assessor model by denoting it with matrix A∈RK×(m+n), where K is the number of examples in an assessor dataset (which is later used to train the assessor model), and m and n are dimensions of system and instance features, respectively. To collect responses from systems likewise in IRT, we can write as a matrix A. (1)A=(s1,1…s1,nx1,1…x1,m………………sN,1…sN,nx1,1…x1,ms1,1…s1,nx2,1…x2,m………………sN,1…sN,nx2,1…x2,m………………………………sN,1…sN,nxM,1…xM,m………………sN,1…sN,nxM,1…xM,m).

Dataset A represents a full combination of pair (si,xj) by joining the emergent behavior vector si and input vector in each row of the matrix. In practice, we cannot build such a combination, so we can randomly select pair (si,xj) to create batch examples. We also need to denote a error of pair (si,xj) by eji which is a error of system i on prediction instance j. These emergent behaviors and errors may vary slightly in different tasks so that we will define them for each task separately later.

Explaining AI/ML systems is one of the challenging parts of AI; therefore, a considerable amount of research efforts has been dedicated over the years. Early assessment of their behaviors before acquiring their outputs may prevent unexpected incidents in the future. One straightforward example of selecting problems from our life is a leader controlling a group to do given tasks and control the results of finished tasks. The problem of the leader is to assign each task to one of the group members. As the leader is experienced, the leader probably knows what tasks certain members can successfully finish and what kind of errors the member can make on a particular task. Like the leader, we also wish to build a meta-model called “assessor” that must satisfy the properties in section Introduction and explain any system for a given task.

The first significant benefit is to estimate probability Pr(success;si,xj) indicating the likelihood of successfully predicting instance i for system j if the prediction results are categorical. Building such a model is easier for a classification task, but if the task is continuous, then another method should be considered. In [7], authors advised using a parametric assessor model by supposing Gaussian distribution. However, in the current study, the assessor learning strategy is the target contribution by introducing an error threshold.

The first step to implementing an assessor model is to construct a meta-model with two inputs: system behaviors and input data (if we join them as shown in Eq. (1), it may have only one input). Then, we should collect responses from each system for each instance as shown in Section 3.1. In practice, the size of A would be N×M if it is built; therefore, we randomly chose system i and instance j and trained the assessor model in batches.

The flow of input instances in the deployed systems with their assessor meta-model: firstly, the input data are used in the assessor model with system features; then, regarding the assessor’s decision, whether it is passed to a desirable system to predict or reject; if not rejected, then, finally, the system output will be fixed with the assessor’s output, and if the actual value of the output exists, then it will be added to the assessor dataset to train the assessor model further.

This methodology is based on learning errors of system responses over instances instead of just assessing a system to see whether it can succeed. To adapt it, assume we have a regression task, and we have given a set of pretrained systems S with their behaviors. For a pair of system i and instance j, we can find the true error as follows:

(2)eji=yj−y^ji,where yj is a given target value of xj, and y^ji is predicted value of target yj for instance j by system i. Now, we can train our assessor model on a joint vector of pair (si,xj) by targeting error eji. After selecting one of the systems²2

In this case, we suppose that the task of the assessor mode is selecting the best model. In general, we can select any system out of the systems, and we fix its error.

(3)yj=y^ji+ftrun(e^ji),where e^ji is a prediction of the assessor since the assessor itself is an ML model, e^ji is not a true error of system i for instance j, and ftrun is a truncation function as follows:

(4)ftrun(e^ji)={e^jiif|e^ji|≤ϵ0otherwise,where ϵ denotes a threshold parameter determined differently. The critical characteristic of assessor models is that they are meta-ML models, similar to other models designed to operate under uncertainty. This inherent uncertainty can sometimes result in significant prediction errors in specific cases, making it necessary to clip such errors to prevent the adverse side effects associated with assessor models. The optimal value of the error threshold, ϵ, depends on the nature of the task, the task type, and the methodology employed to solve it. For instance, an ϵ value of 0.25 may be sufficient for logistic regression tasks, but in regression tasks, ϵ cannot be universally applied, as the range of model errors can be significantly larger. Therefore, each task requires a distinct and carefully calibrated value of ϵ to manage prediction errors effectively. We will primarily use Eq. (4) in experiments in the subsequent section and evaluate its utility in Section 5.1.

After introducing an error-learning strategy by the assessor instead of the success probability of systems, we cannot convert that error to this probability Pr(success;si,xi) as proposed originally for only regression tasks. So, one straightforward method is including maximum-error parameter emax>0. If the predicted error of pair (si,xj) by the assessor is less than the parameter, we can then make a decision that system i can succeed in predicting instance j. However, this approach produces only success or failure, not probability Pr(success;si,xj) as defined in [7]. To overcome this limitation, we might use the following definition for only regression problems, but Eq. (5) cannot give us Pr(success;si,xj) directly, instead it is only “Relative Success”.

Predicting instance performance (PIP) For the pair (si,xj) and a pretrained assessor model with a threshold emax. Eq. (5) computes the performance of predicting (Relative Success) an instance for regression tasks.

(5)Rpip={1−|e^i|emaxif|e^i|≤emax0otherwise,where e^ji is the predicted error by the assessor on object j for a given system i.

Similarly, we adjust the proposed methodology above to logistic regression tasks by elementary changes. The logistic regression with sigmoid output usually varies in the interval [0,1] that identifies the class probability of an input instance. Suppose we have data set by in matrix X∈RM×m with target value yi∈{0,1}. Without loss of generality, we can apply this method to logistic regression tasks. However, if the task is a multiclassification task, where yi={0,1,2,…,k}, a workaround with one-hot encoding should be applied, which is the core of the softmax solution. So, Eq. (3) will be a subtraction of vectors: true one-hot encoded vector of yj=[0,…,1,…,0] and prediction vector by softmax output y^ji∈Rk of system i for instance j. Hence, the assessor model also produces a vector e^ji∈Rk for each one-hot encoding.

The assessor dataset on which we train our assessor model consists of system feature values, instance feature values, and systems’ error responses in predicting these instances. The process of gathering responses might have several considerable challenges with various solutions. We build the behavior of ML models for each type of task slightly differently, mainly concentrating on the architecture of NN and SVM models described in Appendices A and E, including the hidden layer, number of parameters, hyperparameters, and other aspects. In turn, we leverage instance feature values without any preprocessing, but Reference [24] constructs these values from some patterns of texts, such as numbers and dates.

To illustrate the results derived from the assessor model on regression task, we leverage blog feedback [26,27] that contains about 60 k instances extracted from approximately 6 GB plain HTML documents collected from 37,279 Hungarian blog pages. Each object in the dataset is described by 280 features and one target value, which is the number of feedback (comments). The task of the dataset is to forecast the number of feedback for a given piece of news. Additionally, the dataset was split into a train set (52 K objects) and 60 test sets (roughly 127 instances in each) to analyze the model performance.

First, 30 randomly generated systems were trained on the training set, and then how the assessor model assessed these systems was investigated by averaging the results over the examples. Each system is a one-hidden layer NN model with the following architecture shown in Appendix A. Due to the random selection of losses during training systems, we remove some systems with NaN (Not a Number) loss values or some systems’ average PIP is less than a threshold value on the training set³3

According to PIP in Eq. (5), it is computed based on the predicted error by the assessor for pair (S_i, X_j). If we replace the predicted error with the true error after acquiring an output of system i on object j and average it over examples in the training set, we calculate the average PIP for system i.

In Fig. 1, all accuracies are close to 1 except the balanced one. According to these accuracies, the developed assessor model usually predicts well; it reaches near one on a few test sets without dispatching the considered system. Nevertheless, this assessor performance for a system with the lowest average PIP significantly differs from that in Appendix D, as both systems have various behaviors.

Typically, all rates in the latter scenario are markedly higher than in the former. This occurrence is similar to the class imbalance problem in classification settings because the assessor model suffers from few error responses produced by exceptionally reliable systems. This issue also occurs in the proposed feedback mechanism for the best and selected systems while not for the worst system demonstrated in Fig. 2; the corrections have contrasting effects. As the chosen system for each input instance is determined by the assessor’s identification of the lowest error, this phenomenon does not exert a significant negative impact but manifests in highly accurate models.

After correcting the model outputs, the worst system outperforms two out of nine systems, and the rest of the errors are close to the best and selected models. Nevertheless, the correction technique can have little influence on the two systems. In contrast to these outcomes, we see better results for classification problems in the following subsection. Both results are produced on various feedback truncation rules than Eq. (4), as detailed in Appendix B

Furthermore, Table 1 presents additional findings regarding regression tasks across various datasets. Similarly, the preceding results in the regression corrections shown do not demonstrate improvement for “the best” and “the selected” systems. However, they positively affect the worst systems, with the outcomes being the best in two datasets and comparable in others.

Likewise in the regression task, in this case, we also generate all NN models with only one hidden layer randomly and train an assessor model on their error responses across datasets. However, the assessor model is not as large as in the regression. We utilized 10 datasets to showcase the results of the proposed methodology in balanced and F1 scores presented in Table 2 for binary classifications. In most cases in Table 2, the feedback produced by the assessor model has affected on the values of both metrics considerably well (i.e., negative effects occurred on only three datasets for the best system and one dataset for the selected system).

To demonstrate the results for softmax output, we apply the proposed approach for multiclassification problems, and likewise the earlier experiments, we also employ NN models in the same manner. As described in the preceding section, we encode class labels into their one-hot form. For example, the true labels are encoded as y=[0,0,…,1,…,0], the respective predicted ones as y^=[0.02,0.05,…,0.95,…,0], and the errors are computed as yerror=y−y^=[−0.02,−0.05,…,0.05,…,0]. Since the error term in this task is not a scale similar to preceding both experiments, we sum absolute values of yerror to prefer and decide whether an ML model can succeed on a particular example. In turn, our assessor model also produces multiple outputs for each class. Table 3 presents the outcomes of the proposed method on this task. Analogous to outcomes in Table 2, error correction increased the average accuracies of most systems on almost all datasets, except in two cases: MNIST and Fashion MNIST for the best model.

Other models. We conducted analogous experiments employing SVM models for regression and classification tasks on the datasets. However, the behaviors of SVM models differ from those of NN models, resulting in diverse outcomes, as illustrated in Appendix E. Individual datasets and models require distinct approaches when selecting mechanism components.

Truncations. Truncation systems⁷7

Dropping less accurate systems is natural since we do not prefer them in practice.

Broader impacts. Modifying the assessor models’ original concept makes it possible to harness their capabilities. It opens up new research avenues, complementing existing ones outlined in [7], including failure explanation, maintenance, and revision. For instance, this could represent a novel step toward ensemble learning and AutoML, aiming to preempt ML model failures and address their inaccuracies in detail. Specifically, employing each weak learner with an assessor model is a potential avenue for future research, particularly in bootstrap aggregating.