Approaches to Conducting a Comparative Assessment of the Activities of Senior Officials and Executive Authorities of the Regions of the Russian Federation

Kryukov Sergey Vladimirovich ORCID: 0000-0003-4736-7697 Doctor of Economics Professor, Southern Federal University 344002, Russia, Rostov region, Rostov-On-Don, Gorky str., 88, office 105 svkrukov@sfedu.ru Other publications by this author

Introduction

Any system with control needs a feedback mechanism. The more complex the system, the more its successful functioning depends on the availability of an effective feedback mechanism. This fully applies to such a socio-economic system as a country. The Russian Federation is a large socio-economic system in which subsystems - subjects of the Russian Federation – are allocated for the convenience of management.

To ensure regular feedback from the subjects of the Russian Federation at the federal level, an annual assessment of the effectiveness of the activities of senior officials (VDL) of the subjects of the Russian Federation and the activities of the executive authorities of the subjects of the Russian Federation is carried out. Based on the results of this assessment, personnel decisions can be made regarding the VDL of the region, and every year the Government of the Russian Federation decides on the allocation of grants (additional budget funds) to the subjects of the Russian Federation who have achieved the best results following the results of the reporting period.

Thus, the Government of the Russian Federation annually has to solve the problem of ranking (clustering) of the subjects of the Russian Federation, and not just ranking, but obtaining quantitative estimates for each region in order to have grounds for allocating grants of a certain size to the regions. Since any subject of the Russian Federation is also a complex socio-economic system with management, dozens of different qualitative and quantitative indicators can be used to describe the state and development of which, the task of conducting a multifactorial comparative assessment arises.

The effectiveness of feedback and, consequently, the effectiveness of management of both individual regions and the country as a whole will largely depend on the quality of the comparative assessment carried out.

Materials and methods

To analyze existing approaches, as an example, the government methodology for comparative evaluation of the activities of the VDL and the executive authorities of the Subjects of the Russian Federation was used (Decree of the Government of the Russian Federation No. 542 dated 03.04.2021 "On Approval of Methods for Calculating indicators for evaluating the effectiveness of the activities of Senior Officials (heads of the Highest Executive bodies of state Power) of the Subjects of the Russian Federation and the Activities of Executive Authorities of the Subjects of the Russian Federation, as well as on the invalidation of certain provisions of the Decree of the Government of the Russian Federation No. 915 dated July 17, 2019". [electronic resource]. URL: https://publication.pravo.gov.ru/Document/View/0001202104130046).

To conduct a comparative assessment of the level and quality of life in the regions of the Russian Federation, data from Rosstat from 2005 to 2019 were used. (Regions of Russia. Socio-economic indicators - 2021 [Electronic resource]. URL: https://gks.ru/bgd/regl/b21_14p/Main.htm). Data visualization methods were used to present the results of a comparative evaluation of a set of objects (several dozen). To obtain an objective quantitative basis for the differentiation of the subjects of the Russian Federation based on the results of activities for the reporting period, the method of constructing indifference curves was used.

Results and discussion

To provide reliable feedback and improve the efficiency of the country's governance, it is necessary to obtain reliable quantitative estimates of the socio-economic development of each region for the reporting period (year). Is there one objective quantitative indicator that would characterize the overall socio-economic development of the region? To date, researchers have not been able to find such a single indicator that could characterize the overall socio-economic development of the region. That is why the methods of assessing the activities of the VDL and the executive power of the subject of the Russian Federation are based on the analysis of several (sometimes dozens) indicators characterizing various aspects of the socio-economic development of the region.

The task of our research was to identify the strengths and weaknesses of various approaches to conducting a comparative assessment of the activities of the VDL and the executive authorities of the subjects of the Russian Federation, and to suggest ways to improve them.

If we are faced with the task of comparing several objects, each of which can be represented by a vector of several particular indicators (as is the case with the assessment of the activities of the VDL and the executive authorities of the subjects of the Russian Federation), two main approaches are most often used – (1) based on the construction of an integral indicator characterizing each object, and based on the comparison of vectors, consisting of particular indicators characterizing each object.

Many researchers calculate an integral quantitative assessment of an object (for example, a region) based on the application of a formula for averaging the values of several particular indicators ^[1-8].

For example, in a study by a team of authors on the topic "Assessment of the quality of life of the labor force of the regions of Russia", the weighted arithmetic mean formula was used to calculate the integral index of the quality of life of the population ^{[3, p.115]}. A similar approach was used by E.M. Karpenko to conduct a comparative analysis of the level and quality of life of post-Soviet countries ^[4]. Foreign researchers analyze the use of such an integral indicator as "The OECD Better Life Index" ^{[6, 7]}. This index is calculated using the weighted average arithmetic formula based on 11 groups of private indicators (up to 3 indicators in each group).  

When applying this approach based on the calculation of the integral indicator, several serious problems arise. Firstly, there is no one "correct" averaging formula. According to the rule of majorance of averages, applying different averaging formulas for the same data set (partial indicators), we get different integral estimates of the same object. This can lead to random errors during the comparative evaluation of objects, as well as create an opportunity for manipulation with the evaluation results (you can choose a certain averaging formula to obtain a larger or smaller value of the integral indicator).

So, the choice of the averaging formula should not be random, it needs to be justified. Currently, in most studies, the formula for obtaining the arithmetic mean (or weighted arithmetic mean) is used to calculate the integral indicator. This formula has such a feature that allows you to obtain high values of the integral indicator characterizing the object, even if there are very low achievements in certain particular indicators.

For example, close integral estimates can be obtained by two such different regions. In one region (object No. 1), all indicators are approximately at the same level (for example, close to 100 percent in relation to the previous reporting period), respectively, and the value of the integral indicator will also be close to 100 percent. And in another region (object No. 2) - the same value of the integral indicator (about 100 percent) can be obtained if there are "failed" values for half of the private indicators and there are about half of the private indicators with a level of performance much higher than 100 percent (Fig. 1).

 Figure 1. Estimates of two objects by 10 indicators

Such an approach to conducting a comparative assessment of regions (VDL of regions) may allow some of them to receive sufficiently high final grades (remuneration based on the results of the assessment), compensating for failures in certain private indicators with high achievements in other private indicators.

For a comparative assessment of the activities of the VDL and the executive authorities of the subjects of the Russian Federation, a relatively small set of the most important indicators is formed (in different years, this set included – from 15 to 22 indicators). This means that each of these indicators is important for assessing the socio-economic development of the region, and it will be wrong to encourage the leadership of the region if there are "failures" on at least one of the selected indicators.

In order to comply with the principle of taking into account all selected indicators, it is advisable to choose a formula for calculating not the arithmetic mean, but the geometric mean in order to obtain an integral assessment of the object. In this case, the values of particular indicators (relative to the previous period or the target value) are multiplied, then the root of the corresponding degree is taken to obtain an integral estimate. In this case, it will not be possible to offset the low values of some indicators with higher values of other indicators. The value of at least one particular indicator close to zero will "pull" the overall integral estimate to zero. Thus, the conscious choice of the mean geometric formula for obtaining an integral assessment of the regions will in itself motivate the leadership of the region to actively work to maintain all the selected private indicators at a high level.

There is another approach to conducting a comparative assessment of objects characterized by a set of particular indicators. If there are relatively few of these indicators (15-20 units), it is possible to build a profile of each object for all particular indicators and compare objects by their profiles ^[9-13].

A study by a team of authors on the topic "Criteria for assessing the standard of living of the population of the country" compares different countries by individual indicators (minimum wage, average salary, average pension, poverty level, etc.) ^[10]. At the same time, however, there was no objective of a fair distribution of "remuneration" among the "best" countries. In the study of a team of foreign authors, the methods of conducting a cross-country comparative analysis based on ranking countries by individual indicators are analyzed ^[12].

When using this approach, we avoid the disadvantages associated with the problem of choosing an averaging formula to obtain one integral indicator, but other problems arise. If one object is superior to another in all indicators, we can definitely determine which object is better. But there still remains the question of the need to determine the quantitative difference between the estimates of the two objects in order to have a basis for the distribution of "remuneration" (when necessary).

If one object is better by some indicators, and another object is better by others, how to choose which one is generally better than the other? And how to determine – how much better? One of the options in this case is to determine which object is among the best in terms of a greater number of private indicators, and that object will be higher in rank. If we remember that in the case of a comparative assessment of regions (the activities of VDL regions) - each particular indicator is important, it may turn out that the region received a high rank among all the objects being compared, but at the same time "missed" one or two indicators, for example, such as "life expectancy of the population" or "growth In other words, a lot can be done at a high level in the region (investments are growing, roads are in good condition, volunteer movement is developing, etc.), only people live in poverty and life expectancy is low. It is unlikely that the received high assessment of such a region can be considered adequate.

Another possible solution is to conduct a comparative assessment of facilities for each particular indicator, identify the best and, for example, allocate grants separately in 15-20 areas. Then the total budget will need to be divided into separate budgets for each particular indicator. And in this case, not the most successful region may receive remuneration for its achievements in certain indicators, while in other indicators the percentage of completion may be very low. Thus, in the case of comparing objects, each of which is represented by a vector of particular indicators, when using different options, serious problems will still appear that can lead to distortion of the real picture when comparing regions.

Recommendations

We suggest using an approach that is based on reducing the set of important indicators to characterize even such a complex object as a subject of the Russian Federation to the minimum possible. Is it possible to do this? If so, how in this case to conduct a comparative assessment and analysis of the activities of the VDL and the executive authorities of the subjects of the Russian Federation?

In our opinion, this approach can be implemented based on the concept of assessing the level and quality of life of the population of the region. Many researchers use these two concepts as synonyms. Others distinguish them. The standard of living of the population is most often estimated by the degree of satisfaction of the material needs of a person. The quality of life is assessed according to the degree of satisfaction of a person's social and spiritual needs ^{[2, 4, 14, 15, 16]}.

In our study, we selected an indicator that, in our opinion, most accurately characterizes the standard of living of the population of the region – the size of the real per capita income of the population. To assess the region in dynamics, this may be an indicator of the growth rate of real per capita incomes of the population. All other private indicators characterizing the satisfaction of a person's material needs, as a rule, correlate with the indicator of the size of the real per capita income of the population. These are indicators reflecting the purchase of various goods and services, the improvement of housing conditions, the development of tourism, visits to cultural events, etc. To assess the quality of life of the population in the region, the indicator "life expectancy at birth" was selected ^[17].

It is important that each of these indicators (the size of real per capita incomes of the population (or the growth rate of real per capita incomes of the population) and life expectancy at birth) is regularly monitored by Rosstat authorities, including by individual regions, so the values of these indicators can be easily rechecked.

Is it possible to leave only one of these two indicators ("the size of the real per capita income of the population" and "the expected average life expectancy at birth")? The answer is negative, because there is no significant correlation between the values of these two indicators ^[18]. There are regions in the Russian Federation with fairly high values of the size of the real per capita income of the population and at the same time a relatively low average life expectancy, and vice versa. From our point of view, the minimum possible set of indicators characterizing the level and quality of life of the population of the region is a set of two indicators "the size of the real per capita income of the population" and "the expected average life expectancy at birth".

The possibilities of using the proposed approach will be shown by the example of the analysis of the methodology for assessing the activities of the VDL of the subjects of the Russian Federation and the activities of the executive authorities of the subjects of the Russian Federation. In April 2021, a decree of the Government of the Russian Federation was issued in the development of the Decree of the President of the Russian Federation with another set of indicators to assess the effectiveness of the activities of the VDL of the subjects of the Russian Federation and the activities of the executive authorities of the subjects of the Russian Federation (No. 542 of 03.04.2021). Similar resolutions in the development of the Decrees of the President of the Russian Federation were issued in 2012, 2017, 2019 ^[19]. The number and name of the selected indicators were constantly changing.

A meaningful analysis of those indicators that were included in the latest version of the government methodology for assessing the activities of the VDL and executive authorities of the subjects of the Russian Federation was carried out. In the methodology for each subject of the Russian Federation, the value of each particular indicator for the reporting period (year) is compared with the value of a similar indicator for the previous year, or with the target value of this indicator established by the Government of the Russian Federation. The analysis of the indicators was carried out from the point of view of their correlation with the main indicators reflecting the level and quality of life of the population of the region ("the size of real per capita income of the population" and "life expectancy at birth"), as well as taking into account the possibility of collecting reliable and objective data for calculating each indicator. The indicators are listed in the order of their enumeration in the above-mentioned methodology for assessing the activities of the VDL and executive authorities of the subjects of the Russian Federation.

1. The population of the subject of the Russian Federation.

The value of this indicator can be increased (the goal is to increase the value of the indicator) by increasing the birth rate and reducing the mortality rate in the region, by increasing the positive balance of migration, as well as by changing the borders of the region.

This indicator has a high correlation with the indicators selected by us that characterize the level and quality of life in the region. In addition, it is possible to change (increase) the population of the region within the country as a result of changing its borders only if the population of the adjacent region changes (decreases) symmetrically. Changing the borders of individual regions should be carried out in the presence of reasonable arguments within the framework of the implementation of a unified strategy for the development of the country, and should not depend on the aspirations of the VDL and the executive authorities of a particular region. 

2. Life expectancy at birth.

We consider this indicator to be one of the main ones reflecting the quality of life of the population of the region.

3. The level of poverty.

This indicator has a high correlation with the indicator "the size of the real per capita income of the population". In addition, the value of this indicator largely depends on the value of the subsistence minimum established in the region. If the growth of the subsistence minimum lags behind the growth of real incomes of the population, then the poverty level will formally decrease. Therefore, there should be an external independent examination of the establishment of the regional level of the subsistence minimum. Otherwise, it will remain possible to formally reduce the level of poverty in the region by curbing the growth rate of the official subsistence minimum.

4. The proportion of citizens who are systematically engaged in physical culture and sports.

The indicator has an average level of correlation with the selected indicators characterizing the level and quality of life in the region. All other things being equal, the higher level and quality of life in the region create more opportunities for systematic physical education and sports among the population. In addition, the value of this indicator is quite difficult to accurately calculate and verify. If the participants of the official sports sections and clubs can still be counted, then it is almost impossible to count the participants of the physical culture movement. The word "systematically" also adds uncertainty when calculating this indicator ("systematically engaged" - is it once a day, a week, a month?). It is also quite difficult to accurately determine the number of people with contraindications and restrictions for physical education and sports, especially for people who are no longer schoolchildren or students.

5. Level of education

An indicator that has an average level of correlation with selected indicators that characterize the level and quality of life in the region. The higher the standard of living of the population of the region, the more opportunities there are to get access to paid forms of education. The higher the average life expectancy of the population of the region, the longer each person will have access to additional vocational education.  It should also be taken into account that if the availability of places in preschool institutions can really be influenced by the authorities of the region, then their direct influence on the number of employees receiving vocational education and participating in additional vocational education programs is rather weak.

6. The effectiveness of the system for identifying, supporting, and developing abilities and talents in children and youth.

A private, less significant indicator. When calculating the indicator, a certain group of children and youth in the region aged 7 to 30 years is taken into account, which is very heterogeneous (schoolchildren, students, working youth, etc.). Statistics for this group need to be collected separately, Rosstat does not conduct such regular calculations by region.

7. The share of citizens engaged in volunteer activities.

Also a private, less significant indicator, not equivalent to other more significant indicators. The methodology suggests taking into account not only organized volunteering, but also unorganized. It is almost impossible to do this, there is a high probability of errors or attributions when calculating this indicator.

8. Conditions for the upbringing of a harmoniously developed and socially responsible personality.

A private, less significant indicator, when calculating which it is proposed to rely on six more indicators with preset different weight coefficients. It is also quite difficult to calculate this indicator accurately.

9. The number of visits to cultural events.

A private, less significant indicator, depending on the level and quality of life of the population of the region. If the level and quality of life satisfy the population of the region, then there is an opportunity and a desire to attend cultural events (including paid events).

 10. The number of families who have improved their living conditions.

Since a small part of the residents of the Russian Federation can improve their living conditions at the expense of the state, this indicator correlates very closely with the standard of living in the region, which is proposed to be assessed by the indicator "real per capita incomes of the population".

 11. The volume of housing construction.

The presence of the second indicator devoted to the housing issue indicates that there is still an understanding of the fact that the growth of housing construction does not automatically lead to an improvement in the housing conditions of many families. In many cities, there is currently free housing for sale, usually in apartment buildings, which does not find demand for quite a long period (real incomes of the population have practically not been growing in recent years).

 12. The quality of the urban environment.

To obtain quantitative estimates of the quality of the urban environment, the results of surveys of experts or a sample of the population of the region are used. Receiving responses from experts and the public, as well as processing these responses, is often associated with great uncertainty. The quality of the urban environment in the region as a whole is calculated as an arithmetic mean for the cities of the region. Since the cities in the region can be very different, it is necessary to calculate a weighted average arithmetic estimate with different weights for each city, which can still add uncertainty when obtaining the final estimate.

 13. The share of the road network in the largest urban agglomerations corresponding to the standards.

A private, less significant indicator. It is quite difficult to regularly determine the compliance of the road network with many standards. The situation with the road network is changing all the time, there is a lot of uncertainty in the calculations. In addition, this indicator does not characterize the entire region, although there is a road network not only in large urban agglomerations.

 14. Environmental quality.

It is important to assess the quality of the environment in terms of its long-term impact on the quality and standard of living of the population. It is advisable to use such indicators as "ecological footprint" and "bio-capacity" of the territory ^[20]. These indicators allow us to determine the balance /imbalance in the use of the ecological potential of the territory. The imbalance suggests that the current generation is spending the ecological resources of the territory faster than they are being restored.

15. The growth rate of the real average monthly salary.

16. The growth rate of real per capita income.

The real per capita income of the population as a whole can characterize the standard of living in the region. Since in the Russian Federation the majority of the population has the largest share in the structure of monetary income is wages, you can leave one indicator out of two – "the growth rate of real per capita income."

17. The growth rate of the physical volume of investments in fixed assets.

A less significant indicator that partially characterizes the development of the region's economy. It does not directly affect the growth of the level and quality of life of the population of the region.

18. The number of people employed in the field of small and medium-sized businesses, including individual entrepreneurs and self-employed.

A private, less significant indicator. Should this indicator grow all the time in each region? Also, this indicator does not directly affect the level and quality of life of the population of the region.

 19. "Digital maturity" of public authorities, local self-government and organizations in the field of healthcare, education, urban economy and construction, public transport, implying the use of domestic IT solutions by them.

A private, less significant indicator. It does not directly affect the level and quality of life of the population of the region. It can be monitored under a separate federal program, as well as a number of other private indicators.

Thus, in the analyzed government methodology, it is proposed to use rather heterogeneous and unequal indicators, some of which have a high correlation with each other, to assess the activities of the VDL and the executive authorities of the subjects of the Russian Federation. Therefore, we consider it appropriate to leave two main indicators ("the size of the real per capita income of the population" and "the expected average life expectancy at birth") to assess the activities of the VDL and the executive authorities of the subjects of the Russian Federation.

The presence of only two indicators characterizing each object makes it possible to build visualizations of the results of a comparative assessment of regions both in statics and dynamics. It is possible to place regions (their images) in the space of two coordinates: the size of the real per capita income of the population (or the growth rate of the real per capita income of the population) and life expectancy at birth. Figure 2 shows an example of visualization of a comparative assessment of the regions of the Russian Federation according to data for 2019.

Figure 2. Regions of the Russian Federation (images) (according to Rosstat of the Russian Federation).

A clear, easily verifiable and visual tool for comparative assessment of regions appears. The more to the right and higher the region is located in a given coordinate space, the higher the assessment of the VDL and the executive power of the region for the reporting period (year). For example, if you draw vertical and horizontal lines through a point reflecting the position of the Russian Federation in this coordinate space in Figure 2 (the average data for the country - the point has coordinates 35247 (SRD, rubles) and 73.3 (average OPJ, years)), we get four clusters. Cluster 1 includes regions with the worst results in the country in terms of the level and quality of life of the region's population. In cluster 2, there were regions with high values of the indicator characterizing the quality of life in the region, but low values of the indicator characterizing the standard of living of the population of the region. In cluster 3, there were regions with a relatively high standard of living of the population, but with a low life expectancy. The leading regions were included in cluster 1, they have values of indicators characterizing both the level and quality of life higher than the average in Russia.

No less important is the comparative assessment of the regions in dynamics. The trajectory of the movement of the region (its image) in this space will clearly show the dynamics of the change in the level and quality of life of the population of the region as a result of the socio-economic policy pursued by the regional authorities. If the trajectory of the region's movement in this coordinate space is directed to the right and upwards, this is the basis for a positive assessment of the VDL and the executive power of the region for a certain period. Any deviation of the trajectory to the left and down is a signal of trouble in the socio-economic development of the region.

For example, Figure 3 shows the trajectory of the image of St. Petersburg in the coordinate space "relative per capita incomes of the population of the region" (as a percentage) and "relative life expectancy at birth" (as a percentage) according to data for 2005 to 2019 ^[21].

It can be noted that in the period from 2005 to 2011, the quality of life in this subject of the Russian Federation grew, at the same time, the standard of living decreased (relative to the average Russian indicators. From 2011 to 2017, the trend changed to the opposite. Such a visual express analysis allows us to conduct an enlarged comparative assessment of the socio-economic development of the regions in dynamics and draw appropriate conclusions. If necessary, in the future, it is possible to conduct a more detailed analysis of individual private indicators that have affected the growth or decline in the level and (or) quality of life of the population of the region. Based on the data of the Rosstat of the Russian Federation, any researcher will have the same trajectories of moving regions in the space of the selected two coordinates.

Figure 3. The trajectory of the image of St. Petersburg from 2005 to 2019 (calculated according to Rosstat of the Russian Federation).

To use the results of a comparative assessment of regions for an objective distribution of rewards (in the government methodology – grants), you can use the method of constructing indifference curves in the space of the selected two coordinates. The allocation of clusters of regions with approximately the same level and quality of life (located between neighboring indifference curves) will allow the distribution of grants of different sizes among the most successful regions on an objective basis.

Figure 4 shows, for example, the regions of the Southern Federal District divided by indifference curves into 3 clusters.

Figure 4. Regions of the Southern Federal District (calculated according to Rosstat data for 2017).

The Krasnodar Territory was included in the first cluster according to the results of its activities in 2017 (both in terms of living standards and quality of life, the results are higher than the average Russian level, above 100%). This region could receive a grant in the maximum amount among other subjects of the Russian Federation located in the Southern Federal District (for example, 5 percent of the annual budget of the region). The regions included in the second cluster (Astrakhan Region, Volgograd Region, Rostov Region and the Republic of Adygea) could receive grants, for example, in the amount of 3 percent of their annual budget, with the condition that the allocated funds should be used first of all to improve the level or quality of life of the population of the region. These conditions can be determined by analyzing the location of the image of the region in the coordinate space "standard of living - quality of life". The third cluster included one region – the Republic of Kalmykia. This region could be allocated a grant in the amount of 1 percent of the regional budget - purposefully to improve the standard of living of the population (in 2017, according to the report, less than 50% of the average Russian level).

Conclusion and conclusions

Thus, on the basis of the analysis, a new approach is proposed to conduct a comparative assessment of the activities of the VDL and the executive authorities of the subjects of the Russian Federation, based on the use of two objective indicators characterizing the standard of living and quality of life of the population of the region.

The advantages of the proposed approach to assessing the activities of the VDL and the executive authorities of the subjects of the Russian Federation are as follows.

A logical and understandable connection of the selected two indicators with the level and quality of life of the population of the region (the size of the real per capita income of the population (or the growth rate of real per capita income of the population) and life expectancy at birth).

Regular monitoring of these indicators by Rosstat bodies, unambiguous interpretation reduce the level of subjectivity and uncertainty when conducting a comparative assessment of regions. Different researchers will get the same results when conducting a comparative assessment of regions based on this approach.

The ability to visualize a comparative assessment of regions from the point of view of the level and quality of life of the population, both in statics and dynamics, makes it faster and easier to perceive information about the results of a comparative assessment, as well as to understand the logic and consequences of decisions made based on the results of the assessment.

Using the method of constructing indifference curves allows us to obtain an objective basis for the distribution of remuneration (grants) based on the results of a comparative assessment of the performance of the VDL and the executive authorities of the subjects of the Russian Federation for the reporting period.