mvord: Additional examples

Rainer Hirk, Kurt Hornik and Laura Vana

2020-04-17

Load package mvord

library(mvord)

Load data set

data("data_cr")
str(data_cr, vec.len = 3)
## 'data.frame':    690 obs. of  10 variables:
##  $ rater1 : Ord.factor w/ 5 levels "A"<"B"<"C"<"D"<..: 2 3 3 2 5 4 3 4 ...
##  $ rater2 : Ord.factor w/ 5 levels "A"<"B"<"C"<"D"<..: 2 4 4 2 5 NA 3 4 ...
##  $ rater3 : Ord.factor w/ 6 levels "F"<"G"<"H"<"I"<..: 3 NA NA NA 6 NA 2 5 ...
##  $ rater4 : Ord.factor w/ 2 levels "L"<"M": 1 2 2 1 2 2 2 2 ...
##  $ firm_id: int  1 2 3 4 5 6 7 8 ...
##  $ LR     : num  1.72 1.84 2.64 1.31 ...
##  $ LEV    : num  2.114 0.883 2.3 2.638 ...
##  $ PR     : num  0.3779 -0.1503 -0.0521 0.3289 ...
##  $ RSIZE  : num  -6.37 -7.84 -7.98 -5.86 ...
##  $ BETA   : num  0.836 0.49 0.802 1.137 ...
head(data_cr, n = 3)
##   rater1 rater2 rater3 rater4 firm_id       LR       LEV          PR     RSIZE
## 1      B      B      H      L       1 1.720041 2.1144513  0.37792213 -6.365053
## 2      C      D   <NA>      M       2 1.836574 0.8826725 -0.15032402 -7.839813
## 3      C      D   <NA>      M       3 2.638177 2.2997237 -0.05205389 -7.976650
##        BETA
## 1 0.8358773
## 2 0.4895358
## 3 0.8022900

Example 1 - A simple example with 2 raters

We introduce a simple example with 2 raters (rater1 and rater2) rating a set of firms. The data set data_cr has a wide format and hence, we apply the multiple measurement object MMO2 on the left-hand side of the formula object. The firm-level covariates LR, LEV, PR, RSIZE, BETA are passed on the right-hand side of the formula. The thresholds are set equal by the argument thresholds.constraints.

res_cor_probit_2raters <- mvord(formula = MMO2(rater1, rater2) ~ 0 + LR + LEV + PR + RSIZE + BETA,
                               threshold.constraints = c(1, 1),
                               data = data_cr)

The results are displayed by:

summary(res_cor_probit_2raters)
## 
## Call: mvord(formula = MMO2(rater1, rater2) ~ 0 + LR + LEV + PR + RSIZE + 
##     BETA, data = data_cr, threshold.constraints = c(1, 1))
## 
## Formula: MMO2(rater1, rater2) ~ 0 + LR + LEV + PR + RSIZE + BETA
## 
##     link threshold nsubjects ndim   logPL   CLAIC   CLBIC fevals
## mvprobit  flexible       690    2 -799.84 1630.35 1699.91   3218
## 
## Thresholds:
##            Estimate Std. Error z value  Pr(>|z|)    
## rater1 A|B  8.40534    0.39600  21.225 < 2.2e-16 ***
## rater1 B|C  9.87575    0.42661  23.149 < 2.2e-16 ***
## rater1 C|D 11.68479    0.46471  25.145 < 2.2e-16 ***
## rater1 D|E 14.00772    0.53773  26.050 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Coefficients:
##          Estimate Std. Error  z value  Pr(>|z|)    
## LR 1     0.191815   0.059864   3.2042  0.001354 ** 
## LR 2     0.089980   0.069426   1.2961  0.194953    
## LEV 1    0.456827   0.039714  11.5028 < 2.2e-16 ***
## LEV 2    0.441118   0.044323   9.9523 < 2.2e-16 ***
## PR 1    -2.626682   0.174938 -15.0149 < 2.2e-16 ***
## PR 2    -2.761411   0.203692 -13.5568 < 2.2e-16 ***
## RSIZE 1 -1.166861   0.050982 -22.8878 < 2.2e-16 ***
## RSIZE 2 -1.184522   0.050259 -23.5685 < 2.2e-16 ***
## BETA 1   1.644008   0.100520  16.3550 < 2.2e-16 ***
## BETA 2   1.741845   0.128535  13.5515 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error Structure:
##                    Estimate Std. Error z value  Pr(>|z|)    
## corr rater1 rater2 0.872025   0.025461   34.25 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The coefficients can be extracted by:

coef(res_cor_probit_2raters)
##        LR 1        LR 2       LEV 1       LEV 2        PR 1        PR 2 
##  0.19181519  0.08998022  0.45682662  0.44111809 -2.62668211 -2.76141092 
##     RSIZE 1     RSIZE 2      BETA 1      BETA 2 
## -1.16686094 -1.18452182  1.64400769  1.74184465

Example 2 - A simple example with 3 raters

A simple example with 3 raters (rater1, rater2 and rater3) is presented. The regression coefficients are set equal by the argument coef.constraints.

res_cor_probit_3raters <- mvord(formula = MMO2(rater1, rater2, rater 3) ~ 0 + LR + LEV + PR + RSIZE + BETA,
                               coef.constraints = c(1, 1, 1),
                               data = data_cr,
                               link = mvlogit())

The results are displayed by:

summary(res_cor_probit_3raters)
## 
## Call: mvord(formula = MMO2(rater1, rater2, rater3) ~ 0 + LR + LEV + 
##     PR + RSIZE + BETA, data = data_cr, link = mvlogit(), coef.constraints = c(1, 
##     1, 1))
## 
## Formula: MMO2(rater1, rater2, rater3) ~ 0 + LR + LEV + PR + RSIZE + BETA
## 
##    link threshold nsubjects ndim   logPL   CLAIC   CLBIC fevals
## mvlogit  flexible       690    3 -1561.4 3194.64 3357.58   4222
## 
## Thresholds:
##            Estimate Std. Error z value  Pr(>|z|)    
## rater1 A|B 14.71273    0.84962  17.317 < 2.2e-16 ***
## rater1 B|C 17.46132    0.92699  18.837 < 2.2e-16 ***
## rater1 C|D 20.65829    1.03917  19.880 < 2.2e-16 ***
## rater1 D|E 24.68645    1.20698  20.453 < 2.2e-16 ***
## rater2 A|B 14.87020    0.85299  17.433 < 2.2e-16 ***
## rater2 B|C 17.43897    0.93096  18.732 < 2.2e-16 ***
## rater2 C|D 20.47189    1.04358  19.617 < 2.2e-16 ***
## rater2 D|E 24.80045    1.22174  20.299 < 2.2e-16 ***
## rater3 F|G 14.66330    0.86375  16.976 < 2.2e-16 ***
## rater3 G|H 17.35099    0.92572  18.743 < 2.2e-16 ***
## rater3 H|I 20.73809    1.04700  19.807 < 2.2e-16 ***
## rater3 I|J 23.25685    1.15044  20.216 < 2.2e-16 ***
## rater3 J|K 25.33849    1.22854  20.625 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Coefficients:
##          Estimate Std. Error  z value  Pr(>|z|)    
## LR 1     0.333598   0.112624   2.9621  0.003056 ** 
## LEV 1    0.746847   0.076165   9.8057 < 2.2e-16 ***
## PR 1    -4.930267   0.356519 -13.8289 < 2.2e-16 ***
## RSIZE 1 -2.073425   0.108014 -19.1958 < 2.2e-16 ***
## BETA 1   3.021152   0.222735  13.5639 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error Structure:
##                    Estimate Std. Error z value  Pr(>|z|)    
## corr rater1 rater2 0.866603   0.027446  31.575 < 2.2e-16 ***
## corr rater1 rater3 0.903774   0.024724  36.554 < 2.2e-16 ***
## corr rater2 rater3 0.822653   0.045875  17.933 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The threshold parameters are presented by:

thresholds(res_cor_probit_3raters)
## $rater1
##      A|B      B|C      C|D      D|E 
## 14.71273 17.46132 20.65829 24.68645 
## 
## $rater2
##      A|B      B|C      C|D      D|E 
## 14.87020 17.43897 20.47189 24.80045 
## 
## $rater3
##      F|G      G|H      H|I      I|J      J|K 
## 14.66330 17.35099 20.73809 23.25685 25.33849

Example 3a - A model of firm ratings assigned by multiple raters and an investment-speculative grade indicator

This example presents a four dimensional ordinal regression model with probit link and a general correlation error structure cor_general(~ 1). The fourth rater only rates the firms on a binary scale: investment grade vs. speculative grade.

res_cor_probit_simple <- mvord(formula = MMO2(rater1, rater2, rater3, rater4) ~ 0 + LR + LEV + PR + RSIZE + BETA, data = data_cr)

The results are displayed by:

summary(res_cor_probit_simple)
## 
## Call: mvord(formula = MMO2(rater1, rater2, rater3, rater4) ~ 0 + LR + 
##     LEV + PR + RSIZE + BETA, data = data_cr)
## 
## Formula: MMO2(rater1, rater2, rater3, rater4) ~ 0 + LR + LEV + PR + RSIZE + 
##     BETA
## 
##     link threshold nsubjects ndim    logPL   CLAIC   CLBIC fevals
## mvprobit  flexible       690    4 -2925.79 6037.29 6458.57   6139
## 
## Thresholds:
##            Estimate Std. Error z value  Pr(>|z|)    
## rater1 A|B  8.05308    0.44312  18.174 < 2.2e-16 ***
## rater1 B|C  9.57196    0.47384  20.201 < 2.2e-16 ***
## rater1 C|D 11.35469    0.51753  21.940 < 2.2e-16 ***
## rater1 D|E 13.52181    0.60134  22.486 < 2.2e-16 ***
## rater2 A|B  8.59974    0.49820  17.262 < 2.2e-16 ***
## rater2 B|C 10.06007    0.53930  18.654 < 2.2e-16 ***
## rater2 C|D 11.86508    0.59726  19.866 < 2.2e-16 ***
## rater2 D|E 14.34057    0.70069  20.466 < 2.2e-16 ***
## rater3 F|G  8.24546    0.51708  15.946 < 2.2e-16 ***
## rater3 G|H  9.77754    0.55527  17.608 < 2.2e-16 ***
## rater3 H|I 11.70957    0.62261  18.807 < 2.2e-16 ***
## rater3 I|J 13.09715    0.68735  19.055 < 2.2e-16 ***
## rater3 J|K 14.17708    0.72080  19.669 < 2.2e-16 ***
## rater4 L|M 13.54304    1.00738  13.444 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Coefficients:
##          Estimate Std. Error  z value  Pr(>|z|)    
## LR 1     0.208387   0.067996   3.0647  0.002179 ** 
## LR 2     0.153527   0.073349   2.0931  0.036340 *  
## LR 3     0.180650   0.078391   2.3045  0.021195 *  
## LR 4     0.150135   0.112011   1.3404  0.180128    
## LEV 1    0.430524   0.043758   9.8388 < 2.2e-16 ***
## LEV 2    0.433143   0.050132   8.6400 < 2.2e-16 ***
## LEV 3    0.399637   0.050768   7.8719 3.493e-15 ***
## LEV 4    0.626346   0.074278   8.4325 < 2.2e-16 ***
## PR 1    -2.574577   0.194047 -13.2678 < 2.2e-16 ***
## PR 2    -2.829004   0.216932 -13.0410 < 2.2e-16 ***
## PR 3    -2.679726   0.222574 -12.0397 < 2.2e-16 ***
## PR 4    -2.797267   0.281530  -9.9360 < 2.2e-16 ***
## RSIZE 1 -1.130529   0.056380 -20.0518 < 2.2e-16 ***
## RSIZE 2 -1.197017   0.061751 -19.3845 < 2.2e-16 ***
## RSIZE 3 -1.196935   0.066398 -18.0266 < 2.2e-16 ***
## RSIZE 4 -1.567831   0.116397 -13.4696 < 2.2e-16 ***
## BETA 1   1.602576   0.110842  14.4581 < 2.2e-16 ***
## BETA 2   1.802612   0.140077  12.8687 < 2.2e-16 ***
## BETA 3   1.517178   0.139209  10.8985 < 2.2e-16 ***
## BETA 4   1.990449   0.204850   9.7166 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error Structure:
##                    Estimate Std. Error z value  Pr(>|z|)    
## corr rater1 rater2 0.874183   0.024864  35.158 < 2.2e-16 ***
## corr rater1 rater3 0.914814   0.023171  39.481 < 2.2e-16 ***
## corr rater1 rater4 0.900697   0.031939  28.201 < 2.2e-16 ***
## corr rater2 rater3 0.837847   0.041416  20.230 < 2.2e-16 ***
## corr rater2 rater4 0.926213   0.031728  29.192 < 2.2e-16 ***
## corr rater3 rater4 0.845626   0.060134  14.062 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The threshold coefficients can be extracted by:

thresholds(res_cor_probit_simple)
## $rater1
##       A|B       B|C       C|D       D|E 
##  8.053083  9.571962 11.354686 13.521806 
## 
## $rater2
##       A|B       B|C       C|D       D|E 
##  8.599739 10.060068 11.865083 14.340568 
## 
## $rater3
##       F|G       G|H       H|I       I|J       J|K 
##  8.245461  9.777541 11.709568 13.097152 14.177082 
## 
## $rater4
##      L|M 
## 13.54304

The regression coefficients are obtained by:

coef(res_cor_probit_simple)
##       LR 1       LR 2       LR 3       LR 4      LEV 1      LEV 2      LEV 3 
##  0.2083869  0.1535266  0.1806502  0.1501350  0.4305235  0.4331427  0.3996369 
##      LEV 4       PR 1       PR 2       PR 3       PR 4    RSIZE 1    RSIZE 2 
##  0.6263461 -2.5745773 -2.8290041 -2.6797255 -2.7972672 -1.1305294 -1.1970173 
##    RSIZE 3    RSIZE 4     BETA 1     BETA 2     BETA 3     BETA 4 
## -1.1969355 -1.5678310  1.6025757  1.8026120  1.5171782  1.9904487

The error structure for firm with firm_id = 11 is displayed by:

error_structure(res_cor_probit_simple)[[11]]
##           rater1    rater2    rater3    rater4
## rater1 1.0000000 0.8741830 0.9148139 0.9006967
## rater2 0.8741830 1.0000000 0.8378465 0.9262133
## rater3 0.9148139 0.8378465 1.0000000 0.8456261
## rater4 0.9006967 0.9262133 0.8456261 1.0000000

Example 3b - A more elaborate model of ratings assigned by multiple raters

In this example, we extend the setting of the above example by imposing constraints on the regression as well as on the threshold parameters and changing the link function to the multivariate logit link.

res_cor_logit <- mvord(formula = MMO2(rater1, rater2, rater3, rater4) ~
    0 + LR + LEV + PR + RSIZE + BETA, data = data_cr, link = mvlogit(),
    coef.constraints = cbind(LR = c(1, 1, 1, 1), 
                             LEV = c(1, 2, 3, 4),
                             PR = c(1, 1, 1, 1), 
                             RSIZE = c(1, 1, 1, 2), 
                             BETA = c(1, 1, 2, 3)),
    threshold.constraints = c(1, 1, 2, 3))

The results are displayed by:

summary(res_cor_logit)
## 
## Call: mvord(formula = MMO2(rater1, rater2, rater3, rater4) ~ 0 + LR + 
##     LEV + PR + RSIZE + BETA, data = data_cr, link = mvlogit(), 
##     coef.constraints = cbind(c(1, 1, 1, 1), c(1, 2, 3, 4), c(1, 
##         1, 1, 1), c(1, 1, 1, 2), c(1, 1, 2, 3)), threshold.constraints = c(1, 
##         1, 2, 3))
## 
## Formula: MMO2(rater1, rater2, rater3, rater4) ~ 0 + LR + LEV + PR + RSIZE + 
##     BETA
## 
##    link threshold nsubjects ndim    logPL   CLAIC   CLBIC fevals
## mvlogit  flexible       690    4 -2926.42 5987.81 6293.98  10626
## 
## Thresholds:
##            Estimate Std. Error z value  Pr(>|z|)    
## rater1 A|B 15.04918    0.82409  18.262 < 2.2e-16 ***
## rater1 B|C 17.75219    0.89727  19.785 < 2.2e-16 ***
## rater1 C|D 20.97822    1.00773  20.817 < 2.2e-16 ***
## rater1 D|E 25.13048    1.17487  21.390 < 2.2e-16 ***
## rater3 F|G 14.47061    0.83922  17.243 < 2.2e-16 ***
## rater3 G|H 17.17327    0.89515  19.185 < 2.2e-16 ***
## rater3 H|I 20.56635    1.01119  20.339 < 2.2e-16 ***
## rater3 I|J 23.00524    1.11045  20.717 < 2.2e-16 ***
## rater3 J|K 24.97259    1.18725  21.034 < 2.2e-16 ***
## rater4 L|M 23.92769    1.63196  14.662 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Coefficients:
##          Estimate Std. Error  z value  Pr(>|z|)    
## LR 1     0.340210   0.110547   3.0775  0.002087 ** 
## LEV 1    0.784295   0.075977  10.3228 < 2.2e-16 ***
## LEV 2    0.779695   0.078364   9.9497 < 2.2e-16 ***
## LEV 3    0.718330   0.093425   7.6889 1.484e-14 ***
## LEV 4    1.107836   0.123681   8.9572 < 2.2e-16 ***
## PR 1    -4.917965   0.343464 -14.3187 < 2.2e-16 ***
## RSIZE 1 -2.093379   0.103690 -20.1889 < 2.2e-16 ***
## RSIZE 2 -2.746162   0.188731 -14.5507 < 2.2e-16 ***
## BETA 1   3.135693   0.221944  14.1283 < 2.2e-16 ***
## BETA 2   2.733086   0.252960  10.8044 < 2.2e-16 ***
## BETA 3   3.572688   0.349493  10.2225 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error Structure:
##                    Estimate Std. Error z value  Pr(>|z|)    
## corr rater1 rater2 0.859773   0.027907  30.808 < 2.2e-16 ***
## corr rater1 rater3 0.908834   0.024636  36.891 < 2.2e-16 ***
## corr rater1 rater4 0.903959   0.031857  28.375 < 2.2e-16 ***
## corr rater2 rater3 0.834910   0.044258  18.865 < 2.2e-16 ***
## corr rater2 rater4 0.932243   0.032172  28.977 < 2.2e-16 ***
## corr rater3 rater4 0.856221   0.058398  14.662 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The constraints are displayed by:

constraints(res_cor_logit)
## $LR
##     LR 1
## A|B    1
## B|C    1
## C|D    1
## D|E    1
## A|B    1
## B|C    1
## C|D    1
## D|E    1
## F|G    1
## G|H    1
## H|I    1
## I|J    1
## J|K    1
## L|M    1
## 
## $LEV
##     LEV 1 LEV 2 LEV 3 LEV 4
## A|B     1     0     0     0
## B|C     1     0     0     0
## C|D     1     0     0     0
## D|E     1     0     0     0
## A|B     0     1     0     0
## B|C     0     1     0     0
## C|D     0     1     0     0
## D|E     0     1     0     0
## F|G     0     0     1     0
## G|H     0     0     1     0
## H|I     0     0     1     0
## I|J     0     0     1     0
## J|K     0     0     1     0
## L|M     0     0     0     1
## 
## $PR
##     PR 1
## A|B    1
## B|C    1
## C|D    1
## D|E    1
## A|B    1
## B|C    1
## C|D    1
## D|E    1
## F|G    1
## G|H    1
## H|I    1
## I|J    1
## J|K    1
## L|M    1
## 
## $RSIZE
##     RSIZE 1 RSIZE 2
## A|B       1       0
## B|C       1       0
## C|D       1       0
## D|E       1       0
## A|B       1       0
## B|C       1       0
## C|D       1       0
## D|E       1       0
## F|G       1       0
## G|H       1       0
## H|I       1       0
## I|J       1       0
## J|K       1       0
## L|M       0       1
## 
## $BETA
##     BETA 1 BETA 2 BETA 3
## A|B      1      0      0
## B|C      1      0      0
## C|D      1      0      0
## D|E      1      0      0
## A|B      1      0      0
## B|C      1      0      0
## C|D      1      0      0
## D|E      1      0      0
## F|G      0      1      0
## G|H      0      1      0
## H|I      0      1      0
## I|J      0      1      0
## J|K      0      1      0
## L|M      0      0      1

Note that the composite likelihood information criteria can be used for model comparison. For objects of class mvord the composite likelihood AIC and BIC are computed by AIC() and BIC(), respectively. The model fits of examples one and two are compared by means of BIC and AIC. We observe that the model of Example 3b has a lower BIC and AIC indicating a better model fit:

BIC(res_cor_probit_simple, res_cor_logit)
##                             df      BIC
## res_cor_probit_simple 92.85905 6458.566
## res_cor_logit         67.48687 6293.977
AIC(res_cor_probit_simple)
## [1] 6037.293
AIC(res_cor_logit)
## [1] 5987.81

The value of the pairwise log-likelihood of the two models can be extracted by:

logLik(res_cor_probit_simple)
## 'log Lik.' -2925.788 (df=92.85905)
logLik(res_cor_logit)
## 'log Lik.' -2926.418 (df=67.48687)

Marginal predictions are obtained by:

mp <- marginal_predict(res_cor_logit, type = "class")
table(data_cr$rater1, mp$rater1)
##    
##       A   B   C   D   E
##   A  47  23   4   0   0
##   B  11  65  44   3   0
##   C   0  28 135  38   0
##   D   1   2  44 116  17
##   E   0   0   0  30  47
table(data_cr$rater2, mp$rater2)
##    
##       A   B   C   D   E
##   A  29  20   8   0   0
##   B  12  41  22   2   0
##   C   1  18 103  24   0
##   D   0   0  35 109   7
##   E   0   0   0  13  39
table(data_cr$rater3, mp$rater3)
##    
##      F  G  H  I  J  K
##   F 25  8  2  0  0  0
##   G  4 33 23  1  0  0
##   H  1 18 85 13  0  0
##   I  0  0 28 30  3  2
##   J  0  0  2 13 10  7
##   K  0  0  0  6  8 23
table(data_cr$rater4, mp$rater4)
##    
##       L   M
##   L 195  40
##   M  30 425

Joint predictions are obtained by:

jp <- joint_probabilities(res_cor_logit, 
                          response.cat = data_cr[,1:4],  
                          type = "prob")
table(data_cr$rater1, jp$rater1)
##    
##       A   B   C   D   E
##   A  50  20   4   0   0
##   B  11  68  41   3   0
##   C   0  36 130  35   0
##   D   1   2  47 112  18
##   E   0   0   0  25  52
table(data_cr$rater2, jp$rater2)
##    
##       A   B   C   D   E
##   A  33  17   7   0   0
##   B  12  46  17   2   0
##   C   1  24  99  22   0
##   D   0   0  39 103   9
##   E   0   0   0  10  42
table(data_cr$rater3, jp$rater3)
##    
##      F  G  H  I  J  K
##   F 27  7  1  0  0  0
##   G  6 33 21  1  0  0
##   H  1 24 77 15  0  0
##   I  0  0 25 35  1  2
##   J  0  0  2 17  3 10
##   K  0  0  0  9  1 27
table(data_cr$rater4, jp$rater4)
##    
##       L   M
##   L 175  60
##   M  22 433

Example 4 - A longitudinal model

In this example, we present a longitudinal multivariate ordinal probit regression model with a covariate dependent AR(1) error structure using the data set data_cr_panel:

data("data_cr_panel", package = "mvord")
str(data_cr_panel, vec.len = 3)
## 'data.frame':    11320 obs. of  9 variables:
##  $ rating : Ord.factor w/ 5 levels "A"<"B"<"C"<"D"<..: 5 3 3 3 3 1 3 3 ...
##  $ firm_id: int  1 2 3 4 5 6 7 8 ...
##  $ year   : Factor w/ 8 levels "year1","year2",..: 1 1 1 1 1 1 1 1 ...
##  $ LR     : num  572.86 1.38 7.46 10.9 ...
##  $ LEV    : num  1.2008 0.0302 0.1517 0.5485 ...
##  $ PR     : num  0.1459 -0.0396 0.0508 0.1889 ...
##  $ RSIZE  : num  1.423 -1.944 2.024 -0.433 ...
##  $ BETA   : num  1.148 1.693 0.761 2.24 ...
##  $ BSEC   : Factor w/ 8 levels "BSEC1","BSEC2",..: 3 6 3 7 6 7 7 7 ...
head(data_cr_panel, n = 3)
##   rating firm_id  year         LR        LEV          PR     RSIZE      BETA
## 1      E       1 year1 572.864658 1.20084294  0.14585117  1.422948 1.1481020
## 2      C       2 year1   1.379547 0.03022761 -0.03962597 -1.944265 1.6926956
## 3      C       3 year1   7.462706 0.15170420  0.05083517  2.024098 0.7610057
##    BSEC
## 1 BSEC3
## 2 BSEC6
## 3 BSEC3
res_AR1_probit <- mvord(formula = MMO(rating, firm_id, year) ~ LR + LEV +
  PR + RSIZE + BETA, 
  error.structure = cor_ar1(~ BSEC), link = mvprobit(),
  data = data_cr_panel, 
  coef.constraints = c(rep(1, 4), rep(2, 4)),
  threshold.constraints = rep(1, 8), 
  threshold.values = rep(list(c(0, NA, NA, NA)),8),
  control = mvord.control(solver = "BFGS"))

The results of the model can be presented by

summary(res_AR1_probit, short = TRUE, call = FALSE)
## Formula: MMO(rating, firm_id, year) ~ LR + LEV + PR + RSIZE + BETA
## 
##     link threshold nsubjects ndim     logPL     CLAIC     CLBIC fevals
## mvprobit fix1first      1415    8 -77843.09 156104.49 157203.55    189
## 
## Thresholds:
##           Estimate Std. Error z value  Pr(>|z|)    
## year1 A|B 0.000000   0.000000      NA        NA    
## year1 B|C 0.984647   0.025802  38.162 < 2.2e-16 ***
## year1 C|D 2.364711   0.039873  59.306 < 2.2e-16 ***
## year1 D|E 3.728002   0.055724  66.901 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Coefficients:
##                  Estimate  Std. Error  z value  Pr(>|z|)    
## (Intercept) 1  1.42471225  0.04556961  31.2645 < 2.2e-16 ***
## (Intercept) 2  1.49164394  0.05786976  25.7759 < 2.2e-16 ***
## LR 1           0.02142909  0.00054203  39.5346 < 2.2e-16 ***
## LR 2           0.02959425  0.00096574  30.6442 < 2.2e-16 ***
## LEV 1          0.01114252  0.00052558  21.2004 < 2.2e-16 ***
## LEV 2          0.01390128  0.00081658  17.0238 < 2.2e-16 ***
## PR 1          -0.87154954  0.03320032 -26.2512 < 2.2e-16 ***
## PR 2          -0.67501624  0.04542960 -14.8585 < 2.2e-16 ***
## RSIZE 1       -0.34752657  0.00995679 -34.9035 < 2.2e-16 ***
## RSIZE 2       -0.35102290  0.01354452 -25.9162 < 2.2e-16 ***
## BETA 1         0.04802612  0.02197463   2.1855  0.028850 *  
## BETA 2         0.08627324  0.03090444   2.7916  0.005245 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error Structure:
##              Estimate Std. Error  z value  Pr(>|z|)    
## (Intercept)  1.408874   0.054179  26.0042 < 2.2e-16 ***
## BSECBSEC2   -0.487134   0.071649  -6.7989 1.054e-11 ***
## BSECBSEC3   -0.055125   0.064215  -0.8585   0.39064    
## BSECBSEC4   -0.108108   0.062361  -1.7336   0.08299 .  
## BSECBSEC5   -0.069888   0.079575  -0.8783   0.37980    
## BSECBSEC6   -0.599137   0.069668  -8.5999 < 2.2e-16 ***
## BSECBSEC7   -0.764239   0.067277 -11.3597 < 2.2e-16 ***
## BSECBSEC8   -0.653992   0.078939  -8.2848 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The default error structure output for AR(1) models:

error_structure(res_AR1_probit)
## (Intercept)   BSECBSEC2   BSECBSEC3   BSECBSEC4   BSECBSEC5   BSECBSEC6 
##  1.40887376 -0.48713415 -0.05512540 -0.10810818 -0.06988803 -0.59913737 
##   BSECBSEC7   BSECBSEC8 
## -0.76423929 -0.65399207

Correlation parameters \(\rho_i\) for each firm are obtained by choosing type = "corr" in error_structure():

head(error_structure(res_AR1_probit, type = "corr"), n = 3)
##   Correlation
## 1   0.8749351
## 2   0.6694448
## 3   0.8749351

Correlation matrices for each specific firm are obtained by choosing type = "sigmas":

head(error_structure(res_AR1_probit, type = "sigmas"), n = 1)
## $`1`
##           year1     year2     year3     year4     year5     year6     year7
## year1 1.0000000 0.8749351 0.7655115 0.6697729 0.5860078 0.5127188 0.4485957
## year2 0.8749351 1.0000000 0.8749351 0.7655115 0.6697729 0.5860078 0.5127188
## year3 0.7655115 0.8749351 1.0000000 0.8749351 0.7655115 0.6697729 0.5860078
## year4 0.6697729 0.7655115 0.8749351 1.0000000 0.8749351 0.7655115 0.6697729
## year5 0.5860078 0.6697729 0.7655115 0.8749351 1.0000000 0.8749351 0.7655115
## year6 0.5127188 0.5860078 0.6697729 0.7655115 0.8749351 1.0000000 0.8749351
## year7 0.4485957 0.5127188 0.5860078 0.6697729 0.7655115 0.8749351 1.0000000
## year8 0.3924921 0.4485957 0.5127188 0.5860078 0.6697729 0.7655115 0.8749351
##           year8
## year1 0.3924921
## year2 0.4485957
## year3 0.5127188
## year4 0.5860078
## year5 0.6697729
## year6 0.7655115
## year7 0.8749351
## year8 1.0000000

Example 5 - Self-reported health status

In this example we analyze longitudinal data on self-reported health status measured on a five point ordinal scale (5 = poor, 4 = fair, 3 = good, 2 = very good and 1 = excellent) derived from the Health and Retirement Study conducted by the University of Michigan which is available in the LMest package:

data(data_SRHS_long, package = "LMest")
str(data_SRHS_long)
## 'data.frame':    56592 obs. of  6 variables:
##  $ id       : num  1 1 1 1 1 1 1 1 2 2 ...
##  $ gender   : num  1 1 1 1 1 1 1 1 2 2 ...
##  $ race     : num  1 1 1 1 1 1 1 1 1 1 ...
##  $ education: num  3 3 3 3 3 3 3 3 5 5 ...
##  $ age      : num  56 58 60 62 64 66 68 70 54 55 ...
##  $ srhs     : num  4 4 3 3 4 3 3 3 3 3 ...

The dataset contains a self-reported health status srhs together with covariates such as ethnicity (race coded as 1 = white, 2 = black, 3 = others), gender (coded as 1 = male, 2 = female), education level (coded as 1 = high school, 2 = general educational diploma, 3 = high school graduate, 4 = some college, 5 = college and above) and age for 7074 subjects on 8 different (approximately equally spaced) time occasions.

We add a time index to the data set:

data_SRHS_long$time <- rep(1:8, length(unique(data_SRHS_long$id)))

We estimate a multivariate ordinal logit model similar to the one of the models employed by Bartolucci, Pandolfi, and Pennoni (2017), where for every subject in the sample the errors follow an \(AR(1)\) process. Moreover, the threshold and regression coefficients are equal across all years. In order to reduce the computational burden we only consider pairs of observations not more than two time points appart by setting PL.lag = 2.

res_srhs <- mvord(formula = MMO(srhs, id, time) ~ 0 + factor(gender) +
    factor(race) + factor(education) + age, 
    data = data_SRHS_long,
    threshold.constraints = rep(1, 8), 
    coef.constraints = rep(1, 8),
    error.structure = cor_ar1(~ 1), link = mvlogit(), 
    PL.lag = 2)

(runtime 11 minutes).

The persistence in the reported health status is high. The correlation parameter in the cor_ar1 error structure is

unique(error_structure(res_srhs, type = "corr"))
##   Correlation
## 1   0.7621568

The estimated parameters are shown by the function summary():

summary(res_srhs, call = FALSE)
## Formula: MMO(srhs, id, time) ~ 0 + factor(gender) + factor(race) + factor(education) + 
##     age
## 
##    link threshold nsubjects ndim      logPL     CLAIC     CLBIC fevals
## mvlogit  flexible      7074    8 -236814.26 473845.34 474589.46    416
## 
## Thresholds:
##       Estimate Std. Error  z value Pr(>|z|)    
## 1 1|2 -1.78171    0.12730 -13.9958   <2e-16 ***
## 1 2|3 -0.10283    0.12866  -0.7993   0.4241    
## 1 3|4  1.42587    0.13019  10.9525   <2e-16 ***
## 1 4|5  3.05552    0.13322  22.9364   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Coefficients:
##                        Estimate Std. Error  z value  Pr(>|z|)    
## factor(gender)2 1    -0.0042032  0.0346822  -0.1212    0.9035    
## factor(race)2 1       0.6131727  0.0504085  12.1641 < 2.2e-16 ***
## factor(race)3 1       0.5060946  0.0897595   5.6383 1.717e-08 ***
## factor(education)2 1 -0.5348146  0.0803199  -6.6586 2.765e-11 ***
## factor(education)3 1 -1.0149545  0.0465579 -21.7998 < 2.2e-16 ***
## factor(education)4 1 -1.3080628  0.0526356 -24.8513 < 2.2e-16 ***
## factor(education)5 1 -1.7428100  0.0542998 -32.0961 < 2.2e-16 ***
## age 1                 0.0162628  0.0019100   8.5146 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error Structure:
##              Estimate Std. Error z value  Pr(>|z|)    
## (Intercept) 1.0013411  0.0084935   117.9 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

In the logit model the coefficients can be interpreted in terms of log-odds ratios. The results suggest that being non-white increases the chances of reporting a worse health status while higher people with education levels tend to report better health. Moreover, every additional year increases the odds of reporting a worse health status by 1.64% (\(exp(0.0162628)=1.016396\)).

Example 6 - Multirater agreement data

The mvord package can also be employed to measure agreement among several rating sources either with or without additional relevant covariate information.

We use the multirater agreement data from Chapter 5 in Johnson and Albert (1999). These data consist of grades assigned to 198 essays by 5 experts, each of whom rated all essays on a 10-point scale. A score of 10 indicates an excellent essay. In addition, the average word length is also available as an essay characteristic.

N <- "http://www-math.bgsu.edu/~albert/ord_book/Chapter5/essay_data/N.dat"
X <- "http://www-math.bgsu.edu/~albert/ord_book/Chapter5/essay_data/X.dat"
y  <- read.delim(url(N), header = F, sep = "")
wl <- read.delim(url(X), header = F, sep = "")[,2]
df <- cbind.data.frame(y, wl)
colnames(df)[1:5] <- paste0("Judge", 1:5)

The traditional measure of agreement between raters in the social sciences is the polychoric correlation. The polychoric correlation can be assessed using the mvord package by estimating of a model with no covariates and probit link. The correlation parameters of the cor_general error structure are to be interpretated as the measure of agreement.

head(df)
##   Judge1 Judge2 Judge3 Judge4 Judge5    wl
## 1      8      6      9     10      5 4.760
## 2      7      5      3      5      3 4.244
## 3      2      1      1      1      1 4.087
## 4      5      3      4      5      3 4.355
## 5      7      2      7      7      7 4.306
## 6     10      6      8     10      6 4.508

The data is in the wide format (each ordinal response is in one column) so the MMO2 object will be used in the formula object:

res_essay_0 <- mvord(
  formula = MMO2(Judge1, Judge2, Judge3, Judge4, Judge5) ~ -1,
  data = df, threshold.constraints = rep(1, 5),
  coef.constraints = rep(1, 5))

(runtime 19 seconds).

summary(res_essay_0, call = FALSE)
## Formula: MMO2(Judge1, Judge2, Judge3, Judge4, Judge5) ~ -1
## 
##     link threshold nsubjects ndim    logPL    CLAIC    CLBIC fevals
## mvprobit  flexible       198    5 -8528.42 17172.01 17361.38   1661
## 
## Thresholds:
##                Estimate  Std. Error  z value  Pr(>|z|)    
## Judge1 1|2  -1.24483406  0.07950626 -15.6571 < 2.2e-16 ***
## Judge1 2|3  -0.66341394  0.06768901  -9.8009 < 2.2e-16 ***
## Judge1 3|4  -0.32971161  0.06516921  -5.0593 4.208e-07 ***
## Judge1 4|5   0.00013865  0.06328033   0.0022    0.9983    
## Judge1 5|6   0.34966460  0.06546890   5.3409 9.247e-08 ***
## Judge1 6|7   0.63528723  0.06688036   9.4989 < 2.2e-16 ***
## Judge1 7|8   0.99375222  0.07409651  13.4116 < 2.2e-16 ***
## Judge1 8|9   1.33448377  0.08063648  16.5494 < 2.2e-16 ***
## Judge1 9|10  1.91969997  0.10649013  18.0270 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Coefficients:
##      Estimate Std. Error z value Pr(>|z|)
## 
## Error Structure:
##                    Estimate Std. Error z value  Pr(>|z|)    
## corr Judge1 Judge2 0.476079   0.061835  7.6992 1.370e-14 ***
## corr Judge1 Judge3 0.569194   0.046682 12.1930 < 2.2e-16 ***
## corr Judge1 Judge4 0.645197   0.046595 13.8468 < 2.2e-16 ***
## corr Judge1 Judge5 0.322529   0.090318  3.5710 0.0003556 ***
## corr Judge2 Judge3 0.396377   0.056243  7.0475 1.821e-12 ***
## corr Judge2 Judge4 0.634421   0.046930 13.5186 < 2.2e-16 ***
## corr Judge2 Judge5 0.621900   0.041815 14.8725 < 2.2e-16 ***
## corr Judge3 Judge4 0.391911   0.072272  5.4227 5.871e-08 ***
## corr Judge3 Judge5 0.382999   0.066020  5.8012 6.582e-09 ***
## corr Judge4 Judge5 0.693040   0.041728 16.6087 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The word length can be included as a covariate in the model:

res_essay_wl <- mvord(
  formula = MMO2(Judge1, Judge2, Judge3, Judge4, Judge5) ~ 0 + wl,
  data = df, threshold.constraints = rep(1, 5),
  coef.constraints = rep(1, 5))

(runtime 1 minutes).

summary(res_essay_wl, call = FALSE)
## Formula: MMO2(Judge1, Judge2, Judge3, Judge4, Judge5) ~ 0 + wl
## 
##     link threshold nsubjects ndim    logPL    CLAIC    CLBIC fevals
## mvprobit  flexible       198    5 -8488.74 17110.73 17329.81   2164
## 
## Thresholds:
##             Estimate Std. Error z value  Pr(>|z|)    
## Judge1 1|2   1.09799    0.78039  1.4070 0.1594326    
## Judge1 2|3   1.68728    0.78221  2.1571 0.0309994 *  
## Judge1 3|4   2.02523    0.78287  2.5869 0.0096838 ** 
## Judge1 4|5   2.36059    0.78294  3.0150 0.0025694 ** 
## Judge1 5|6   2.71667    0.78472  3.4620 0.0005362 ***
## Judge1 6|7   3.00649    0.78550  3.8275 0.0001295 ***
## Judge1 7|8   3.36963    0.78891  4.2712 1.944e-05 ***
## Judge1 8|9   3.71636    0.79304  4.6862 2.783e-06 ***
## Judge1 9|10  4.31272    0.79820  5.4031 6.551e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Coefficients:
##      Estimate Std. Error z value Pr(>|z|)   
## wl 1  0.54193    0.17837  3.0382  0.00238 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Error Structure:
##                    Estimate Std. Error z value  Pr(>|z|)    
## corr Judge1 Judge2 0.459154   0.062913  7.2983 2.915e-13 ***
## corr Judge1 Judge3 0.558919   0.047414 11.7880 < 2.2e-16 ***
## corr Judge1 Judge4 0.631977   0.048175 13.1182 < 2.2e-16 ***
## corr Judge1 Judge5 0.295952   0.089837  3.2943 0.0009866 ***
## corr Judge2 Judge3 0.382554   0.056902  6.7230 1.780e-11 ***
## corr Judge2 Judge4 0.621086   0.048802 12.7266 < 2.2e-16 ***
## corr Judge2 Judge5 0.608996   0.043499 14.0004 < 2.2e-16 ***
## corr Judge3 Judge4 0.374733   0.073984  5.0651 4.082e-07 ***
## corr Judge3 Judge5 0.368174   0.067220  5.4771 4.323e-08 ***
## corr Judge4 Judge5 0.681338   0.044116 15.4442 < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The probabilities of agreement among all five judges which implied by the model can be computed by the function joint_probabilities():

agree_prob_list <- lapply(1:10, function(i)
  joint_probabilities(res_essay_wl, rep(i, 5)))
agree_prob <- Reduce("+", agree_prob_list)
summary(agree_prob)
##     Min.  1st Qu.   Median     Mean  3rd Qu.     Max. 
## 0.005150 0.006445 0.008724 0.009701 0.011795 0.023869

In order to assess the relationship between the average word length and the agreement probabilities, we plot the probabilities of agreement implied by the model against the word length.

plot(df$wl, agree_prob,
  xlab = "word length", ylab = "probability of agreement")

The graphic suggests that the judges tend to agree more on the quality of essays with lower average word length than on the essays with larger average word length.

References

Bartolucci, Francesco, Silvia Pandolfi, and Fulvia Pennoni. 2017. “LMest: An R Package for Latent Markov Models for Longitudinal Categorical Data.” Journal of Statistical Software 81 (4): 1–38. doi:10.18637/jss.v081.i04.

Johnson, Valen E, and J Albert. 1999. Ordinal Data Modeling. Statistics for Social Science and Public Policy. Springer-Verlag New York Incorporated.