1 Missingness

1.1 Why is it missing?

  • MCAR (Missing Completely At Random): “There’s no relationship between whether a data point is missing and any values in the data set, missing or observed.”

  • MAR (Missing At Random): “means the propensity for a data point to be missing is not related to the missing data, but it is related to some of the observed data”. In other words, the response is missing because of another question you asked. You asked me if I like spiders, I said no. Next you ask me if I have a pet spider. I dont respond.

  • MNAR (Missing Not At Random): “means the propensity for a data point to be missing is related to the missing data”. You have no idea why the response/question is missing, thus you cannot infer it.

Best MCAR. Worst is MNAR.

1.2 How big is my data set

  • Normal to Huge data set: Losing 5-10% of your data is not a problem. Move on in life. If larger, then you have to start thinking about missing analysis and replacement.
  • Very small data set (5-10 people per IV): Tea leaf reading or voodoo.

2 Common methods to Impute Missing data

  • Impute: “assign (a value) to something by inference from the value of the products or processes to which it contributes”.

2.1 Classical Methods (assumining MCAR)

  • Listwise deletion: Removes whole subject is one data point is missing. Easy to implement, but losses alot of data.
  • Pairwise deletion: Removes just that case of that subjects data (keeps most data). Can be to implement (can mess up the error term or make DF differ from model to model), but less data is lost.
  • Mean imputation: Fill in the missing data points with the mean of that variable. Lowers variance overall and keeps subject. No data loss, but you are affecting the error term and keeping the DF. This can create problems in very small samples.
  • Conditional mean imputation: Replaces missing values with the predicted scores from a linear regression equation. [Very complicated when MAR is an issue]

  • Most common in psychology tend to be Mean or Pair- or listwise (SPSS defaults depending on analysis. R defaults to listwise in most cases)

  • For a review of the theory see: Peugh, J. L., & Enders, C. K. (2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of Educational Research, 74, 525-556.

2.1.1 Classical Methods

  • We will do the listwise and mean replacement types (the conditions means can be done with the mice package we well)
  • Simulate a simple regression - remove cases completely at random and test

2.1.1.1 Typical Sample Size

library(car)
set.seed(42)
n <- 50
# IVs
X <- runif(n, -10, 10)
Z <- rnorm(n, -10, 10)
# Our equation to  create Y
Y <- .8*X - .4*Z + 2 + rnorm(n, sd=10)
#Built our data frame
DataSet1<-data.frame(DV=Y,IV1=X,IV2=Z)
  • Remove IVs and DVs at random (ampute) at about 50%
library(mice)
set.seed(666)
Amputed.Data<-ampute(data =DataSet1, prop = 0.5, mech = "MCAR")
DataSet1.M<-Amputed.Data$amp
  • Lets visualize the missingness
library(VIM)
aggr(DataSet1.M, col=c('navyblue','red'), numbers=TRUE, 
     sortVars=TRUE, labels=names(data), cex.axis=.7, 
     gap=3, 
     ylab=c("Histogram of missing data","Pattern"))

## 
##  Variables sorted by number of missings: 
##  Variable Count
##       IV1  0.24
##       IV2  0.20
##        DV  0.14
  • Lets visualize pairwise scatterplots based on where the missing data seems to be
marginplot(DataSet1.M[c(1,2)])

marginplot(DataSet1.M[c(2,3)])

marginplot(DataSet1.M[c(1,3)])

  • Run the regression of the original and missing data side by side
  • R defaults to listwise deletion (most conservative approach)
  • Pairwise-deletion regression is not build into R by default (but there is is pairwise deletion for other methods)
  • Original, Listwise, mean imputed (using the mice package)
Orginal<-lm(DV~IV1+IV2, data= DataSet1)
Listwise<-lm(DV~IV1+IV2, data= DataSet1.M)

# Mean impute using MICE 
DataSet1.MM<- complete(mice(DataSet1.M, meth='mean',printFlag = FALSE))
MeanImpute<-lm(DV~IV1+IV2, data= DataSet1.MM)

library(stargazer)
stargazer(Orginal,Listwise,MeanImpute,type="html",
          column.labels = c("Orginal", "Listwise","Mean Impute"),
          intercept.bottom = FALSE,
          single.row=FALSE, 
          notes.append = FALSE,
          header=FALSE)
Dependent variable:
DV
Orginal Listwise Mean Impute
(1) (2) (3)
Constant 2.612 5.068 3.370*
(1.909) (3.138) (1.841)
IV1 0.133 0.030 0.021
(0.209) (0.333) (0.225)
IV2 -0.428*** -0.267 -0.416***
(0.131) (0.197) (0.132)
Observations 50 21 50
R2 0.192 0.098 0.176
Adjusted R2 0.158 -0.002 0.140
Residual Std. Error 8.876 (df = 47) 9.336 (df = 18) 8.223 (df = 47)
F Statistic 5.590*** (df = 2; 47) 0.980 (df = 2; 18) 5.004** (df = 2; 47)
Note: p<0.1; p<0.05; p<0.01

2.1.1.2 Huge Data Set

  • N = 500
set.seed(42)
n <- 500
# IVs
X <- runif(n, -10, 10)
Z <- rnorm(n, -10, 10)

# Our equation to  create Y
Y <- .8*X - .4*Z + 2 + rnorm(n, sd=10)

#Built our data frame
DataSet2<-data.frame(DV=Y,IV1=X,IV2=Z)

#remove cases at random 
set.seed(666)
Amputed.Data.2<-ampute(data =DataSet2, prop = 0.5, mech = "MCAR")
DataSet2.M<-Amputed.Data.2$amp
Orginal.2<-lm(DV~IV1+IV2, data= DataSet2)
Listwise.2<-lm(DV~IV1+IV2, data= DataSet2.M)

# Mean impute using MICE 
DataSet2.MM<- complete(mice(DataSet2.M, meth='mean',printFlag = FALSE))
MeanImpute.2<-lm(DV~IV1+IV2, data= DataSet2.MM)

stargazer(Orginal.2,Listwise.2,MeanImpute.2,type="html",
          column.labels = c("Orginal", "Listwise","Mean Impute"),
          intercept.bottom = FALSE,
          single.row=FALSE, 
          notes.append = FALSE,
          header=FALSE)
Dependent variable:
DV
Orginal Listwise Mean Impute
(1) (2) (3)
Constant 2.249*** 1.769* 3.066***
(0.680) (0.969) (0.692)
IV1 0.817*** 0.839*** 0.700***
(0.079) (0.111) (0.084)
IV2 -0.378*** -0.392*** -0.285***
(0.047) (0.068) (0.050)
Observations 500 263 500
R2 0.258 0.250 0.168
Adjusted R2 0.255 0.244 0.165
Residual Std. Error 10.368 (df = 497) 10.390 (df = 260) 10.066 (df = 497)
F Statistic 86.263*** (df = 2; 497) 43.386*** (df = 2; 260) 50.286*** (df = 2; 497)
Note: p<0.1; p<0.05; p<0.01

2.1.1.3 Tiny Sample Size

  • N = 15
set.seed(42)
n <- 15
# IVs
X <- runif(n, -10, 10)
Z <- rnorm(n, -10, 10)

# Our equation to  create Y
Y <- .8*X - .4*Z + 2 + rnorm(n, sd=10)

#Built our data frame
DataSet3<-data.frame(DV=Y,IV1=X,IV2=Z)

#remove cases at random 
set.seed(666)
Amputed.Data.3<-ampute(data =DataSet3, prop = 0.5, mech = "MCAR")
DataSet3.M<-Amputed.Data.3$amp
Orginal.3<-lm(DV~IV1+IV2, data= DataSet3)
Listwise.3<-lm(DV~IV1+IV2, data= DataSet3.M)

# Mean impute using MICE 
DataSet3.MM<- complete(mice(DataSet3.M, meth='mean',printFlag = FALSE))
MeanImpute.3<-lm(DV~IV1+IV2, data= DataSet3.MM)

stargazer(Orginal.3,Listwise.3,MeanImpute.3,type="html",
          column.labels = c("Orginal", "Listwise","Mean Impute"),
          intercept.bottom = FALSE,
          single.row=FALSE, 
          notes.append = FALSE,
          header=FALSE)
Dependent variable:
DV
Orginal Listwise Mean Impute
(1) (2) (3)
Constant 3.822 1.437 0.169
(3.024) (8.258) (3.093)
IV1 1.310** 0.892 1.157*
(0.526) (1.249) (0.548)
IV2 -0.225 -0.163 -0.703**
(0.272) (1.212) (0.267)
Observations 15 7 15
R2 0.405 0.121 0.429
Adjusted R2 0.306 -0.319 0.334
Residual Std. Error 9.768 (df = 12) 13.827 (df = 4) 8.786 (df = 12)
F Statistic 4.081** (df = 2; 12) 0.274 (df = 2; 4) 4.506** (df = 2; 12)
Note: p<0.1; p<0.05; p<0.01

2.1.1.4 Typical Samples with MNAR

  • N = 50
#remove cases at random from our orginal data set
set.seed(666)
Amputed.Data.4<-ampute(data =DataSet1, prop = 0.5, mech = "MNAR")
DataSet4.M<-Amputed.Data.4$amp
Orginal.4<-lm(DV~IV1+IV2, data= DataSet1)
Listwise.4<-lm(DV~IV1+IV2, data= DataSet4.M)

# Mean impute using MICE 
DataSet4.MM<- complete(mice(DataSet4.M, meth='mean',printFlag = FALSE))
MeanImpute.4<-lm(DV~IV1+IV2, data= DataSet4.MM)

stargazer(Orginal.4,Listwise.4,MeanImpute.4,type="html",
          column.labels = c("Orginal", "Listwise","Mean Impute"),
          intercept.bottom = FALSE,
          single.row=FALSE, 
          notes.append = FALSE,
          header=FALSE)
Dependent variable:
DV
Orginal Listwise Mean Impute
(1) (2) (3)
Constant 2.612 1.071 2.196
(1.909) (2.621) (1.714)
IV1 0.133 -0.185 -0.008
(0.209) (0.327) (0.211)
IV2 -0.428*** -0.412** -0.390***
(0.131) (0.167) (0.118)
Observations 50 24 50
R2 0.192 0.232 0.189
Adjusted R2 0.158 0.159 0.155
Residual Std. Error 8.876 (df = 47) 9.254 (df = 21) 7.816 (df = 47)
F Statistic 5.590*** (df = 2; 47) 3.172* (df = 2; 21) 5.477*** (df = 2; 47)
Note: p<0.1; p<0.05; p<0.01

2.1.1.5 Typical Samples with MAR

  • N = 50
#remove cases from our orginal data set
set.seed(666)
Amputed.Data.5<-ampute(data =DataSet1, prop = 0.5, mech = "MAR")
DataSet5.M<-Amputed.Data.5$amp
Orginal.5<-lm(DV~IV1+IV2, data= DataSet1)
Listwise.5<-lm(DV~IV1+IV2, data= DataSet5.M)

# Mean impute using MICE 
DataSet5.MM<- complete(mice(DataSet5.M, meth='mean',printFlag = FALSE))
MeanImpute.5<-lm(DV~IV1+IV2, data= DataSet5.MM)

stargazer(Orginal.5,Listwise.5,MeanImpute.5,type="html",
          column.labels = c("Orginal", "Listwise","Mean Impute"),
          intercept.bottom = FALSE,
          single.row=FALSE, 
          notes.append = FALSE,
          header=FALSE)
Dependent variable:
DV
Orginal Listwise Mean Impute
(1) (2) (3)
Constant 2.612 -1.810 2.752
(1.909) (2.196) (2.042)
IV1 0.133 0.032 0.102
(0.209) (0.244) (0.236)
IV2 -0.428*** -0.491*** -0.387**
(0.131) (0.141) (0.148)
Observations 50 27 50
R2 0.192 0.339 0.128
Adjusted R2 0.158 0.284 0.090
Residual Std. Error 8.876 (df = 47) 7.364 (df = 24) 8.586 (df = 47)
F Statistic 5.590*** (df = 2; 47) 6.158*** (df = 2; 24) 3.437** (df = 2; 47)
Note: p<0.1; p<0.05; p<0.01

2.1.2 Notes

  • Mean imputing works OK in medium to large sample sizes
  • But it creates large bias in small samples
  • If you don’t have MCAR you have to turn to more complex methods

2.2 Modern approaches

  • There are many modern approaches often based on maximum likelihood estimation (ML) or Multiple Imputation (MI) or even Bayesian approaches
  • “ML estimation is to identify the population parameter values most likely to have produced a particular sample of data. This usually requires an iterative process whereby the model fitting program”tries out" different values for the parameters of interest (e.g., regression coefficients) en route to identifying the values most likely to have produced the sample data." (Peugh & Enders, 2004)
  • Multiple Imputation: “Rather than treating a single set of imputed values as”true" estimates of the missing values, MI creates a number of imputed data sets (frequently between 5 and 10), each of which contains a different plausible estimate of the missing values." (Peugh & Enders, 2004)
  • Lets test out MI as its the newest and considered an improvement over ML

2.3 Multiple Imputation by Chained Equations

  • See Azur et al, 2011 for details
  • MICE package in R: Lots of old and modern algorithms to choose from (also for design beyond linear regression)
  • Predictive mean matching is a good version to use as it “is a general purpose semi-parametric imputation method. … imputations are restricted to the observed values and that it can preserve non-linear relations…” (Buuren & Groothuis-Oudshoorn, 2011)
  • We will work from our MAR missing Dataset5.M
  • We can compare this to our classical methods and some other methods which the package allows (Bootstapping & Bayesian)

2.3.1 Typical sample size

#Linear regression using bootstrap
DataDataSet5.Boot<- complete(mice(DataSet5.M, m = 10,meth='norm.boot',printFlag = FALSE, seed = 666))
Boot<-lm(DV~IV1+IV2, data= DataDataSet5.Boot)

#Bayesian linear regression
DataDataSet5.BLR<- complete(mice(DataSet5.M, m = 10,meth='norm',printFlag = FALSE, seed = 666))
BLR<-lm(DV~IV1+IV2, data= DataDataSet5.BLR)

#Predictive mean matching 
DataDataSet5.pmm<- complete(mice(DataSet5.M, m = 10,meth='pmm',printFlag = FALSE, seed = 666))
PMM<-lm(DV~IV1+IV2, data= DataDataSet5.pmm)

stargazer(Orginal.5, Boot, BLR, PMM,type="html",
          column.labels = c("Orginal","Boot","BLR","PMM"),
          intercept.bottom = FALSE,
          single.row=FALSE, 
          notes.append = FALSE,
          header=FALSE)
Dependent variable:
DV
Orginal Boot BLR PMM
(1) (2) (3) (4)
Constant 2.612 2.198 2.146 2.467
(1.909) (1.938) (2.088) (2.099)
IV1 0.133 0.356 0.133 0.187
(0.209) (0.223) (0.224) (0.239)
IV2 -0.428*** -0.440*** -0.361*** -0.392***
(0.131) (0.134) (0.134) (0.143)
Observations 50 50 50 50
R2 0.192 0.203 0.136 0.140
Adjusted R2 0.158 0.169 0.099 0.104
Residual Std. Error (df = 47) 8.876 8.544 9.088 9.041
F Statistic (df = 2; 47) 5.590*** 5.970*** 3.702** 3.839**
Note: p<0.1; p<0.05; p<0.01
  • Predictive mean matching is the default of this program and you see it does a good job

2.3.2 Small sample size

  • Use the same exact small data (N = 15)
#Linear regression using bootstrap
DataDataSet3.Boot<- complete(mice(DataSet3.M, m = 15,meth='norm.boot',printFlag = FALSE, seed = 666))
Boot.3<-lm(DV~IV1+IV2, data= DataDataSet3.Boot)

#Bayesian linear regression
DataDataSet3.BLR<- complete(mice(DataSet3.M, m = 15,meth='norm',printFlag = FALSE, seed = 666))
BLR.3<-lm(DV~IV1+IV2, data= DataDataSet3.BLR)

#Predictive mean matching 
DataDataSet3.pmm<- complete(mice(DataSet3.M, m = 15,meth='pmm',printFlag = FALSE, seed = 666))
PMM.3<-lm(DV~IV1+IV2, data= DataDataSet3.pmm)

stargazer(Orginal.3, Boot.3, BLR.3, PMM.3,type="html",
          column.labels = c("Orginal","Boot","BLR","PMM"),
          intercept.bottom = FALSE,
          single.row=FALSE, 
          notes.append = FALSE,
          header=FALSE)
Dependent variable:
DV
Orginal Boot BLR PMM
(1) (2) (3) (4)
Constant 3.822 -2.926 1.351 -1.584
(3.024) (4.232) (3.624) (3.647)
IV1 1.310** 1.352* 1.299** 0.971
(0.526) (0.700) (0.541) (0.581)
IV2 -0.225 -1.026** -0.467 -0.991**
(0.272) (0.383) (0.274) (0.365)
Observations 15 15 15 15
R2 0.405 0.375 0.378 0.382
Adjusted R2 0.306 0.270 0.274 0.278
Residual Std. Error (df = 12) 9.768 9.319 9.680 9.394
F Statistic (df = 2; 12) 4.081** 3.595* 3.646* 3.701*
Note: p<0.1; p<0.05; p<0.01
  • We need to find practicer of the dark arts, as none of the methods did a good job

2.3.2.1 Notes

  • You have to manually set the number of iterations (number of passes)
  • You will have to research each algorithm for your particular data missingness

3 Best Practices

  • Each situation is different and which method you use depends on many factors
  • Also you must think carefully about the assumptions you are making
  • For example, if you mean (or PMM) replace you are assuming the data are normal
  • Thee are other advanced methods for specific situation: “hot decking” for survey research; nearest neighbor, splines, or autoregressive methods for time-series or longitudinal data
  • Best practice: compare serval logical replacement methods and also compare it to listwise removal. If the story you are trying to tell is very different based each method and very different as it relates to the listwise removal it suggests you have not factored in some assumption correctly and best to rethink what you are doing.
LS0tDQp0aXRsZTogJ01pc3NpbmcgRGF0YScNCm91dHB1dDoNCiAgaHRtbF9kb2N1bWVudDoNCiAgICBjb2RlX2Rvd25sb2FkOiB5ZXMNCiAgICBmb250c2l6ZTogOHB0DQogICAgaGlnaGxpZ2h0OiB0ZXh0bWF0ZQ0KICAgIG51bWJlcl9zZWN0aW9uczogeWVzDQogICAgdGhlbWU6IGZsYXRseQ0KICAgIHRvYzogeWVzDQogICAgdG9jX2Zsb2F0Og0KICAgICAgY29sbGFwc2VkOiBubw0KDQotLS0NCg0KDQpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0NCmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSkNCmtuaXRyOjpvcHRzX2NodW5rJHNldChtZXNzYWdlID0gRkFMU0UpDQprbml0cjo6b3B0c19jaHVuayRzZXQoZmlnLndpZHRoPTUpDQprbml0cjo6b3B0c19jaHVuayRzZXQoZmlnLmhlaWdodD0zLjc1KQ0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGZpZy5hbGlnbj0nY2VudGVyJykgDQpgYGANCg0KDQojIE1pc3NpbmduZXNzDQoNCg0KIyMgV2h5IGlzIGl0IG1pc3Npbmc/DQotIE1DQVIgKE1pc3NpbmcgQ29tcGxldGVseSBBdCBSYW5kb20pOiAgIlRoZXJlJ3Mgbm8gcmVsYXRpb25zaGlwIGJldHdlZW4gd2hldGhlciBhIGRhdGEgcG9pbnQgaXMgbWlzc2luZyBhbmQgYW55IHZhbHVlcyBpbiB0aGUgZGF0YSBzZXQsIG1pc3Npbmcgb3Igb2JzZXJ2ZWQuIiANCg0KLSBNQVIgKE1pc3NpbmcgQXQgUmFuZG9tKTogIm1lYW5zIHRoZSBwcm9wZW5zaXR5IGZvciBhIGRhdGEgcG9pbnQgdG8gYmUgbWlzc2luZyBpcyBub3QgcmVsYXRlZCB0byB0aGUgbWlzc2luZyBkYXRhLCBidXQgaXQgaXMgcmVsYXRlZCB0byBzb21lIG9mIHRoZSBvYnNlcnZlZCBkYXRhIi4gSW4gb3RoZXIgd29yZHMsIHRoZSByZXNwb25zZSBpcyBtaXNzaW5nIGJlY2F1c2Ugb2YgYW5vdGhlciBxdWVzdGlvbiB5b3UgYXNrZWQuIFlvdSBhc2tlZCBtZSBpZiBJIGxpa2Ugc3BpZGVycywgSSBzYWlkIG5vLiBOZXh0IHlvdSBhc2sgbWUgaWYgSSBoYXZlIGEgcGV0IHNwaWRlci4gSSBkb250IHJlc3BvbmQuIA0KDQotIE1OQVIgKE1pc3NpbmcgTm90IEF0IFJhbmRvbSk6ICJtZWFucyB0aGUgcHJvcGVuc2l0eSBmb3IgYSBkYXRhIHBvaW50IHRvIGJlIG1pc3NpbmcgaXMgcmVsYXRlZCB0byB0aGUgbWlzc2luZyBkYXRhIi4gWW91IGhhdmUgbm8gaWRlYSB3aHkgdGhlIHJlc3BvbnNlL3F1ZXN0aW9uIGlzIG1pc3NpbmcsIHRodXMgeW91IGNhbm5vdCBpbmZlciBpdC4gDQoNCkJlc3QgTUNBUi4gV29yc3QgaXMgTU5BUi4gDQogDQotIFNlZTogaHR0cDovL3d3dy50aGVhbmFseXNpc2ZhY3Rvci5jb20vbWFyLWFuZC1tY2FyLW1pc3NpbmctZGF0YS8NCi0gU2VlOiBodHRwczovL3d3dy5yLWJsb2dnZXJzLmNvbS9pbXB1dGluZy1taXNzaW5nLWRhdGEtd2l0aC1yLW1pY2UtcGFja2FnZS8NCg0KIyMgSG93IGJpZyBpcyBteSBkYXRhIHNldA0KLSBOb3JtYWwgdG8gSHVnZSBkYXRhIHNldDogTG9zaW5nIDUtMTAlIG9mIHlvdXIgZGF0YSBpcyBub3QgYSBwcm9ibGVtLiBNb3ZlIG9uIGluIGxpZmUuIElmIGxhcmdlciwgdGhlbiB5b3UgaGF2ZSB0byBzdGFydCB0aGlua2luZyBhYm91dCBtaXNzaW5nIGFuYWx5c2lzIGFuZCByZXBsYWNlbWVudC4NCi0gVmVyeSBzbWFsbCBkYXRhIHNldCAoNS0xMCBwZW9wbGUgcGVyIElWKTogVGVhIGxlYWYgcmVhZGluZyBvciB2b29kb28uICANCg0KDQojIENvbW1vbiBtZXRob2RzIHRvIEltcHV0ZSBNaXNzaW5nIGRhdGENCi0gSW1wdXRlOiAiYXNzaWduIChhIHZhbHVlKSB0byBzb21ldGhpbmcgYnkgaW5mZXJlbmNlIGZyb20gdGhlIHZhbHVlIG9mIHRoZSBwcm9kdWN0cyBvciBwcm9jZXNzZXMgdG8gd2hpY2ggaXQgY29udHJpYnV0ZXMiLg0KDQojIyBDbGFzc2ljYWwgTWV0aG9kcyAoYXNzdW1pbmluZyBNQ0FSKQ0KLSBMaXN0d2lzZSBkZWxldGlvbjogUmVtb3ZlcyB3aG9sZSBzdWJqZWN0IGlzIG9uZSBkYXRhIHBvaW50IGlzIG1pc3NpbmcuIEVhc3kgdG8gaW1wbGVtZW50LCBidXQgbG9zc2VzIGFsb3Qgb2YgZGF0YS4gDQotIFBhaXJ3aXNlIGRlbGV0aW9uOiBSZW1vdmVzIGp1c3QgdGhhdCBjYXNlIG9mIHRoYXQgc3ViamVjdHMgZGF0YSAoa2VlcHMgbW9zdCBkYXRhKS4gQ2FuIGJlIHRvIGltcGxlbWVudCAoY2FuIG1lc3MgdXAgdGhlIGVycm9yIHRlcm0gb3IgbWFrZSBERiBkaWZmZXIgZnJvbSBtb2RlbCB0byBtb2RlbCksIGJ1dCBsZXNzIGRhdGEgaXMgbG9zdC4gDQotIE1lYW4gaW1wdXRhdGlvbjogRmlsbCBpbiB0aGUgbWlzc2luZyBkYXRhIHBvaW50cyB3aXRoIHRoZSBtZWFuIG9mIHRoYXQgdmFyaWFibGUuIExvd2VycyB2YXJpYW5jZSBvdmVyYWxsIGFuZCBrZWVwcyBzdWJqZWN0LiBObyBkYXRhIGxvc3MsIGJ1dCB5b3UgYXJlIGFmZmVjdGluZyB0aGUgZXJyb3IgdGVybSBhbmQga2VlcGluZyB0aGUgREYuIFRoaXMgY2FuIGNyZWF0ZSBwcm9ibGVtcyBpbiB2ZXJ5IHNtYWxsIHNhbXBsZXMuDQotIENvbmRpdGlvbmFsIG1lYW4gaW1wdXRhdGlvbjogIFJlcGxhY2VzIG1pc3NpbmcgdmFsdWVzIHdpdGggdGhlIHByZWRpY3RlZCBzY29yZXMgZnJvbSBhIGxpbmVhciByZWdyZXNzaW9uIGVxdWF0aW9uLiBbVmVyeSBjb21wbGljYXRlZCB3aGVuIE1BUiBpcyBhbiBpc3N1ZV0NCg0KLSBNb3N0IGNvbW1vbiBpbiBwc3ljaG9sb2d5IHRlbmQgdG8gYmUgTWVhbiBvciBQYWlyLSBvciBsaXN0d2lzZSAoU1BTUyBkZWZhdWx0cyBkZXBlbmRpbmcgb24gYW5hbHlzaXMuIFIgZGVmYXVsdHMgdG8gbGlzdHdpc2UgaW4gbW9zdCBjYXNlcykNCg0KLSBGb3IgYSByZXZpZXcgb2YgdGhlIHRoZW9yeSBzZWU6IFBldWdoLCBKLiBMLiwgJiBFbmRlcnMsIEMuIEsuICgyMDA0KS4gTWlzc2luZyBkYXRhIGluIGVkdWNhdGlvbmFsIHJlc2VhcmNoOiBBIHJldmlldyBvZiByZXBvcnRpbmcgcHJhY3RpY2VzIGFuZCBzdWdnZXN0aW9ucyBmb3IgaW1wcm92ZW1lbnQuIFJldmlldyBvZiBFZHVjYXRpb25hbCBSZXNlYXJjaCwgNzQsIDUyNS01NTYuDQoNCiMjIyBDbGFzc2ljYWwgTWV0aG9kcw0KLSBXZSB3aWxsIGRvIHRoZSBsaXN0d2lzZSBhbmQgbWVhbiByZXBsYWNlbWVudCB0eXBlcyAodGhlIGNvbmRpdGlvbnMgbWVhbnMgY2FuIGJlIGRvbmUgd2l0aCB0aGUgbWljZSBwYWNrYWdlIHdlIHdlbGwpDQotIFNpbXVsYXRlIGEgc2ltcGxlIHJlZ3Jlc3Npb24gLSByZW1vdmUgY2FzZXMgY29tcGxldGVseSBhdCByYW5kb20gYW5kIHRlc3QNCg0KIyMjIyBUeXBpY2FsIFNhbXBsZSBTaXplDQpgYGB7ciwgZWNobz1UUlVFLCB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlcz0gRkFMU0V9DQpsaWJyYXJ5KGNhcikNCnNldC5zZWVkKDQyKQ0KbiA8LSA1MA0KIyBJVnMNClggPC0gcnVuaWYobiwgLTEwLCAxMCkNClogPC0gcm5vcm0obiwgLTEwLCAxMCkNCiMgT3VyIGVxdWF0aW9uIHRvICBjcmVhdGUgWQ0KWSA8LSAuOCpYIC0gLjQqWiArIDIgKyBybm9ybShuLCBzZD0xMCkNCiNCdWlsdCBvdXIgZGF0YSBmcmFtZQ0KRGF0YVNldDE8LWRhdGEuZnJhbWUoRFY9WSxJVjE9WCxJVjI9WikNCg0KYGBgDQoNCi0gUmVtb3ZlIElWcyBhbmQgRFZzIGF0IHJhbmRvbSAoYW1wdXRlKSBhdCBhYm91dCA1MCUNCg0KYGBge3IsIGVjaG89VFJVRSwgd2FybmluZz1GQUxTRSxtZXNzYWdlcz0gRkFMU0V9DQpsaWJyYXJ5KG1pY2UpDQpzZXQuc2VlZCg2NjYpDQpBbXB1dGVkLkRhdGE8LWFtcHV0ZShkYXRhID1EYXRhU2V0MSwgcHJvcCA9IDAuNSwgbWVjaCA9ICJNQ0FSIikNCkRhdGFTZXQxLk08LUFtcHV0ZWQuRGF0YSRhbXANCmBgYA0KIA0KIC0gTGV0cyB2aXN1YWxpemUgdGhlIG1pc3NpbmduZXNzDQpgYGB7ciwgZWNobz1UUlVFLCB3YXJuaW5nPUZBTFNFfQ0KbGlicmFyeShWSU0pDQphZ2dyKERhdGFTZXQxLk0sIGNvbD1jKCduYXZ5Ymx1ZScsJ3JlZCcpLCBudW1iZXJzPVRSVUUsIA0KICAgICBzb3J0VmFycz1UUlVFLCBsYWJlbHM9bmFtZXMoZGF0YSksIGNleC5heGlzPS43LCANCiAgICAgZ2FwPTMsIA0KICAgICB5bGFiPWMoIkhpc3RvZ3JhbSBvZiBtaXNzaW5nIGRhdGEiLCJQYXR0ZXJuIikpDQpgYGANCiANCiANCiAtIExldHMgdmlzdWFsaXplIHBhaXJ3aXNlIHNjYXR0ZXJwbG90cyBiYXNlZCBvbiB3aGVyZSB0aGUgbWlzc2luZyBkYXRhIHNlZW1zIHRvIGJlDQogDQpgYGB7ciwgZWNobz1UUlVFLCB3YXJuaW5nPUZBTFNFfQ0KbWFyZ2lucGxvdChEYXRhU2V0MS5NW2MoMSwyKV0pDQptYXJnaW5wbG90KERhdGFTZXQxLk1bYygyLDMpXSkNCm1hcmdpbnBsb3QoRGF0YVNldDEuTVtjKDEsMyldKQ0KYGBgDQoNCi0gUnVuIHRoZSByZWdyZXNzaW9uIG9mIHRoZSBvcmlnaW5hbCBhbmQgbWlzc2luZyBkYXRhIHNpZGUgYnkgc2lkZQ0KLSBSIGRlZmF1bHRzIHRvIGxpc3R3aXNlIGRlbGV0aW9uIChtb3N0IGNvbnNlcnZhdGl2ZSBhcHByb2FjaCkNCi0gUGFpcndpc2UtZGVsZXRpb24gcmVncmVzc2lvbiBpcyBub3QgYnVpbGQgaW50byBSIGJ5IGRlZmF1bHQgKGJ1dCB0aGVyZSBpcyBpcyBwYWlyd2lzZSBkZWxldGlvbiBmb3Igb3RoZXIgbWV0aG9kcykNCi0gT3JpZ2luYWwsIExpc3R3aXNlLCBtZWFuIGltcHV0ZWQgKHVzaW5nIHRoZSBtaWNlIHBhY2thZ2UpDQogDQpgYGB7ciwgZWNobz1UUlVFLCB3YXJuaW5nPUZBTFNFLG1lc3NhZ2U9RkFMU0UscmVzdWx0cz0nYXNpcyd9DQpPcmdpbmFsPC1sbShEVn5JVjErSVYyLCBkYXRhPSBEYXRhU2V0MSkNCkxpc3R3aXNlPC1sbShEVn5JVjErSVYyLCBkYXRhPSBEYXRhU2V0MS5NKQ0KDQojIE1lYW4gaW1wdXRlIHVzaW5nIE1JQ0UgDQpEYXRhU2V0MS5NTTwtIGNvbXBsZXRlKG1pY2UoRGF0YVNldDEuTSwgbWV0aD0nbWVhbicscHJpbnRGbGFnID0gRkFMU0UpKQ0KTWVhbkltcHV0ZTwtbG0oRFZ+SVYxK0lWMiwgZGF0YT0gRGF0YVNldDEuTU0pDQoNCmxpYnJhcnkoc3RhcmdhemVyKQ0Kc3RhcmdhemVyKE9yZ2luYWwsTGlzdHdpc2UsTWVhbkltcHV0ZSx0eXBlPSJodG1sIiwNCiAgICAgICAgICBjb2x1bW4ubGFiZWxzID0gYygiT3JnaW5hbCIsICJMaXN0d2lzZSIsIk1lYW4gSW1wdXRlIiksDQogICAgICAgICAgaW50ZXJjZXB0LmJvdHRvbSA9IEZBTFNFLA0KICAgICAgICAgIHNpbmdsZS5yb3c9RkFMU0UsIA0KICAgICAgICAgIG5vdGVzLmFwcGVuZCA9IEZBTFNFLA0KICAgICAgICAgIGhlYWRlcj1GQUxTRSkNCg0KDQpgYGANCg0KIyMjIyBIdWdlIERhdGEgU2V0DQotIE4gPSA1MDANCg0KYGBge3IsIGVjaG89VFJVRSwgd2FybmluZz1GQUxTRX0NCnNldC5zZWVkKDQyKQ0KbiA8LSA1MDANCiMgSVZzDQpYIDwtIHJ1bmlmKG4sIC0xMCwgMTApDQpaIDwtIHJub3JtKG4sIC0xMCwgMTApDQoNCiMgT3VyIGVxdWF0aW9uIHRvICBjcmVhdGUgWQ0KWSA8LSAuOCpYIC0gLjQqWiArIDIgKyBybm9ybShuLCBzZD0xMCkNCg0KI0J1aWx0IG91ciBkYXRhIGZyYW1lDQpEYXRhU2V0MjwtZGF0YS5mcmFtZShEVj1ZLElWMT1YLElWMj1aKQ0KDQojcmVtb3ZlIGNhc2VzIGF0IHJhbmRvbSANCnNldC5zZWVkKDY2NikNCkFtcHV0ZWQuRGF0YS4yPC1hbXB1dGUoZGF0YSA9RGF0YVNldDIsIHByb3AgPSAwLjUsIG1lY2ggPSAiTUNBUiIpDQpEYXRhU2V0Mi5NPC1BbXB1dGVkLkRhdGEuMiRhbXANCmBgYA0KDQoNCmBgYHtyLCBlY2hvPVRSVUUsIHdhcm5pbmc9RkFMU0UsbWVzc2FnZT1GQUxTRSxyZXN1bHRzPSdhc2lzJ30NCg0KT3JnaW5hbC4yPC1sbShEVn5JVjErSVYyLCBkYXRhPSBEYXRhU2V0MikNCkxpc3R3aXNlLjI8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFTZXQyLk0pDQoNCiMgTWVhbiBpbXB1dGUgdXNpbmcgTUlDRSANCkRhdGFTZXQyLk1NPC0gY29tcGxldGUobWljZShEYXRhU2V0Mi5NLCBtZXRoPSdtZWFuJyxwcmludEZsYWcgPSBGQUxTRSkpDQpNZWFuSW1wdXRlLjI8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFTZXQyLk1NKQ0KDQpzdGFyZ2F6ZXIoT3JnaW5hbC4yLExpc3R3aXNlLjIsTWVhbkltcHV0ZS4yLHR5cGU9Imh0bWwiLA0KICAgICAgICAgIGNvbHVtbi5sYWJlbHMgPSBjKCJPcmdpbmFsIiwgIkxpc3R3aXNlIiwiTWVhbiBJbXB1dGUiKSwNCiAgICAgICAgICBpbnRlcmNlcHQuYm90dG9tID0gRkFMU0UsDQogICAgICAgICAgc2luZ2xlLnJvdz1GQUxTRSwgDQogICAgICAgICAgbm90ZXMuYXBwZW5kID0gRkFMU0UsDQogICAgICAgICAgaGVhZGVyPUZBTFNFKQ0KDQpgYGANCg0KIyMjIyBUaW55IFNhbXBsZSBTaXplDQotIE4gPSAxNQ0KDQpgYGB7ciwgZWNobz1UUlVFLCB3YXJuaW5nPUZBTFNFfQ0Kc2V0LnNlZWQoNDIpDQpuIDwtIDE1DQojIElWcw0KWCA8LSBydW5pZihuLCAtMTAsIDEwKQ0KWiA8LSBybm9ybShuLCAtMTAsIDEwKQ0KDQojIE91ciBlcXVhdGlvbiB0byAgY3JlYXRlIFkNClkgPC0gLjgqWCAtIC40KlogKyAyICsgcm5vcm0obiwgc2Q9MTApDQoNCiNCdWlsdCBvdXIgZGF0YSBmcmFtZQ0KRGF0YVNldDM8LWRhdGEuZnJhbWUoRFY9WSxJVjE9WCxJVjI9WikNCg0KI3JlbW92ZSBjYXNlcyBhdCByYW5kb20gDQpzZXQuc2VlZCg2NjYpDQpBbXB1dGVkLkRhdGEuMzwtYW1wdXRlKGRhdGEgPURhdGFTZXQzLCBwcm9wID0gMC41LCBtZWNoID0gIk1DQVIiKQ0KRGF0YVNldDMuTTwtQW1wdXRlZC5EYXRhLjMkYW1wDQpgYGANCg0KYGBge3IsIGVjaG89VFJVRSwgd2FybmluZz1GQUxTRSxtZXNzYWdlPUZBTFNFLHJlc3VsdHM9J2FzaXMnfQ0KT3JnaW5hbC4zPC1sbShEVn5JVjErSVYyLCBkYXRhPSBEYXRhU2V0MykNCkxpc3R3aXNlLjM8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFTZXQzLk0pDQoNCiMgTWVhbiBpbXB1dGUgdXNpbmcgTUlDRSANCkRhdGFTZXQzLk1NPC0gY29tcGxldGUobWljZShEYXRhU2V0My5NLCBtZXRoPSdtZWFuJyxwcmludEZsYWcgPSBGQUxTRSkpDQpNZWFuSW1wdXRlLjM8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFTZXQzLk1NKQ0KDQpzdGFyZ2F6ZXIoT3JnaW5hbC4zLExpc3R3aXNlLjMsTWVhbkltcHV0ZS4zLHR5cGU9Imh0bWwiLA0KICAgICAgICAgIGNvbHVtbi5sYWJlbHMgPSBjKCJPcmdpbmFsIiwgIkxpc3R3aXNlIiwiTWVhbiBJbXB1dGUiKSwNCiAgICAgICAgICBpbnRlcmNlcHQuYm90dG9tID0gRkFMU0UsDQogICAgICAgICAgc2luZ2xlLnJvdz1GQUxTRSwgDQogICAgICAgICAgbm90ZXMuYXBwZW5kID0gRkFMU0UsDQogICAgICAgICAgaGVhZGVyPUZBTFNFKQ0KDQpgYGANCg0KIyMjIyBUeXBpY2FsIFNhbXBsZXMgd2l0aCBNTkFSDQotIE4gPSA1MA0KDQpgYGB7ciwgZWNobz1UUlVFLCB3YXJuaW5nPUZBTFNFfQ0KI3JlbW92ZSBjYXNlcyBhdCByYW5kb20gZnJvbSBvdXIgb3JnaW5hbCBkYXRhIHNldA0Kc2V0LnNlZWQoNjY2KQ0KQW1wdXRlZC5EYXRhLjQ8LWFtcHV0ZShkYXRhID1EYXRhU2V0MSwgcHJvcCA9IDAuNSwgbWVjaCA9ICJNTkFSIikNCkRhdGFTZXQ0Lk08LUFtcHV0ZWQuRGF0YS40JGFtcA0KYGBgDQoNCg0KYGBge3IsIGVjaG89VFJVRSwgd2FybmluZz1GQUxTRSxtZXNzYWdlPUZBTFNFLHJlc3VsdHM9J2FzaXMnfQ0KDQpPcmdpbmFsLjQ8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFTZXQxKQ0KTGlzdHdpc2UuNDwtbG0oRFZ+SVYxK0lWMiwgZGF0YT0gRGF0YVNldDQuTSkNCg0KIyBNZWFuIGltcHV0ZSB1c2luZyBNSUNFIA0KRGF0YVNldDQuTU08LSBjb21wbGV0ZShtaWNlKERhdGFTZXQ0Lk0sIG1ldGg9J21lYW4nLHByaW50RmxhZyA9IEZBTFNFKSkNCk1lYW5JbXB1dGUuNDwtbG0oRFZ+SVYxK0lWMiwgZGF0YT0gRGF0YVNldDQuTU0pDQoNCnN0YXJnYXplcihPcmdpbmFsLjQsTGlzdHdpc2UuNCxNZWFuSW1wdXRlLjQsdHlwZT0iaHRtbCIsDQogICAgICAgICAgY29sdW1uLmxhYmVscyA9IGMoIk9yZ2luYWwiLCAiTGlzdHdpc2UiLCJNZWFuIEltcHV0ZSIpLA0KICAgICAgICAgIGludGVyY2VwdC5ib3R0b20gPSBGQUxTRSwNCiAgICAgICAgICBzaW5nbGUucm93PUZBTFNFLCANCiAgICAgICAgICBub3Rlcy5hcHBlbmQgPSBGQUxTRSwNCiAgICAgICAgICBoZWFkZXI9RkFMU0UpDQoNCmBgYA0KDQoNCiMjIyMgVHlwaWNhbCBTYW1wbGVzIHdpdGggTUFSDQotIE4gPSA1MA0KDQpgYGB7ciwgZWNobz1UUlVFLCB3YXJuaW5nPUZBTFNFfQ0KI3JlbW92ZSBjYXNlcyBmcm9tIG91ciBvcmdpbmFsIGRhdGEgc2V0DQpzZXQuc2VlZCg2NjYpDQpBbXB1dGVkLkRhdGEuNTwtYW1wdXRlKGRhdGEgPURhdGFTZXQxLCBwcm9wID0gMC41LCBtZWNoID0gIk1BUiIpDQpEYXRhU2V0NS5NPC1BbXB1dGVkLkRhdGEuNSRhbXANCmBgYA0KDQoNCmBgYHtyLCBlY2hvPVRSVUUsIHdhcm5pbmc9RkFMU0UsbWVzc2FnZT1GQUxTRSxyZXN1bHRzPSdhc2lzJ30NCg0KT3JnaW5hbC41PC1sbShEVn5JVjErSVYyLCBkYXRhPSBEYXRhU2V0MSkNCkxpc3R3aXNlLjU8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFTZXQ1Lk0pDQoNCiMgTWVhbiBpbXB1dGUgdXNpbmcgTUlDRSANCkRhdGFTZXQ1Lk1NPC0gY29tcGxldGUobWljZShEYXRhU2V0NS5NLCBtZXRoPSdtZWFuJyxwcmludEZsYWcgPSBGQUxTRSkpDQpNZWFuSW1wdXRlLjU8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFTZXQ1Lk1NKQ0KDQpzdGFyZ2F6ZXIoT3JnaW5hbC41LExpc3R3aXNlLjUsTWVhbkltcHV0ZS41LHR5cGU9Imh0bWwiLA0KICAgICAgICAgIGNvbHVtbi5sYWJlbHMgPSBjKCJPcmdpbmFsIiwgIkxpc3R3aXNlIiwiTWVhbiBJbXB1dGUiKSwNCiAgICAgICAgICBpbnRlcmNlcHQuYm90dG9tID0gRkFMU0UsDQogICAgICAgICAgc2luZ2xlLnJvdz1GQUxTRSwgDQogICAgICAgICAgbm90ZXMuYXBwZW5kID0gRkFMU0UsDQogICAgICAgICAgaGVhZGVyPUZBTFNFKQ0KDQpgYGANCg0KIyMjIE5vdGVzDQotIE1lYW4gaW1wdXRpbmcgd29ya3MgT0sgaW4gbWVkaXVtIHRvIGxhcmdlIHNhbXBsZSBzaXplcw0KLSBCdXQgaXQgY3JlYXRlcyBsYXJnZSBiaWFzIGluIHNtYWxsIHNhbXBsZXMNCi0gSWYgeW91IGRvbid0IGhhdmUgTUNBUiB5b3UgaGF2ZSB0byB0dXJuIHRvIG1vcmUgY29tcGxleCBtZXRob2RzIA0KDQojIyBNb2Rlcm4gYXBwcm9hY2hlcw0KLSBUaGVyZSBhcmUgbWFueSBtb2Rlcm4gYXBwcm9hY2hlcyBvZnRlbiBiYXNlZCBvbiBtYXhpbXVtIGxpa2VsaWhvb2QgZXN0aW1hdGlvbiAoTUwpIG9yIE11bHRpcGxlIEltcHV0YXRpb24gKE1JKSBvciBldmVuIEJheWVzaWFuIGFwcHJvYWNoZXMNCi0gIk1MIGVzdGltYXRpb24gaXMgdG8gaWRlbnRpZnkgdGhlIHBvcHVsYXRpb24gcGFyYW1ldGVyIHZhbHVlcyBtb3N0IGxpa2VseSB0byBoYXZlIHByb2R1Y2VkIGEgcGFydGljdWxhciBzYW1wbGUgb2YgZGF0YS4gVGhpcyB1c3VhbGx5IHJlcXVpcmVzIGFuIGl0ZXJhdGl2ZSBwcm9jZXNzIHdoZXJlYnkgdGhlIG1vZGVsIGZpdHRpbmcgcHJvZ3JhbSAidHJpZXMgb3V0IiBkaWZmZXJlbnQgdmFsdWVzIGZvciB0aGUgcGFyYW1ldGVycyBvZiBpbnRlcmVzdCAoZS5nLiwgcmVncmVzc2lvbiBjb2VmZmljaWVudHMpIGVuIHJvdXRlIHRvIGlkZW50aWZ5aW5nIHRoZSB2YWx1ZXMgbW9zdCBsaWtlbHkgdG8gaGF2ZSBwcm9kdWNlZCB0aGUgc2FtcGxlIGRhdGEuIiAoUGV1Z2ggJiBFbmRlcnMsIDIwMDQpDQotIE11bHRpcGxlIEltcHV0YXRpb246ICJSYXRoZXIgdGhhbiB0cmVhdGluZyBhIHNpbmdsZSBzZXQgb2YgaW1wdXRlZCB2YWx1ZXMgYXMgInRydWUiIGVzdGltYXRlcyBvZiB0aGUgbWlzc2luZyB2YWx1ZXMsIE1JIGNyZWF0ZXMgYSBudW1iZXIgb2YgaW1wdXRlZCBkYXRhIHNldHMgKGZyZXF1ZW50bHkgYmV0d2VlbiA1IGFuZCAxMCksIGVhY2ggb2Ygd2hpY2ggY29udGFpbnMgYSBkaWZmZXJlbnQgcGxhdXNpYmxlIGVzdGltYXRlIG9mIHRoZSBtaXNzaW5nIHZhbHVlcy4iIChQZXVnaCAmIEVuZGVycywgMjAwNCkNCi0gTGV0cyB0ZXN0IG91dCBNSSBhcyBpdHMgdGhlIG5ld2VzdCBhbmQgY29uc2lkZXJlZCBhbiBpbXByb3ZlbWVudCBvdmVyIE1MDQoNCiMjIE11bHRpcGxlIEltcHV0YXRpb24gYnkgQ2hhaW5lZCBFcXVhdGlvbnMNCi0gIFNlZSBBenVyIGV0IGFsLCAyMDExIGZvciBkZXRhaWxzDQotIE1JQ0UgcGFja2FnZSBpbiBSOiBMb3RzIG9mIG9sZCBhbmQgbW9kZXJuIGFsZ29yaXRobXMgdG8gY2hvb3NlIGZyb20gKGFsc28gZm9yIGRlc2lnbiBiZXlvbmQgbGluZWFyIHJlZ3Jlc3Npb24pIA0KLSBQcmVkaWN0aXZlIG1lYW4gbWF0Y2hpbmcgaXMgYSBnb29kIHZlcnNpb24gdG8gdXNlIGFzIGl0ICJpcyBhIGdlbmVyYWwgcHVycG9zZSBzZW1pLXBhcmFtZXRyaWMgaW1wdXRhdGlvbiBtZXRob2QuIC4uLiBpbXB1dGF0aW9ucyBhcmUgcmVzdHJpY3RlZCB0byB0aGUgb2JzZXJ2ZWQgdmFsdWVzIGFuZCB0aGF0IGl0IGNhbiBwcmVzZXJ2ZSBub24tbGluZWFyIHJlbGF0aW9ucy4uLiIgKEJ1dXJlbiAmIEdyb290aHVpcy1PdWRzaG9vcm4sIDIwMTEpDQotIFdlIHdpbGwgd29yayBmcm9tIG91ciBNQVIgbWlzc2luZyBEYXRhc2V0NS5NIA0KLSBXZSBjYW4gY29tcGFyZSB0aGlzIHRvIG91ciBjbGFzc2ljYWwgbWV0aG9kcyBhbmQgc29tZSBvdGhlciBtZXRob2RzIHdoaWNoIHRoZSBwYWNrYWdlIGFsbG93cyAoQm9vdHN0YXBwaW5nICYgQmF5ZXNpYW4pDQoNCiMjIyBUeXBpY2FsIHNhbXBsZSBzaXplDQoNCmBgYHtyLCBlY2hvPVRSVUUsIHdhcm5pbmc9RkFMU0UsbWVzc2FnZT1GQUxTRSxyZXN1bHRzPSdhc2lzJ30NCg0KI0xpbmVhciByZWdyZXNzaW9uIHVzaW5nIGJvb3RzdHJhcA0KRGF0YURhdGFTZXQ1LkJvb3Q8LSBjb21wbGV0ZShtaWNlKERhdGFTZXQ1Lk0sIG0gPSAxMCxtZXRoPSdub3JtLmJvb3QnLHByaW50RmxhZyA9IEZBTFNFLCBzZWVkID0gNjY2KSkNCkJvb3Q8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFEYXRhU2V0NS5Cb290KQ0KDQojQmF5ZXNpYW4gbGluZWFyIHJlZ3Jlc3Npb24NCkRhdGFEYXRhU2V0NS5CTFI8LSBjb21wbGV0ZShtaWNlKERhdGFTZXQ1Lk0sIG0gPSAxMCxtZXRoPSdub3JtJyxwcmludEZsYWcgPSBGQUxTRSwgc2VlZCA9IDY2NikpDQpCTFI8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFEYXRhU2V0NS5CTFIpDQoNCiNQcmVkaWN0aXZlIG1lYW4gbWF0Y2hpbmcgDQpEYXRhRGF0YVNldDUucG1tPC0gY29tcGxldGUobWljZShEYXRhU2V0NS5NLCBtID0gMTAsbWV0aD0ncG1tJyxwcmludEZsYWcgPSBGQUxTRSwgc2VlZCA9IDY2NikpDQpQTU08LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFEYXRhU2V0NS5wbW0pDQoNCnN0YXJnYXplcihPcmdpbmFsLjUsIEJvb3QsIEJMUiwgUE1NLHR5cGU9Imh0bWwiLA0KICAgICAgICAgIGNvbHVtbi5sYWJlbHMgPSBjKCJPcmdpbmFsIiwiQm9vdCIsIkJMUiIsIlBNTSIpLA0KICAgICAgICAgIGludGVyY2VwdC5ib3R0b20gPSBGQUxTRSwNCiAgICAgICAgICBzaW5nbGUucm93PUZBTFNFLCANCiAgICAgICAgICBub3Rlcy5hcHBlbmQgPSBGQUxTRSwNCiAgICAgICAgICBoZWFkZXI9RkFMU0UpDQoNCmBgYA0KDQotIFByZWRpY3RpdmUgbWVhbiBtYXRjaGluZyBpcyB0aGUgZGVmYXVsdCBvZiB0aGlzIHByb2dyYW0gYW5kIHlvdSBzZWUgaXQgZG9lcyBhIGdvb2Qgam9iDQoNCg0KDQojIyMgU21hbGwgc2FtcGxlIHNpemUNCi0gVXNlIHRoZSBzYW1lIGV4YWN0IHNtYWxsIGRhdGEgKE4gPSAxNSkNCg0KYGBge3IsIGVjaG89VFJVRSwgd2FybmluZz1GQUxTRSxtZXNzYWdlPUZBTFNFLHJlc3VsdHM9J2FzaXMnfQ0KI0xpbmVhciByZWdyZXNzaW9uIHVzaW5nIGJvb3RzdHJhcA0KRGF0YURhdGFTZXQzLkJvb3Q8LSBjb21wbGV0ZShtaWNlKERhdGFTZXQzLk0sIG0gPSAxNSxtZXRoPSdub3JtLmJvb3QnLHByaW50RmxhZyA9IEZBTFNFLCBzZWVkID0gNjY2KSkNCkJvb3QuMzwtbG0oRFZ+SVYxK0lWMiwgZGF0YT0gRGF0YURhdGFTZXQzLkJvb3QpDQoNCiNCYXllc2lhbiBsaW5lYXIgcmVncmVzc2lvbg0KRGF0YURhdGFTZXQzLkJMUjwtIGNvbXBsZXRlKG1pY2UoRGF0YVNldDMuTSwgbSA9IDE1LG1ldGg9J25vcm0nLHByaW50RmxhZyA9IEZBTFNFLCBzZWVkID0gNjY2KSkNCkJMUi4zPC1sbShEVn5JVjErSVYyLCBkYXRhPSBEYXRhRGF0YVNldDMuQkxSKQ0KDQojUHJlZGljdGl2ZSBtZWFuIG1hdGNoaW5nIA0KRGF0YURhdGFTZXQzLnBtbTwtIGNvbXBsZXRlKG1pY2UoRGF0YVNldDMuTSwgbSA9IDE1LG1ldGg9J3BtbScscHJpbnRGbGFnID0gRkFMU0UsIHNlZWQgPSA2NjYpKQ0KUE1NLjM8LWxtKERWfklWMStJVjIsIGRhdGE9IERhdGFEYXRhU2V0My5wbW0pDQoNCnN0YXJnYXplcihPcmdpbmFsLjMsIEJvb3QuMywgQkxSLjMsIFBNTS4zLHR5cGU9Imh0bWwiLA0KICAgICAgICAgIGNvbHVtbi5sYWJlbHMgPSBjKCJPcmdpbmFsIiwiQm9vdCIsIkJMUiIsIlBNTSIpLA0KICAgICAgICAgIGludGVyY2VwdC5ib3R0b20gPSBGQUxTRSwNCiAgICAgICAgICBzaW5nbGUucm93PUZBTFNFLCANCiAgICAgICAgICBub3Rlcy5hcHBlbmQgPSBGQUxTRSwNCiAgICAgICAgICBoZWFkZXI9RkFMU0UpDQoNCmBgYA0KDQotIFdlIG5lZWQgdG8gZmluZCBwcmFjdGljZXIgb2YgdGhlIGRhcmsgYXJ0cywgYXMgbm9uZSBvZiB0aGUgbWV0aG9kcyBkaWQgYSBnb29kIGpvYg0KDQojIyMjIE5vdGVzDQotIFlvdSBoYXZlIHRvIG1hbnVhbGx5IHNldCB0aGUgbnVtYmVyIG9mIGl0ZXJhdGlvbnMgKG51bWJlciBvZiBwYXNzZXMpDQotIFlvdSB3aWxsIGhhdmUgdG8gcmVzZWFyY2ggZWFjaCBhbGdvcml0aG0gZm9yIHlvdXIgcGFydGljdWxhciBkYXRhIG1pc3NpbmduZXNzDQoNCiMgQmVzdCBQcmFjdGljZXMNCi0gRWFjaCBzaXR1YXRpb24gaXMgZGlmZmVyZW50IGFuZCB3aGljaCBtZXRob2QgeW91IHVzZSBkZXBlbmRzIG9uIG1hbnkgZmFjdG9ycw0KLSBBbHNvIHlvdSBtdXN0IHRoaW5rIGNhcmVmdWxseSBhYm91dCB0aGUgYXNzdW1wdGlvbnMgeW91IGFyZSBtYWtpbmcNCi0gRm9yIGV4YW1wbGUsIGlmIHlvdSBtZWFuIChvciBQTU0pIHJlcGxhY2UgeW91IGFyZSBhc3N1bWluZyB0aGUgZGF0YSBhcmUgbm9ybWFsDQotIFRoZWUgYXJlIG90aGVyIGFkdmFuY2VkIG1ldGhvZHMgZm9yIHNwZWNpZmljIHNpdHVhdGlvbjogImhvdCBkZWNraW5nIiBmb3Igc3VydmV5IHJlc2VhcmNoOyAgbmVhcmVzdCBuZWlnaGJvciwgc3BsaW5lcywgb3IgYXV0b3JlZ3Jlc3NpdmUgbWV0aG9kcyBmb3IgdGltZS1zZXJpZXMgb3IgbG9uZ2l0dWRpbmFsIGRhdGENCi0gQmVzdCBwcmFjdGljZTogY29tcGFyZSBzZXJ2YWwgbG9naWNhbCByZXBsYWNlbWVudCBtZXRob2RzIGFuZCBhbHNvIGNvbXBhcmUgaXQgdG8gbGlzdHdpc2UgcmVtb3ZhbC4gSWYgdGhlIHN0b3J5IHlvdSBhcmUgdHJ5aW5nIHRvIHRlbGwgaXMgdmVyeSBkaWZmZXJlbnQgYmFzZWQgZWFjaCBtZXRob2QgYW5kIHZlcnkgZGlmZmVyZW50IGFzIGl0IHJlbGF0ZXMgdG8gdGhlIGxpc3R3aXNlIHJlbW92YWwgaXQgc3VnZ2VzdHMgeW91IGhhdmUgbm90IGZhY3RvcmVkIGluIHNvbWUgYXNzdW1wdGlvbiBjb3JyZWN0bHkgYW5kIGJlc3QgdG8gcmV0aGluayB3aGF0IHlvdSBhcmUgZG9pbmcuIA0KDQoNCjxzY3JpcHQ+DQogIChmdW5jdGlvbihpLHMsbyxnLHIsYSxtKXtpWydHb29nbGVBbmFseXRpY3NPYmplY3QnXT1yO2lbcl09aVtyXXx8ZnVuY3Rpb24oKXsNCiAgKGlbcl0ucT1pW3JdLnF8fFtdKS5wdXNoKGFyZ3VtZW50cyl9LGlbcl0ubD0xKm5ldyBEYXRlKCk7YT1zLmNyZWF0ZUVsZW1lbnQobyksDQogIG09cy5nZXRFbGVtZW50c0J5VGFnTmFtZShvKVswXTthLmFzeW5jPTE7YS5zcmM9ZzttLnBhcmVudE5vZGUuaW5zZXJ0QmVmb3JlKGEsbSkNCiAgfSkod2luZG93LGRvY3VtZW50LCdzY3JpcHQnLCdodHRwczovL3d3dy5nb29nbGUtYW5hbHl0aWNzLmNvbS9hbmFseXRpY3MuanMnLCdnYScpOw0KDQogIGdhKCdjcmVhdGUnLCAnVUEtOTA0MTUxNjAtMScsICdhdXRvJyk7DQogIGdhKCdzZW5kJywgJ3BhZ2V2aWV3Jyk7DQoNCjwvc2NyaXB0Pg==