glmnet/ 0000755 0001762 0000144 00000000000 12672525200 011541 5 ustar ligges users glmnet/inst/ 0000755 0001762 0000144 00000000000 12672345374 012532 5 ustar ligges users glmnet/inst/CITATION 0000644 0001762 0000144 00000003315 11532776611 013665 0 ustar ligges users citHeader("To cite glmnet in publications use:")
citEntry(entry = "Article",
title = "Regularization Paths for Generalized Linear Models via Coordinate Descent",
author = personList(as.person("Jerome Friedman"),
as.person("Trevor Hastie"),
as.person("Robert Tibshirani")),
journal = "Journal of Statistical Software",
year = "2010",
volume = "33",
number = "1",
pages = "1--22",
url = "http://www.jstatsoft.org/v33/i01/",
textVersion =
paste("Jerome Friedman, Trevor Hastie, Robert Tibshirani (2010).",
"Regularization Paths for Generalized Linear Models via Coordinate Descent.",
"Journal of Statistical Software, 33(1), 1-22.",
"URL http://www.jstatsoft.org/v33/i01/.")
)
citEntry(entry = "Article",
title = "Regularization Paths for Cox's Proportional Hazards Model via Coordinate Descent",
author = personList(as.person("Noah Simon"),
as.person("Jerome Friedman"),
as.person("Trevor Hastie"),
as.person("Rob Tibshirani")),
journal = "Journal of Statistical Software",
year = "2011",
volume = "39",
number = "5",
pages = "1--13",
url = "http://www.jstatsoft.org/v39/i05/",
textVersion =
paste("Noah Simon, Jerome Friedman, Trevor Hastie, Rob Tibshirani (2011).",
"Regularization Paths for Cox's Proportional Hazards Model via Coordinate Descent.",
"Journal of Statistical Software, 39(5), 1-13.",
"URL http://www.jstatsoft.org/v39/i05/."),
header = "If coxnet is used, please also cite:"
)
glmnet/inst/doc/ 0000755 0001762 0000144 00000000000 12672345374 013277 5 ustar ligges users glmnet/inst/doc/glmnet_beta.html 0000644 0001762 0000144 00004355530 12672345374 016465 0 ustar ligges users
Glmnet Vignette
Glmnet Vignette
Trevor Hastie and Junyang Qian
Stanford June 26, 2014
Introduction
Installation
Quick Start
Linear Regression
Logistic Regression
Poisson Models
Cox Models
Sparse Matrices
Appendix 1: Internal Parameters
Appendix 2: Comparison with Otherther Packages
Introduction
Glmnet is a package that fits a generalized linear model via penalized maximum likelihood. The regularization path is computed for the lasso or elasticnet penalty at a grid of values for the regularization parameter lambda. The algorithm is extremely fast, and can exploit sparsity in the input matrix x. It fits linear, logistic and multinomial, poisson, and Cox regression models. A variety of predictions can be made from the fitted models. It can also fit multi-response linear regression.
The authors of glmnet are Jerome Friedman, Trevor Hastie, Rob Tibshirani and Noah Simon, and the R package is maintained by Trevor Hastie. The matlab version of glmnet is maintained by Junyang Qian. This vignette describes the usage of glmnet in R.
glmnet solves the following problem
\[
\min_{\beta_0,\beta} \frac{1}{N} \sum_{i=1}^{N} w_i l(y_i,\beta_0+\beta^T x_i) + \lambda\left[(1-\alpha)||\beta||_2^2/2 + \alpha ||\beta||_1\right],
\]
over a grid of values of \(\lambda\) covering the entire range. Here \(l(y,\eta)\) is the negative log-likelihood contribution for observation \(i\); e.g. for the Gaussian case it is \(\frac{1}{2}(y-\eta)^2\). The elastic-net penalty is controlled by \(\alpha\), and bridges the gap between lasso (\(\alpha=1\), the default) and ridge (\(\alpha=0\)). The tuning parameter \(\lambda\) controls the overall strength of the penalty.
It is known that the ridge penalty shrinks the coefficients of correlated predictors towards each other while the lasso tends to pick one of them and discard the others. The elastic-net penalty mixes these two; if predictors are correlated in groups, an \(\alpha=0.5\) tends to select the groups in or out together. This is a higher level parameter, and users might pick a value upfront, else experiment with a few different values. One use of \(\alpha\) is for numerical stability; for example, the elastic net with \(\alpha = 1 - \epsilon\) for some small \(\epsilon > 0\) performs much like the lasso, but removes any degeneracies and wild behavior caused by extreme correlations.
The glmnet algorithms use cyclical coordinate descent, which successively optimizes the objective function over each parameter with others fixed, and cycles repeatedly until convergence. The package also makes use of the strong rules for efficient restriction of the active set. Due to highly efficient updates and techniques such as warm starts and active-set convergence, our algorithms can compute the solution path very fast.
The code can handle sparse input-matrix formats, as well as range constraints on coefficients. The core of glmnet is a set of fortran subroutines, which make for very fast execution.
The package also includes methods for prediction and plotting, and a function that performs K-fold cross-validation.
Installation
Like many other R packages, the simplest way to obtain glmnet is to install it directly from CRAN. Type the following command in R console:
install.packages("glmnet", repos = "http://cran.us.r-project.org")
Users may change the repos options depending on their locations and preferences. Other options such as the directories where to install the packages can be altered in the command. For more details, see help(install.packages).
Here the R package has been downloaded and installed to the default directories.
Alternatively, users can download the package source at http://cran.r-project.org/package=glmnet and type Unix commands to install it to the desired location.
Back to Top
Quick Start
The purpose of this section is to give users a general sense of the package, including the components, what they do and some basic usage. We will briefly go over the main functions, see the basic operations and have a look at the outputs. Users may have a better idea after this section what functions are available, which one to choose, or at least where to seek help. More details are given in later sections.
First, we load the glmnet package:
library(glmnet)
## Loading required package: Matrix
## Loading required package: foreach
## Loaded glmnet 2.0-5
The default model used in the package is the Guassian linear model or “least squares”, which we will demonstrate in this section. We load a set of data created beforehand for illustration. Users can either load their own data or use those saved in the workspace.
data(QuickStartExample)
The command loads an input matrix x and a response vector y from this saved R data archive.
We fit the model using the most basic call to glmnet.
fit = glmnet(x, y)
“fit” is an object of class glmnet that contains all the relevant information of the fitted model for further use. We do not encourage users to extract the components directly. Instead, various methods are provided for the object such as plot, print, coef and predict that enable us to execute those tasks more elegantly.
We can visualize the coefficients by executing the plot function:
plot(fit)
Each curve corresponds to a variable. It shows the path of its coefficient against the \(\ell_1\)-norm of the whole coefficient vector at as \(\lambda\) varies. The axis above indicates the number of nonzero coefficients at the current \(\lambda\), which is the effective degrees of freedom (df) for the lasso. Users may also wish to annotate the curves; this can be done by setting label = TRUE in the plot command.
A summary of the glmnet path at each step is displayed if we just enter the object name or use the print function:
print(fit)
##
## Call: glmnet(x = x, y = y)
##
## Df %Dev Lambda
## [1,] 0 0.00000 1.631000
## [2,] 2 0.05528 1.486000
## [3,] 2 0.14590 1.354000
## [4,] 2 0.22110 1.234000
## [5,] 2 0.28360 1.124000
## [6,] 2 0.33540 1.024000
## [7,] 4 0.39040 0.933200
## [8,] 5 0.45600 0.850300
## [9,] 5 0.51540 0.774700
## [10,] 6 0.57350 0.705900
## [11,] 6 0.62550 0.643200
## [12,] 6 0.66870 0.586100
## [13,] 6 0.70460 0.534000
## [14,] 6 0.73440 0.486600
## [15,] 7 0.76210 0.443300
## [16,] 7 0.78570 0.404000
## [17,] 7 0.80530 0.368100
## [18,] 7 0.82150 0.335400
## [19,] 7 0.83500 0.305600
## [20,] 7 0.84620 0.278400
## [21,] 7 0.85550 0.253700
## [22,] 7 0.86330 0.231200
## [23,] 8 0.87060 0.210600
## [24,] 8 0.87690 0.191900
## [25,] 8 0.88210 0.174900
## [26,] 8 0.88650 0.159300
## [27,] 8 0.89010 0.145200
## [28,] 8 0.89310 0.132300
## [29,] 8 0.89560 0.120500
## [30,] 8 0.89760 0.109800
## [31,] 9 0.89940 0.100100
## [32,] 9 0.90100 0.091170
## [33,] 9 0.90230 0.083070
## [34,] 9 0.90340 0.075690
## [35,] 10 0.90430 0.068970
## [36,] 11 0.90530 0.062840
## [37,] 11 0.90620 0.057260
## [38,] 12 0.90700 0.052170
## [39,] 15 0.90780 0.047540
## [40,] 16 0.90860 0.043310
## [41,] 16 0.90930 0.039470
## [42,] 16 0.90980 0.035960
## [43,] 17 0.91030 0.032770
## [44,] 17 0.91070 0.029850
## [45,] 18 0.91110 0.027200
## [46,] 18 0.91140 0.024790
## [47,] 19 0.91170 0.022580
## [48,] 19 0.91200 0.020580
## [49,] 19 0.91220 0.018750
## [50,] 19 0.91240 0.017080
## [51,] 19 0.91250 0.015570
## [52,] 19 0.91260 0.014180
## [53,] 19 0.91270 0.012920
## [54,] 19 0.91280 0.011780
## [55,] 19 0.91290 0.010730
## [56,] 19 0.91290 0.009776
## [57,] 19 0.91300 0.008908
## [58,] 19 0.91300 0.008116
## [59,] 19 0.91310 0.007395
## [60,] 19 0.91310 0.006738
## [61,] 19 0.91310 0.006140
## [62,] 20 0.91310 0.005594
## [63,] 20 0.91310 0.005097
## [64,] 20 0.91310 0.004644
## [65,] 20 0.91320 0.004232
## [66,] 20 0.91320 0.003856
## [67,] 20 0.91320 0.003513
It shows from left to right the number of nonzero coefficients (Df), the percent (of null) deviance explained (%dev) and the value of \(\lambda\) (Lambda). Although by default glmnet calls for 100 values of lambda the program stops early if %dev% does not change sufficently from one lambda to the next (typically near the end of the path.)
We can obtain the actual coefficients at one or more \(\lambda\)'s within the range of the sequence:
coef(fit,s=0.1)
## 21 x 1 sparse Matrix of class "dgCMatrix"
## 1
## (Intercept) 0.150928072
## V1 1.320597195
## V2 .
## V3 0.675110234
## V4 .
## V5 -0.817411518
## V6 0.521436671
## V7 0.004829335
## V8 0.319415917
## V9 .
## V10 .
## V11 0.142498519
## V12 .
## V13 .
## V14 -1.059978702
## V15 .
## V16 .
## V17 .
## V18 .
## V19 .
## V20 -1.021873704
(why s and not lambda? In case later we want to allow one to specify the model size in other ways.)
Users can also make predictions at specific \(\lambda\)'s with new input data:
nx = matrix(rnorm(10*20),10,20)
predict(fit,newx=nx,s=c(0.1,0.05))
## 1 2
## [1,] -5.5155462 -5.6471349
## [2,] 4.2992190 4.4266457
## [3,] 1.6747706 1.6740427
## [4,] 1.2253111 1.2508196
## [5,] 3.2800685 3.4072855
## [6,] -1.5786571 -1.7047168
## [7,] -3.6281723 -3.8167525
## [8,] 0.9106243 0.8953048
## [9,] 3.2410747 3.3610994
## [10,] 1.7576098 1.9716290
The function glmnet returns a sequence of models for the users to choose from. In many cases, users may prefer the software to select one of them. Cross-validation is perhaps the simplest and most widely used method for that task.
cv.glmnet is the main function to do cross-validation here, along with various supporting methods such as plotting and prediction. We still act on the sample data loaded before.
cvfit = cv.glmnet(x, y)
cv.glmnet returns a cv.glmnet object, which is “cvfit” here, a list with all the ingredients of the cross-validation fit. As for glmnet, we do not encourage users to extract the components directly except for viewing the selected values of \(\lambda\). The package provides well-designed functions for potential tasks.
We can plot the object.
plot(cvfit)
It includes the cross-validation curve (red dotted line), and upper and lower standard deviation curves along the \(\lambda\) sequence (error bars). Two selected \(\lambda\)'s are indicated by the vertical dotted lines (see below).
We can view the selected \(\lambda\)'s and the corresponding coefficients. For example,
cvfit$lambda.min
## [1] 0.06896889
lambda.min is the value of \(\lambda\) that gives minimum mean cross-validated error. The other \(\lambda\) saved is lambda.1se, which gives the most regularized model such that error is within one standard error of the minimum. To use that, we only need to replace lambda.min with lambda.1se above.
coef(cvfit, s = "lambda.min")
## 21 x 1 sparse Matrix of class "dgCMatrix"
## 1
## (Intercept) 0.147927056
## V1 1.337393911
## V2 .
## V3 0.704086481
## V4 .
## V5 -0.842853150
## V6 0.549403480
## V7 0.032703914
## V8 0.342430521
## V9 .
## V10 0.001206608
## V11 0.178995989
## V12 .
## V13 .
## V14 -1.079993473
## V15 .
## V16 .
## V17 .
## V18 .
## V19 .
## V20 -1.061382444
Note that the coefficients are represented in the sparse matrix format. The reason is that the solutions along the regularization path are often sparse, and hence it is more efficient in time and space to use a sparse format. If you prefer non-sparse format, pipe the output through as.matrix().
Predictions can be made based on the fitted cv.glmnet object. Let's see a toy example.
predict(cvfit, newx = x[1:5,], s = "lambda.min")
## 1
## [1,] -1.3626977
## [2,] 2.5736774
## [3,] 0.5767335
## [4,] 2.0062604
## [5,] 1.5410061
newx is for the new input matrix and s, as before, is the value(s) of \(\lambda\) at which predictions are made.
That is the end of glmnet 101. With the tools introduced so far, users are able to fit the entire elastic net family, including ridge regression, using squared-error loss. In the package, there are many more options that give users a great deal of flexibility. To learn more, move on to later sections.
Back to Top
Linear Regression
Linear regression here refers to two families of models. One is gaussian, the Gaussian family, and the other is mgaussian, the multiresponse Gaussian family. We first discuss the ordinary Gaussian and the multiresponse one after that.
Gaussian Family
gaussian is the default family option in the function glmnet. Suppose we have observations \(x_i \in \mathbb{R}^p\) and the responses \(y_i \in \mathbb{R}, i = 1, \ldots, N\). The objective function for the Gaussian family is
\[
\min_{(\beta_0, \beta) \in \mathbb{R}^{p+1}}\frac{1}{2N} \sum_{i=1}^N (y_i -\beta_0-x_i^T \beta)^2+\lambda \left[ (1-\alpha)||\beta||_2^2/2 + \alpha||\beta||_1\right],
\]
where \(\lambda \geq 0\) is a complexity parameter and \(0 \leq \alpha \leq 1\) is a compromise between ridge (\(\alpha = 0\)) and lasso (\(\alpha = 1\)).
Coordinate descent is applied to solve the problem. Specifically, suppose we have current estimates \(\tilde{\beta_0}\) and \(\tilde{\beta}_\ell\) \(\forall j\in 1,]\ldots,p\). By computing the gradient at \(\beta_j = \tilde{\beta}_j\) and simple calculus, the update is
\[
\tilde{\beta}_j \leftarrow \frac{S(\frac{1}{N}\sum_{i=1}^N x_{ij}(y_i-\tilde{y}_i^{(j)}),\lambda \alpha)}{1+\lambda(1-\alpha)},
\]
where \(\tilde{y}_i^{(j)} = \tilde{\beta}_0 + \sum_{\ell \neq j} x_{i\ell} \tilde{\beta}_\ell\), and \(S(z, \gamma)\) is the soft-thresholding operator with value \(\text{sign}(z)(|z|-\gamma)_+\).
This formula above applies when the x variables are standardized to have unit variance (the default); it is slightly more complicated when they are not. Note that for “family=gaussian”, glmnet standardizes \(y\) to have unit variance before computing its lambda sequence (and then unstandardizes the resulting coefficients); if you wish to reproduce/compare results with other software, best to supply a standardized \(y\) first (Using the “1/N” variance formula).
glmnet provides various options for users to customize the fit. We introduce some commonly used options here and they can be specified in the glmnet function.
alpha is for the elastic-net mixing parameter \(\alpha\), with range \(\alpha \in [0,1]\). \(\alpha = 1\) is the lasso (default) and \(\alpha = 0\) is the ridge.
weights is for the observation weights. Default is 1 for each observation. (Note: glmnet rescales the weights to sum to N, the sample size.)
nlambda is the number of \(\lambda\) values in the sequence. Default is 100.
lambda can be provided, but is typically not and the program constructs a sequence. When automatically generated, the \(\lambda\) sequence is determined by lambda.max and lambda.min.ratio. The latter is the ratio of smallest value of the generated \(\lambda\) sequence (say lambda.min) to lambda.max. The program then generated nlambda values linear on the log scale from lambda.max down to lambda.min. lambda.max is not given, but easily computed from the input \(x\) and \(y\); it is the smallest value for lambda such that all the coefficients are zero. For alpha=0 (ridge) lambda.max would be \(\infty\); hence for this case we pick a value corresponding to a small value for alpha close to zero.)
standardize is a logical flag for x variable standardization, prior to fitting the model sequence. The coefficients are always returned on the original scale. Default is standardize=TRUE.
For more information, type help(glmnet) or simply ?glmnet.
As an example, we set \(\alpha = 0.2\) (more like a ridge regression), and give double weights to the latter half of the observations. To avoid too long a display here, we set nlambda to 20. In practice, however, the number of values of \(\lambda\) is recommended to be 100 (default) or more. In most cases, it does not come with extra cost because of the warm-starts used in the algorithm, and for nonlinear models leads to better convergence properties.
fit = glmnet(x, y, alpha = 0.2, weights = c(rep(1,50),rep(2,50)), nlambda = 20)
We can then print the glmnet object.
print(fit)
##
## Call: glmnet(x = x, y = y, weights = c(rep(1, 50), rep(2, 50)), alpha = 0.2, nlambda = 20)
##
## Df %Dev Lambda
## [1,] 0 0.0000 7.939000
## [2,] 4 0.1789 4.889000
## [3,] 7 0.4445 3.011000
## [4,] 7 0.6567 1.854000
## [5,] 8 0.7850 1.142000
## [6,] 9 0.8539 0.703300
## [7,] 10 0.8867 0.433100
## [8,] 11 0.9025 0.266700
## [9,] 14 0.9101 0.164300
## [10,] 17 0.9138 0.101200
## [11,] 17 0.9154 0.062300
## [12,] 17 0.9160 0.038370
## [13,] 19 0.9163 0.023630
## [14,] 20 0.9164 0.014550
## [15,] 20 0.9164 0.008962
## [16,] 20 0.9165 0.005519
## [17,] 20 0.9165 0.003399
This displays the call that produced the object fit and a three-column matrix with columns Df (the number of nonzero coefficients), %dev (the percent deviance explained) and Lambda (the corresponding value of \(\lambda\)).
(Note that the digits option can used to specify significant digits in the printout.)
Here the actual number of \(\lambda\)'s here is less than specified in the call. The reason lies in the stopping criteria of the algorithm. According to the default internal settings, the computations stop if either the fractional change in deviance down the path is less than \(10^{-5}\) or the fraction of explained deviance reaches \(0.999\). From the last few lines , we see the fraction of deviance does not change much and therefore the computation ends when meeting the stopping criteria. We can change such internal parameters. For details, see the Appendix section or type help(glmnet.control).
We can plot the fitted object as in the previous section. There are more options in the plot function.
Users can decide what is on the X-axis. xvar allows three measures: “norm” for the \(\ell_1\)-norm of the coefficients (default), “lambda” for the log-lambda value and “dev” for %deviance explained.
Users can also label the curves with variable sequence numbers simply by setting label = TRUE.
Let's plot “fit” against the log-lambda value and with each curve labeled.
plot(fit, xvar = "lambda", label = TRUE)
Now when we plot against %deviance we get a very different picture. This is percent deviance explained on the training data. What we see here is that toward the end of the path this value are not changing much, but the coefficients are “blowing up” a bit. This lets us focus attention on the parts of the fit that matter. This will especially be true for other models, such as logistic regression.
plot(fit, xvar = "dev", label = TRUE)
We can extract the coefficients and make predictions at certain values of \(\lambda\). Two commonly used options are:
s specifies the value(s) of \(\lambda\) at which extraction is made.
exact indicates whether the exact values of coefficients are desired or not. That is, if exact = TRUE, and predictions are to be made at values of s not included in the original fit, these values of s are merged with object$lambda, and the model is refit before predictions are made. If exact=FALSE (default), then the predict function uses linear interpolation to make predictions for values of s that do not coincide with lambdas used in the fitting algorithm.
A simple example is:
any(fit$lambda == 0.5)
## [1] FALSE
coef.exact = coef(fit, s = 0.5, exact = TRUE)
coef.apprx = coef(fit, s = 0.5, exact = FALSE)
cbind2(coef.exact, coef.apprx)
## 21 x 2 sparse Matrix of class "dgCMatrix"
## 1 1
## (Intercept) 0.19657091 0.199098747
## V1 1.17495906 1.174650452
## V2 . .
## V3 0.52934375 0.531934651
## V4 . .
## V5 -0.76126245 -0.760959480
## V6 0.46627405 0.468209413
## V7 0.06148079 0.061926756
## V8 0.38048793 0.380301491
## V9 . .
## V10 . .
## V11 0.14213614 0.143260991
## V12 . .
## V13 . .
## V14 -0.91090226 -0.911207368
## V15 . .
## V16 . .
## V17 . .
## V18 . 0.009196628
## V19 . .
## V20 -0.86099392 -0.863117051
The left column is for exact = TRUE and the right for FALSE. We see from the above that 0.01 is not in the sequence and that hence there are some difference, though not much. Linear interpolation is mostly enough if there are no special requirements.
Users can make predictions from the fitted object. In addition to the options in coef, the primary argument is newx, a matrix of new values for x. The type option allows users to choose the type of prediction:
“link” gives the fitted values
“response” the sames as “link” for “gaussian” family.
“coefficients” computes the coefficients at values of s
“nonzero” retuns a list of the indices of the nonzero coefficients for each value of s.
For example,
predict(fit, newx = x[1:5,], type = "response", s = 0.05)
## 1
## [1,] -0.9802591
## [2,] 2.2992453
## [3,] 0.6010886
## [4,] 2.3572668
## [5,] 1.7520421
gives the fitted values for the first 5 observations at \(\lambda = 0.05\). If multiple values of s are supplied, a matrix of predictions is produced.
Users can customize K-fold cross-validation. In addition to all the glmnet parameters, cv.glmnet has its special parameters including nfolds (the number of folds), foldid (user-supplied folds), type.measure(the loss used for cross-validation):
As an example,
cvfit = cv.glmnet(x, y, type.measure = "mse", nfolds = 20)
does 20-fold cross-validation, based on mean squared error criterion (default though).
Parallel computing is also supported by cv.glmnet. To make it work, users must register parallel beforehand. We give a simple example of comparison here. Unfortunately, the package doMC is not available on Windows platforms (it is on others), so we cannot run the code here, but we make it looks as if we have.
require(doMC)
registerDoMC(cores=2)
X = matrix(rnorm(1e4 * 200), 1e4, 200)
Y = rnorm(1e4)
system.time(cv.glmnet(X, Y))
## user system elapsed
## 2.440 0.080 2.518
system.time(cv.glmnet(X, Y, parallel = TRUE))
## user system elapsed
## 2.450 0.157 1.567
As suggested from the above, parallel computing can significantly speed up the computation process especially for large-scale problems.
Functions coef and predict on cv.glmnet object are similar to those for a glmnet object, except that two special strings are also supported by s (the values of \(\lambda\) requested):
cvfit$lambda.min
## [1] 0.08307327
coef(cvfit, s = "lambda.min")
## 21 x 1 sparse Matrix of class "dgCMatrix"
## 1
## (Intercept) 0.14936467
## V1 1.32975267
## V2 .
## V3 0.69096092
## V4 .
## V5 -0.83122558
## V6 0.53669611
## V7 0.02005438
## V8 0.33193760
## V9 .
## V10 .
## V11 0.16239419
## V12 .
## V13 .
## V14 -1.07081121
## V15 .
## V16 .
## V17 .
## V18 .
## V19 .
## V20 -1.04340741
predict(cvfit, newx = x[1:5,], s = "lambda.min")
## 1
## [1,] -1.3647490
## [2,] 2.5686013
## [3,] 0.5705879
## [4,] 1.9682289
## [5,] 1.4964211
Users can control the folds used. Here we use the same folds so we can also select a value for \(\alpha\).
foldid=sample(1:10,size=length(y),replace=TRUE)
cv1=cv.glmnet(x,y,foldid=foldid,alpha=1)
cv.5=cv.glmnet(x,y,foldid=foldid,alpha=.5)
cv0=cv.glmnet(x,y,foldid=foldid,alpha=0)
There are no built-in plot functions to put them all on the same plot, so we are on our own here:
par(mfrow=c(2,2))
plot(cv1);plot(cv.5);plot(cv0)
plot(log(cv1$lambda),cv1$cvm,pch=19,col="red",xlab="log(Lambda)",ylab=cv1$name)
points(log(cv.5$lambda),cv.5$cvm,pch=19,col="grey")
points(log(cv0$lambda),cv0$cvm,pch=19,col="blue")
legend("topleft",legend=c("alpha= 1","alpha= .5","alpha 0"),pch=19,col=c("red","grey","blue"))
We see that lasso (alpha=1) does about the best here. We also see that the range of lambdas used differs with alpha.
Coefficient upper and lower bounds
These are recently added features that enhance the scope of the models. Suppose we want to fit our model, but limit the coefficients to be bigger than -0.7 and less than 0.5. This is easily achieved via the upper.limits and lower.limits arguments:
tfit=glmnet(x,y,lower=-.7,upper=.5)
plot(tfit)
These are rather arbitrary limits; often we want the coefficients to be positive, so we can set only lower.limit to be 0.
(Note, the lower limit must be no bigger than zero, and the upper limit no smaller than zero.)
These bounds can be a vector, with different values for each coefficient. If given as a scalar, the same number gets recycled for all.
Penalty factors
This argument allows users to apply separate penalty factors to each coefficient. Its default is 1 for each parameter, but other values can be specified. In particular, any variable with penalty.factor equal to zero is not penalized at all! Let \(v_j\) denote the penalty factor for \(j\) th variable. The penalty term becomes
\[
\lambda \sum_{j=1}^p \boldsymbol{v_j} P_\alpha(\beta_j) = \lambda \sum_{j=1}^p \boldsymbol{v_j} \left[ (1-\alpha)\frac{1}{2} \beta_j^2 + \alpha |\beta_j| \right].
\]
Note the penalty factors are internally rescaled to sum to nvars.
This is very useful when people have prior knowledge or preference over the variables. In many cases, some variables may be so important that one wants to keep them all the time, which can be achieved by setting corresponding penalty factors to 0:
p.fac = rep(1, 20)
p.fac[c(5, 10, 15)] = 0
pfit = glmnet(x, y, penalty.factor = p.fac)
plot(pfit, label = TRUE)
We see from the labels that the three variables with 0 penalty factors always stay in the model, while the others follow typical regularization paths and shrunken to 0 eventually.
Some other useful arguments. exclude allows one to block certain variables from being the model at all. Of course, one could simply subset these out of x, but sometimes exclude is more useful, since it returns a full vector of coefficients, just with the excluded ones set to zero. There is also an intercept argument which defaults to TRUE; if FALSE the intercept is forced to be zero.
Customizing plots
Sometimes, especially when the number of variables is small, we want to add variable labels to a plot. Since glmnet is intended primarily for wide data, this is not supprted in plot.glmnet. However, it is easy to do, as the following little toy example shows.
We first generate some data, with 10 variables, and for lack of imagination and ease we give them simple character names.
We then fit a glmnet model, and make the standard plot.
set.seed(101)
x=matrix(rnorm(1000),100,10)
y=rnorm(100)
vn=paste("var",1:10)
fit=glmnet(x,y)
plot(fit)
We wish to label the curves with the variable names. Here s a simple way to do this, using the axis command in R (and a little research into how to customize it). We need to have the positions of the coefficients at the end of the path, and we need to make some space using the par command, so that our labels will fit in.
This requires knowing how long your labels are, but here they are all quite short.
par(mar=c(4.5,4.5,1,4))
plot(fit)
vnat=coef(fit)
vnat=vnat[-1,ncol(vnat)] # remove the intercept, and get the coefficients at the end of the path
axis(4, at=vnat,line=-.5,label=vn,las=1,tick=FALSE, cex.axis=0.5)
We have done nothing here to avoid overwriting of labels, in the event that they are close together. This would be a bit more work, but perhaps best left alone, anyway.
Multiresponse Gaussian Family
The multiresponse Gaussian family is obtained using family = "mgaussian" option in glmnet. It is very similar to the single-response case above. This is useful when there are a number of (correlated) responses - the so-called “multi-task learning” problem. Here the sharing involves which variables are selected, since when a variable is selected, a coefficient is fit for each response. Most of the options are the same, so we focus here on the differences with the single response model.
Obviously, as the name suggests, \(y\) is not a vector, but a matrix of quantitative responses in this section. The coefficients at each value of lambda are also a matrix as a result.
Here we solve the following problem:
\[
\min_{(\beta_0, \beta) \in \mathbb{R}^{(p+1)\times K}}\frac{1}{2N} \sum_{i=1}^N ||y_i -\beta_0-\beta^T x_i||^2_F+\lambda \left[ (1-\alpha)||\beta||_F^2/2 + \alpha\sum_{j=1}^p||\beta_j||_2\right].
\]
Here \(\beta_j\) is the jth row of the \(p\times K\) coefficient matrix \(\beta\), and we replace the absolute penalty on each single coefficient by a group-lasso penalty on each coefficient K-vector \(\beta_j\) for a single predictor \(x_j\).
We use a set of data generated beforehand for illustration.
data(MultiGaussianExample)
We fit the data, with an object “mfit” returned.
mfit = glmnet(x, y, family = "mgaussian")
For multiresponse Gaussian, the options in glmnet are almost the same as the single-response case, such as alpha, weights, nlambda, standardize. A exception to be noticed is that standardize.response is only for mgaussian family. The default value is FALSE. If standardize.response = TRUE, it standardizes the response variables.
To visualize the coefficients, we use the plot function.
plot(mfit, xvar = "lambda", label = TRUE, type.coef = "2norm")
Note that we set type.coef = "2norm". Under this setting, a single curve is plotted per variable, with value equal to the \(\ell_2\) norm. The default setting is type.coef = "coef", where a coefficient plot is created for each response (multiple figures).
xvar and label are two other options besides ordinary graphical parameters. They are the same as the single-response case.
We can extract the coefficients at requested values of \(\lambda\) by using the function coef and make predictions by predict. The usage is similar and we only provide an example of predict here.
predict(mfit, newx = x[1:5,], s = c(0.1, 0.01))
## , , 1
##
## y1 y2 y3 y4
## [1,] -4.7106263 -1.1634574 0.6027634 3.740989
## [2,] 4.1301735 -3.0507968 -1.2122630 4.970141
## [3,] 3.1595229 -0.5759621 0.2607981 2.053976
## [4,] 0.6459242 2.1205605 -0.2252050 3.146286
## [5,] -1.1791890 0.1056262 -7.3352965 3.248370
##
## , , 2
##
## y1 y2 y3 y4
## [1,] -4.6415158 -1.2290282 0.6118289 3.779521
## [2,] 4.4712843 -3.2529658 -1.2572583 5.266039
## [3,] 3.4735228 -0.6929231 0.4684037 2.055574
## [4,] 0.7353311 2.2965083 -0.2190297 2.989371
## [5,] -1.2759930 0.2892536 -7.8259206 3.205211
The prediction result is saved in a three-dimensional array with the first two dimensions being the prediction matrix for each response variable and the third indicating the response variables.
We can also do k-fold cross-validation. The options are almost the same as the ordinary Gaussian family and we do not expand here.
cvmfit = cv.glmnet(x, y, family = "mgaussian")
We plot the resulting cv.glmnet object “cvmfit”.
plot(cvmfit)
To show explicitly the selected optimal values of \(\lambda\), type
cvmfit$lambda.min
## [1] 0.04731812
cvmfit$lambda.1se
## [1] 0.1316655
As before, the first one is the value at which the minimal mean squared error is achieved and the second is for the most regularized model whose mean squared error is within one standard error of the minimal.
Prediction for cv.glmnet object works almost the same as for glmnet object. We omit the details here.
Back to Top
Logistic Regression
Logistic regression is another widely-used model when the response is categorical. If there are two possible outcomes, we use the binomial distribution, else we use the multinomial.
Binomial Models
For the binomial model, suppose the response variable takes value in \(\mathcal{G}=\{1,2\}\). Denote \(y_i = I(g_i=1)\). We model
\[\mbox{Pr}(G=2|X=x)+\frac{e^{\beta_0+\beta^Tx}}{1+e^{\beta_0+\beta^Tx}},\]
which can be written in the following form
\[\log\frac{\mbox{Pr}(G=2|X=x)}{\mbox{Pr}(G=1|X=x)}=\beta_0+\beta^Tx,\]
the so-called “logistic” or log-odds transformation.
The objective function for the penalized logistic regression uses the negative binomial log-likelihood, and is
\[
\min_{(\beta_0, \beta) \in \mathbb{R}^{p+1}} -\left[\frac{1}{N} \sum_{i=1}^N y_i \cdot (\beta_0 + x_i^T \beta) - \log (1+e^{(\beta_0+x_i^T \beta)})\right] + \lambda \big[ (1-\alpha)||\beta||_2^2/2 + \alpha||\beta||_1\big].
\]
Logistic regression is often plagued with degeneracies when \(p > N\) and exhibits wild behavior even when \(N\) is close to \(p\);
the elastic-net penalty alleviates these issues, and regularizes and selects variables as well.
Our algorithm uses a quadratic approximation to the log-likelihood, and then coordinate descent on the resulting penalized weighted least-squares problem. These constitute an outer and inner loop.
For illustration purpose, we load pre-generated input matrix x and the response vector y from the data file.
data(BinomialExample)
The input matrix \(x\) is the same as other families. For binomial logistic regression, the response variable \(y\) should be either a factor with two levels, or a two-column matrix of counts or proportions.
Other optional arguments of glmnet for binomial regression are almost same as those for Gaussian family. Don't forget to set family option to “binomial”.
fit = glmnet(x, y, family = "binomial")
Like before, we can print and plot the fitted object, extract the coefficients at specific \(\lambda\)'s and also make predictions. For plotting, the optional arguments such as xvar and label are similar to the Gaussian. We plot against the deviance explained and show the labels.
plot(fit, xvar = "dev", label = TRUE)
Prediction is a little different for logistic from Gaussian, mainly in the option type. “link” and “response” are never equivalent and “class” is only available for logistic regression. In summary,
“link” gives the linear predictors
“response” gives the fitted probabilities
“class” produces the class label corresponding to the maximum probability.
“coefficients” computes the coefficients at values of s
“nonzero” retuns a list of the indices of the nonzero coefficients for each value of s.
For “binomial” models, results (“link”, “response”, “coefficients”, “nonzero”) are returned only for the class corresponding to the second level of the factor response.
In the following example, we make prediction of the class labels at \(\lambda = 0.05, 0.01\).
predict(fit, newx = x[1:5,], type = "class", s = c(0.05, 0.01))
## 1 2
## [1,] "0" "0"
## [2,] "1" "1"
## [3,] "1" "1"
## [4,] "0" "0"
## [5,] "1" "1"
For logistic regression, cv.glmnet has similar arguments and usage as Gaussian. nfolds, weights, lambda, parallel are all available to users. There are some differences in type.measure: “deviance” and “mse” do not both mean squared loss and “class” is enabled. Hence,
“mse” uses squared loss.
“deviance” uses actual deviance.
“mae” uses mean absolute error.
“class” gives misclassification error.
“auc” (for two-class logistic regression ONLY) gives area under the ROC curve.
For example,
cvfit = cv.glmnet(x, y, family = "binomial", type.measure = "class")
It uses misclassification error as the criterion for 10-fold cross-validation.
We plot the object and show the optimal values of \(\lambda\).
plot(cvfit)
cvfit$lambda.min
## [1] 0.0147554
cvfit$lambda.1se
## [1] 0.02578548
coef and predict are simliar to the Gaussian case and we omit the details. We review by some examples.
coef(cvfit, s = "lambda.min")
## 31 x 1 sparse Matrix of class "dgCMatrix"
## 1
## (Intercept) 0.24371065
## V1 0.06897051
## V2 0.66252343
## V3 -0.54275157
## V4 -1.13692797
## V5 -0.19143183
## V6 -0.95852340
## V7 .
## V8 -0.56528880
## V9 0.77454157
## V10 -1.45079395
## V11 -0.04363484
## V12 -0.06893736
## V13 .
## V14 .
## V15 .
## V16 0.36685293
## V17 .
## V18 -0.04013948
## V19 .
## V20 .
## V21 .
## V22 0.20882114
## V23 0.34014021
## V24 .
## V25 0.66310204
## V26 -0.33695819
## V27 -0.10570477
## V28 0.24317817
## V29 -0.22445233
## V30 0.11091002
As mentioned previously, the results returned here are only for the second level of the factor response.
predict(cvfit, newx = x[1:10,], s = "lambda.min", type = "class")
## 1
## [1,] "0"
## [2,] "1"
## [3,] "1"
## [4,] "0"
## [5,] "1"
## [6,] "0"
## [7,] "0"
## [8,] "0"
## [9,] "1"
## [10,] "1"
Like other GLMs, glmnet allows for an “offset”. This is a fixed vector of N numbers that is added into the linear predictor.
For example, you may have fitted some other logistic regression using other variables (and data), and now you want to see if the present variables can add anything. So you use the predicted logit from the other model as an offset in.
Multinomial Models
For the multinomial model, suppose the response variable has \(K\) levels \({\cal G}=\{1,2,\ldots,K\}\). Here we model
\[\mbox{Pr}(G=k|X=x)=\frac{e^{\beta_{0k}+\beta_k^Tx}}{\sum_{\ell=1}^Ke^{\beta_{0\ell}+\beta_\ell^Tx}}.\]
Let \({Y}\) be the \(N \times K\) indicator response matrix, with elements \(y_{i\ell} = I(g_i=\ell)\). Then the elastic-net penalized negative log-likelihood function becomes
\[
\ell(\{\beta_{0k},\beta_{k}\}_1^K) = -\left[\frac{1}{N} \sum_{i=1}^N \Big(\sum_{k=1}^Ky_{il} (\beta_{0k} + x_i^T \beta_k)- \log \big(\sum_{k=1}^K e^{\beta_{0k}+x_i^T \beta_k}\big)\Big)\right] +\lambda \left[ (1-\alpha)||\beta||_F^2/2 + \alpha\sum_{j=1}^p||\beta_j||_q\right].
\]
Here we really abuse notation! \(\beta\) is a \(p\times K\) matrix of coefficients. \(\beta_k\) refers to the kth column (for outcome category k), and \(\beta_j\) the jth row (vector of K coefficients for variable j).
The last penalty term is \(||\beta_j||_q\), we have two options for q: \(q\in \{1,2\}\).
When q=1, this is a lasso penalty on each of the parameters. When q=2, this is a grouped-lasso penalty on all the K coefficients for a particular variables, which makes them all be zero or nonzero together.
The standard Newton algorithm can be tedious here. Instead, we use a so-called partial Newton algorithm by making a partial quadratic approximation to the log-likelihood, allowing only \((\beta_{0k}, \beta_k)\) to vary for a single class at a time.
For each value of \(\lambda\), we first cycle over all classes indexed by \(k\), computing each time a partial quadratic approximation about the parameters of the current class. Then the inner procedure is almost the same as for the binomial case.
This is the case for lasso (q=1). When q=2, we use a different approach, which we wont dwell on here.
For the multinomial case, the usage is similar to logistic regression, and we mainly illustrate by examples and address any differences. We load a set of generated data.
data(MultinomialExample)
The optional arguments in glmnet for multinomial logistic regression are mostly similar to binomial regression except for a few cases.
The response variable can be a nc >= 2 level factor, or a nc-column matrix of counts or proportions.
Internally glmnet will make the rows of this matrix sum to 1, and absorb the total mass into the weight for that observation.
offset should be a nobs x nc matrix if there is one.
A special option for multinomial regression is type.multinomial, which allows the usage of a grouped lasso penalty if type.multinomial = "grouped". This will ensure that the multinomial coefficients for a variable are all in or out together, just like for the multi-response Gaussian.
fit = glmnet(x, y, family = "multinomial", type.multinomial = "grouped")
We plot the resulting object “fit”.
plot(fit, xvar = "lambda", label = TRUE, type.coef = "2norm")
The options are xvar, label and type.coef, in addition to other ordinary graphical parameters.
xvar and label are the same as other families while type.coef is only for multinomial regression and multiresponse Gaussian model. It can produce a figure of coefficients for each response variable if type.coef = "coef" or a figure showing the \(\ell_2\)-norm in one figure if type.coef = "2norm"
We can also do cross-validation and plot the returned object.
cvfit=cv.glmnet(x, y, family="multinomial", type.multinomial = "grouped", parallel = TRUE)
## Warning: executing %dopar% sequentially: no parallel backend registered
plot(cvfit)
Note that although type.multinomial is not a typical argument in cv.glmnet, in fact any argument that can be passed to glmnet is valid in the argument list of cv.glmnet. We also use parallel computing to accelerate the calculation.
Users may wish to predict at the optimally selected \(\lambda\):
predict(cvfit, newx = x[1:10,], s = "lambda.min", type = "class")
## 1
## [1,] "3"
## [2,] "2"
## [3,] "2"
## [4,] "1"
## [5,] "1"
## [6,] "3"
## [7,] "3"
## [8,] "1"
## [9,] "1"
## [10,] "2"
Back to Top
Poisson Models
Poisson regression is used to model count data under the assumption of Poisson error, or otherwise non-negative data where the mean and variance are proportional. Like the Gaussian and binomial model, the Poisson is a member of the exponential family of distributions. We usually model its positive mean on the log scale: \(\log \mu(x) = \beta_0+\beta' x\).
The log-likelihood for observations \(\{x_i,y_i\}_1^N\) is given my
\[
l(\beta|X, Y) = \sum_{i=1}^N (y_i (\beta_0+\beta' x_i) - e^{\beta_0+\beta^Tx_i}.
\]
As before, we optimize the penalized log-lielihood:
\[
\min_{\beta_0,\beta} -\frac1N l(\beta|X, Y) + \lambda \left((1-\alpha) \sum_{i=1}^N \beta_i^2/2) +\alpha \sum_{i=1}^N |\beta_i|\right).
\]
Glmnet uses an outer Newton loop, and an inner weighted least-squares loop (as in logistic regression) to optimize this criterion.
First, we load a pre-generated set of Poisson data.
data(PoissonExample)
We apply the function glmnet with the "poisson" option.
fit = glmnet(x, y, family = "poisson")
The optional input arguments of glmnet for "poisson" family are similar to those for others.
offset is a useful argument particularly in Poisson models.
When dealing with rate data in Poisson models, the counts collected are often based on different exposures, such as length of time observed, area and years. A poisson rate \(\mu(x)\) is relative to a unit exposure time, so if an observation \(y_i\) was exposed for \(E_i\) units of time, then the expected count would be \(E_i\mu(x)\), and the log mean would be \(\log(E_i)+\log(\mu(x)\). In a case like this, we would supply an offset \(\log(E_i)\) for each observation.
Hence offset is a vector of length nobs that is included in the linear predictor. Other families can also use options, typically for different reasons.
(Warning: if offset is supplied in glmnet, offsets must also also be supplied to predict to make reasonable predictions.)
Again, we plot the coefficients to have a first sense of the result.
plot(fit)
Like before, we can extract the coefficients and make predictions at certain \(\lambda\)'s by using coef and predict respectively. The optional input arguments are similar to those for other families. In function predict, the option type, which is the type of prediction required, has its own specialties for Poisson family. That is,
- “link” (default) gives the linear predictors like others
- “response” gives the fitted mean
- “coefficients” computes the coefficients at the requested values for
s, which can also be realized by coef function
- “nonzero” returns a a list of the indices of the nonzero coefficients for each value of
s.
For example, we can do as follows.
coef(fit, s = 1)
## 21 x 1 sparse Matrix of class "dgCMatrix"
## 1
## (Intercept) 0.61123371
## V1 0.45819758
## V2 -0.77060709
## V3 1.34015128
## V4 0.04350500
## V5 -0.20325967
## V6 .
## V7 .
## V8 .
## V9 .
## V10 .
## V11 .
## V12 0.01816309
## V13 .
## V14 .
## V15 .
## V16 .
## V17 .
## V18 .
## V19 .
## V20 .
predict(fit, newx = x[1:5,], type = "response", s = c(0.1,1))
## 1 2
## [1,] 2.4944232 4.4263365
## [2,] 10.3513120 11.0586174
## [3,] 0.1179704 0.1781626
## [4,] 0.9713412 1.6828778
## [5,] 1.1133472 1.9934537
We may also use cross-validation to find the optimal \(\lambda\)'s and thus make inferences.
cvfit = cv.glmnet(x, y, family = "poisson")
Options are almost the same as the Gaussian family except that for type.measure,
- “deviance” (default) gives the deviance
- “mse” stands for mean squared error
- “mae” is for mean absolute error.
We can plot the cv.glmnet object.
plot(cvfit)
We can also show the optimal \(\lambda\)'s and the corresponding coefficients.
opt.lam = c(cvfit$lambda.min, cvfit$lambda.1se)
coef(cvfit, s = opt.lam)
## 21 x 2 sparse Matrix of class "dgCMatrix"
## 1 2
## (Intercept) 0.031262578 0.185696196
## V1 0.619053001 0.575373801
## V2 -0.984550161 -0.932121975
## V3 1.525234026 1.470567302
## V4 0.231590890 0.196923579
## V5 -0.336659414 -0.304694503
## V6 0.001025945 .
## V7 -0.012829626 .
## V8 . .
## V9 . .
## V10 0.015983078 .
## V11 . .
## V12 0.030866986 0.025850171
## V13 -0.027970571 .
## V14 0.032750270 .
## V15 -0.005932709 .
## V16 0.017505507 .
## V17 . .
## V18 0.004026211 .
## V19 -0.033579041 .
## V20 0.012048941 0.009929548
The predict method is similar and we do not repeat it here.
Back to Top
Cox Models
The Cox proportional hazards model is commonly used for the study of the relationship beteween predictor variables and survival time. In the usual survival analysis framework, we have data of the form \((y_1, x_1, \delta_1), \ldots, (y_n, x_n, \delta_n)\) where \(y_i\), the observed time, is a time of failure if \(\delta_i\) is 1 or right-censoring if \(\delta_i\) is 0. We also let \(t_1 < t_2 < \ldots < t_m\) be the increasing list of unique failure times, and \(j(i)\) denote the index of the observation failing at time \(t_i\).
The Cox model assumes a semi-parametric form for the hazard
\[
h_i(t) = h_0(t) e^{x_i^T \beta},
\]
where \(h_i(t)\) is the hazard for patient \(i\) at time \(t\), \(h_0(t)\) is a shared baseline hazard, and \(\beta\) is a fixed, length \(p\) vector. In the classic setting \(n \geq p\), inference is made via the partial likelihood
\[
L(\beta) = \prod_{i=1}^m \frac{e^{x_{j(i)}^T \beta}}{\sum_{j \in R_i} e^{x_j^T \beta}},
\]
where \(R_i\) is the set of indices \(j\) with \(y_j \geq t_i\) (those at risk at time \(t_i\)).
Note there is no intercept in the Cox mode (its built into the baseline hazard, and like it, would cancel in the partial likelihood.)
We penalize the negative log of the partial likelihood, just like the other models, with an elastic-net penalty.
We use a pre-generated set of sample data and response. Users can load their own data and follow a similar procedure. In this case \(x\) must be an \(n\times p\) matrix of covariate values - each row corresponds to a patient and each column a covariate. \(y\) is an \(n \times 2\) matrix, with a column “time” of failure/censoring times, and “status” a 0/1 indicator, with 1 meaning the time is a failure time, and zero a censoring time.
data(CoxExample)
y[1:5,]
## time status
## [1,] 1.76877757 1
## [2,] 0.54528404 1
## [3,] 0.04485918 0
## [4,] 0.85032298 0
## [5,] 0.61488426 1
The Surv function in the package survival can create such a matrix. Note, however, that the coxph and related linear models can handle interval and other fors of censoring, while glmnet can only handle right censoring in its present form.
We apply the glmnet function to compute the solution path under default settings.
fit = glmnet(x, y, family = "cox")
All the standard options are available such as alpha, weights, nlambda and standardize. Their usage is similar as in the Gaussian case and we omit the details here. Users can also refer to the help file help(glmnet).
We can plot the coefficients.
plot(fit)
As before, we can extract the coefficients at certain values of \(\lambda\).
coef(fit, s = 0.05)
## 30 x 1 sparse Matrix of class "dgCMatrix"
## 1
## V1 0.37693638
## V2 -0.09547797
## V3 -0.13595972
## V4 0.09814146
## V5 -0.11437545
## V6 -0.38898545
## V7 0.24291400
## V8 0.03647596
## V9 0.34739813
## V10 0.03865115
## V11 .
## V12 .
## V13 .
## V14 .
## V15 .
## V16 .
## V17 .
## V18 .
## V19 .
## V20 .
## V21 .
## V22 .
## V23 .
## V24 .
## V25 .
## V26 .
## V27 .
## V28 .
## V29 .
## V30 .
Since the Cox Model is not commonly used for prediction, we do not give an illustrative example on prediction. If needed, users can refer to the help file by typing help(predict.glmnet).
Also, the function cv.glmnet can be used to compute \(k\)-fold cross-validation for the Cox model. The usage is similar to that for other families except for two main differences.
One is that type.measure only supports “deviance”(also default), which gives the partial-likelihood.
The other is in the option grouped. grouped = TRUE obtains the CV partial likelihood for the Kth fold by subtraction; by subtracting the log partial likelihood evaluated on the full dataset from that evaluated on the on the (K-1)/K dataset. This makes more efficient use of risk sets. With grouped=FALSE the log partial likelihood is computed only on the Kth fold, which is only reasonable if each fold has a large number of observations.
cvfit = cv.glmnet(x, y, family = "cox")
Once fit, we can view the optimal \(\lambda\) value and a cross validated error plot to help evaluate our model.
plot(cvfit)
As previously, the left vertical line in our plot shows us where the CV-error curve hits its minimum. The right vertical line shows us the most regularized model with CV-error within 1 standard deviation of the minimum. We also extract such optimal \(\lambda\)'s.
cvfit$lambda.min
## [1] 0.01594373
cvfit$lambda.1se
## [1] 0.04868986
We can check the active covariates in our model and see their coefficients.
coef.min = coef(cvfit, s = "lambda.min")
active.min = which(coef.min != 0)
index.min = coef.min[active.min]
index.min
## [1] 0.491297030 -0.174600537 -0.218648771 0.175112406 -0.186673099
## [6] -0.490250398 0.335197365 0.091586519 0.450169329 0.115921793
## [11] 0.017594799 -0.018364649 -0.002805776 -0.001422613 -0.023429230
## [16] 0.001687662 -0.008236046
coef.min
## 30 x 1 sparse Matrix of class "dgCMatrix"
## 1
## V1 0.491297030
## V2 -0.174600537
## V3 -0.218648771
## V4 0.175112406
## V5 -0.186673099
## V6 -0.490250398
## V7 0.335197365
## V8 0.091586519
## V9 0.450169329
## V10 0.115921793
## V11 .
## V12 .
## V13 0.017594799
## V14 .
## V15 .
## V16 .
## V17 -0.018364649
## V18 .
## V19 .
## V20 .
## V21 -0.002805776
## V22 -0.001422613
## V23 .
## V24 .
## V25 -0.023429230
## V26 .
## V27 0.001687662
## V28 .
## V29 .
## V30 -0.008236046
Back to Top
Sparse Matrices
Our package supports sparse input matrices, which allow efficient storage and operations of large matrices but with only a few nonzero entries. It is available for all families except for the cox family. The usage of sparse matrices (inherit from class "sparseMatrix" as in package Matrix) in glmnet is the same as if a regular matrix is provided.
We load a set of sample data created beforehand.
data(SparseExample)
It loads x, a 100*20 sparse input matrix and y, the response vector.
class(x)
## [1] "dgCMatrix"
## attr(,"package")
## [1] "Matrix"
Users can create a sparse matrix with the function sparseMatrix by providing the locations and values of the nonzero entries. Alternatively, Matrix function can also be used to contruct a sparse matrix by setting sparse = TRUE, but this defeats the purpose somewhat.
We can fit the model the same way as before.
fit = glmnet(x, y)
We also do the cross-validation and plot the resulting object.
cvfit = cv.glmnet(x, y)
plot(cvfit)
The usage of other functions are similar and we do not expand here.
Note that sparse matrices can also be used for newx, the new input matrix in the predict function. For example,
i = sample(1:5, size = 25, replace = TRUE)
j = sample(1:20, size = 25, replace = TRUE)
x = rnorm(25)
nx = sparseMatrix(i = i, j = j, x = x, dims = c(5, 20))
predict(cvfit, newx = nx, s = "lambda.min")
## 1
## [1,] 0.3825586
## [2,] -0.2172428
## [3,] -1.6621635
## [4,] -0.4174700
## [5,] -1.3940989
Back to Top
Appendix 1: Internal Parameters
Our package has a set of internal parameters which control some aspects of the computation of the path. The factory default settings are expected to serve in most cases, and users do not need to make changes unless there are special requirements.
There are several parameters that users can change:
fdev - minimum fractional change in deviance for stopping path; factory default = 1.0e-5
devmax - maximum fraction of explained deviance for stopping path; factory default = 0.999
eps - minimum value of lambda.min.ratio (see glmnet); factory default= 1.0e-6
big - large floating point number; factory default = 9.9e35. Inf in definition of upper.limit is set to big
mnlam - minimum number of path points (lambda values) allowed; factory default = 5
pmin - minimum null probability for any class; factory default = 1.0e-5
exmx - maximum allowed exponent; factory default = 250.0
prec - convergence threshold for multi-response bounds adjustment solution; factory default = 1.0e-10
mxit - maximum iterations for multiresponse bounds adjustment solution; factory default = 100
factory - If TRUE, reset all the parameters to the factory default; default is FALSE
We illustrate the usage by an example. Note that any changes made hold for the duration of the R session, or unless they are changed by the user with a subsequent call to glmnet.control.
data(QuickStartExample)
fit = glmnet(x, y)
print(fit)
##
## Call: glmnet(x = x, y = y)
##
## Df %Dev Lambda
## [1,] 0 0.00000 1.631000
## [2,] 2 0.05528 1.486000
## [3,] 2 0.14590 1.354000
## [4,] 2 0.22110 1.234000
## [5,] 2 0.28360 1.124000
## [6,] 2 0.33540 1.024000
## [7,] 4 0.39040 0.933200
## [8,] 5 0.45600 0.850300
## [9,] 5 0.51540 0.774700
## [10,] 6 0.57350 0.705900
## [11,] 6 0.62550 0.643200
## [12,] 6 0.66870 0.586100
## [13,] 6 0.70460 0.534000
## [14,] 6 0.73440 0.486600
## [15,] 7 0.76210 0.443300
## [16,] 7 0.78570 0.404000
## [17,] 7 0.80530 0.368100
## [18,] 7 0.82150 0.335400
## [19,] 7 0.83500 0.305600
## [20,] 7 0.84620 0.278400
## [21,] 7 0.85550 0.253700
## [22,] 7 0.86330 0.231200
## [23,] 8 0.87060 0.210600
## [24,] 8 0.87690 0.191900
## [25,] 8 0.88210 0.174900
## [26,] 8 0.88650 0.159300
## [27,] 8 0.89010 0.145200
## [28,] 8 0.89310 0.132300
## [29,] 8 0.89560 0.120500
## [30,] 8 0.89760 0.109800
## [31,] 9 0.89940 0.100100
## [32,] 9 0.90100 0.091170
## [33,] 9 0.90230 0.083070
## [34,] 9 0.90340 0.075690
## [35,] 10 0.90430 0.068970
## [36,] 11 0.90530 0.062840
## [37,] 11 0.90620 0.057260
## [38,] 12 0.90700 0.052170
## [39,] 15 0.90780 0.047540
## [40,] 16 0.90860 0.043310
## [41,] 16 0.90930 0.039470
## [42,] 16 0.90980 0.035960
## [43,] 17 0.91030 0.032770
## [44,] 17 0.91070 0.029850
## [45,] 18 0.91110 0.027200
## [46,] 18 0.91140 0.024790
## [47,] 19 0.91170 0.022580
## [48,] 19 0.91200 0.020580
## [49,] 19 0.91220 0.018750
## [50,] 19 0.91240 0.017080
## [51,] 19 0.91250 0.015570
## [52,] 19 0.91260 0.014180
## [53,] 19 0.91270 0.012920
## [54,] 19 0.91280 0.011780
## [55,] 19 0.91290 0.010730
## [56,] 19 0.91290 0.009776
## [57,] 19 0.91300 0.008908
## [58,] 19 0.91300 0.008116
## [59,] 19 0.91310 0.007395
## [60,] 19 0.91310 0.006738
## [61,] 19 0.91310 0.006140
## [62,] 20 0.91310 0.005594
## [63,] 20 0.91310 0.005097
## [64,] 20 0.91310 0.004644
## [65,] 20 0.91320 0.004232
## [66,] 20 0.91320 0.003856
## [67,] 20 0.91320 0.003513
We can change the minimum fractional change in deviance for stopping path and compare the results.
glmnet.control(fdev = 0)
fit = glmnet(x, y)
print(fit)
##
## Call: glmnet(x = x, y = y)
##
## Df %Dev Lambda
## [1,] 0 0.00000 1.6310000
## [2,] 2 0.05528 1.4860000
## [3,] 2 0.14590 1.3540000
## [4,] 2 0.22110 1.2340000
## [5,] 2 0.28360 1.1240000
## [6,] 2 0.33540 1.0240000
## [7,] 4 0.39040 0.9332000
## [8,] 5 0.45600 0.8503000
## [9,] 5 0.51540 0.7747000
## [10,] 6 0.57350 0.7059000
## [11,] 6 0.62550 0.6432000
## [12,] 6 0.66870 0.5861000
## [13,] 6 0.70460 0.5340000
## [14,] 6 0.73440 0.4866000
## [15,] 7 0.76210 0.4433000
## [16,] 7 0.78570 0.4040000
## [17,] 7 0.80530 0.3681000
## [18,] 7 0.82150 0.3354000
## [19,] 7 0.83500 0.3056000
## [20,] 7 0.84620 0.2784000
## [21,] 7 0.85550 0.2537000
## [22,] 7 0.86330 0.2312000
## [23,] 8 0.87060 0.2106000
## [24,] 8 0.87690 0.1919000
## [25,] 8 0.88210 0.1749000
## [26,] 8 0.88650 0.1593000
## [27,] 8 0.89010 0.1452000
## [28,] 8 0.89310 0.1323000
## [29,] 8 0.89560 0.1205000
## [30,] 8 0.89760 0.1098000
## [31,] 9 0.89940 0.1001000
## [32,] 9 0.90100 0.0911700
## [33,] 9 0.90230 0.0830700
## [34,] 9 0.90340 0.0756900
## [35,] 10 0.90430 0.0689700
## [36,] 11 0.90530 0.0628400
## [37,] 11 0.90620 0.0572600
## [38,] 12 0.90700 0.0521700
## [39,] 15 0.90780 0.0475400
## [40,] 16 0.90860 0.0433100
## [41,] 16 0.90930 0.0394700
## [42,] 16 0.90980 0.0359600
## [43,] 17 0.91030 0.0327700
## [44,] 17 0.91070 0.0298500
## [45,] 18 0.91110 0.0272000
## [46,] 18 0.91140 0.0247900
## [47,] 19 0.91170 0.0225800
## [48,] 19 0.91200 0.0205800
## [49,] 19 0.91220 0.0187500
## [50,] 19 0.91240 0.0170800
## [51,] 19 0.91250 0.0155700
## [52,] 19 0.91260 0.0141800
## [53,] 19 0.91270 0.0129200
## [54,] 19 0.91280 0.0117800
## [55,] 19 0.91290 0.0107300
## [56,] 19 0.91290 0.0097760
## [57,] 19 0.91300 0.0089080
## [58,] 19 0.91300 0.0081160
## [59,] 19 0.91310 0.0073950
## [60,] 19 0.91310 0.0067380
## [61,] 19 0.91310 0.0061400
## [62,] 20 0.91310 0.0055940
## [63,] 20 0.91310 0.0050970
## [64,] 20 0.91310 0.0046440
## [65,] 20 0.91320 0.0042320
## [66,] 20 0.91320 0.0038560
## [67,] 20 0.91320 0.0035130
## [68,] 20 0.91320 0.0032010
## [69,] 20 0.91320 0.0029170
## [70,] 20 0.91320 0.0026580
## [71,] 20 0.91320 0.0024220
## [72,] 20 0.91320 0.0022060
## [73,] 20 0.91320 0.0020100
## [74,] 20 0.91320 0.0018320
## [75,] 20 0.91320 0.0016690
## [76,] 20 0.91320 0.0015210
## [77,] 20 0.91320 0.0013860
## [78,] 20 0.91320 0.0012630
## [79,] 20 0.91320 0.0011500
## [80,] 20 0.91320 0.0010480
## [81,] 20 0.91320 0.0009551
## [82,] 20 0.91320 0.0008703
## [83,] 20 0.91320 0.0007930
## [84,] 20 0.91320 0.0007225
## [85,] 20 0.91320 0.0006583
## [86,] 20 0.91320 0.0005999
## [87,] 20 0.91320 0.0005466
## [88,] 20 0.91320 0.0004980
## [89,] 20 0.91320 0.0004538
## [90,] 20 0.91320 0.0004135
## [91,] 20 0.91320 0.0003767
## [92,] 20 0.91320 0.0003433
## [93,] 20 0.91320 0.0003128
## [94,] 20 0.91320 0.0002850
## [95,] 20 0.91320 0.0002597
## [96,] 20 0.91320 0.0002366
## [97,] 20 0.91320 0.0002156
## [98,] 20 0.91320 0.0001964
## [99,] 20 0.91320 0.0001790
## [100,] 20 0.91320 0.0001631
We set fdev = 0 to continue all along the path, even without much change. The length of the sequence becomes 100, which is the default of nlambda.
Users can also reset to the default settings.
glmnet.control(factory = TRUE)
The current settings are obtained as follows.
glmnet.control()
## $fdev
## [1] 1e-05
##
## $eps
## [1] 1e-06
##
## $big
## [1] 9.9e+35
##
## $mnlam
## [1] 5
##
## $devmax
## [1] 0.999
##
## $pmin
## [1] 1e-09
##
## $exmx
## [1] 250
##
## $prec
## [1] 1e-10
##
## $mxit
## [1] 100
Back to Top
Appendix 2: Comparison with Other Packages
Some people may want to use glmnet to solve the Lasso or elastic-net problem at a single \(\lambda\). We compare here the solution by glmnet with other packages (such as CVX), and also as an illustration of parameter settings in this situation.
Warning: Though such problems can be solved by glmnet, it is not recommended and is not the spirit of the package. glmnet fits the entire solution path for Lasso or elastic-net problems efficiently with various techniques such as warm start. Those advantages will disappear if the \(\lambda\) sequence is forced to be only one value.
Nevertheless, we still illustrate with a typical example in linear model in the following for the purpose of comparison. Given \(X, Y\) and \(\lambda_0 > 0\), we want to find \(\beta\) such that
\[
\min_{\beta} ||Y - X\beta||_2^2 + \lambda_0 ||\beta||_1,
\]
where, say, \(\lambda_0 = 8\).
We first solve using glmnet. Notice that there is no intercept term in the objective function, and the columns of \(X\) are not necessarily standardized. Corresponding parameters have to be set to make it work correctly. In addition, there is a \(1/(2n)\) factor before the quadratic term by default, we need to adjust \(\lambda\) accordingly. For the purpose of comparison, the thresh option is specified to be 1e-20. However, this is not necessary in many practical applications.
fit = glmnet(x, y, intercept = F, standardize = F, lambda = 8/(2*dim(x)[1]), thresh = 1e-20)
We then extract the coefficients (with no intercept).
beta_glmnet = as.matrix(predict(fit, type = "coefficients")[-1,])
In linear model as here this approach worked because we were using squared error loss, but with any nonlinear family, it will probably fail. The reason is we are not using step length optimization, and so rely on very good warm starts to put us in the quadratic region of the loss function.
Alternatively, a more stable and strongly recommended way to perform this task is to first fit the entire Lasso or elastic-net path without specifying lambda, but then provide the requested \(\lambda_0\) to predict function to extract the corresponding coefficients. In fact, if \(\lambda_0\) is not in the \(\lambda\) sequence generated by glmnet, the path will be refitted along a new \(\lambda\) sequence that includes the requested value \(\lambda_0\) and the old sequence, and the coefficients will be returned at \(\lambda_0\) based on the new fit. Remember to set exact = TRUE in predict function to get the exact solution. Otherwise, it will be approximated by linear interpolation.
fit = glmnet(x, y, intercept = F, standardize = F, thresh = 1e-20)
beta_glmnet = as.matrix(predict(fit, s = 8/(2*dim(x)[1]), type = "coefficients", exact = T)[-1,])
We also use CVX, a general convex optimization solver, to solve this specific Lasso problem. Users could also call CVX from R using the CVXfromR package and solve the problem as follows.
library(CVXfromR)
setup.dir = "change/this/to/your/cvx/directory"
n = dim(x)[1]; p = dim(x)[2]
cvxcode = paste("variables beta(p)",
"minimize(square_pos(norm(y - x * beta, 2)) + lambda * norm(beta, 1))",
sep = ";")
Lasso = CallCVX(cvxcode, const.var = list(p = p, x = x, y = y, lambda = 8), opt.var.names = "beta", setup.dir = setup.dir, matlab.call = "change/this/to/path/to/matlab")
beta_CVX = Lasso$beta
For convenience here, the results were saved in CVXResult.RData, and we simply load in the results.
data(CVXResults)
In addition, we use lars to solve the same problem.
require(lars)
fit_lars = lars(x, y, type = "lasso", intercept = F, normalize = F)
beta_lars = predict(fit_lars, s = 8/2, type = "coefficients", mode = "lambda")$coefficients
The results are listed below up to 6 decimal digits (due to convergence thresholds).
cmp = round(cbind(beta_glmnet, beta_lars, beta_CVX), digits = 6)
colnames(cmp) = c("beta_glmnet", "beta_lars", "beta_CVX")
cmp
## beta_glmnet beta_lars beta_CVX
## V1 1.389118 1.389118 1.389118
## V2 0.007991 0.007991 0.007991
## V3 0.731234 0.731234 0.731234
## V4 0.031119 0.031119 0.031119
## V5 -0.866793 -0.866793 -0.866793
## V6 0.564867 0.564867 0.564867
## V7 0.069678 0.069678 0.069678
## V8 0.358346 0.358346 0.358346
## V9 0.000000 0.000000 0.000000
## V10 0.070565 0.070565 0.070565
## V11 0.173464 0.173464 0.173464
## V12 -0.027472 -0.027472 -0.027472
## V13 -0.017960 -0.017960 -0.017960
## V14 -1.138053 -1.138053 -1.138053
## V15 -0.070990 -0.070990 -0.070990
## V16 0.000000 0.000000 0.000000
## V17 0.000000 0.000000 0.000000
## V18 0.000000 0.000000 0.000000
## V19 0.000000 0.000000 0.000000
## V20 -1.097528 -1.097528 -1.097528
Back to Top
References
Jerome Friedman, Trevor Hastie and Rob Tibshirani. (2008).
Regularization Paths for Generalized Linear Models via Coordinate Descent
Journal of Statistical Software, Vol. 33(1), 1-22 Feb 2010.
Noah Simon, Jerome Friedman, Trevor Hastie and Rob Tibshirani. (2011).
Regularization Paths for Cox’s Proportional Hazards Model via Coordinate Descent
Journal of Statistical Software, Vol. 39(5) 1-13.
Robert Tibshirani, Jacob Bien, Jerome Friedman, Trevor Hastie, Noah Simon, Jonathan Taylor, Ryan J. Tibshirani. (2010).
Strong Rules for Discarding Predictors in Lasso-type Problems
Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(2), 245-266.
Noah Simon, Jerome Friedman and Trevor Hastie (2013).
A Blockwise Descent Algorithm for Group-penalized Multiresponse and Multinomial Regression
(in arXiv, submitted)
glmnet/inst/doc/Coxnet.pdf 0000644 0001762 0000144 00000153405 12672525200 015226 0 ustar ligges users %PDF-1.5
%¿÷¢þ
1 0 obj
<< /Type /ObjStm /Length 2423 /Filter /FlateDecode /N 32 /First 237 >>
stream
xœíY[wÛ6~ß_·&g×q%¸§§çØrܸMšÔvoÚ<Ð-q#‹*)¹N~ý~3 $ê’ÄNzö©'&43˜o. -a„R™°B£uÂX#¼°>©p™A¤Ê‹L«„JDæØAe˜¦AZ+‘.ŠH‡q2xyÊ9tbȼ`(LÊ„ÖÚ íÀ§•0:Á¯&h#ðgµÁ¯Å: 1 —³N{ôce6Ã{.¤Xx&¼²Xz"¼ F%|êa˜ÆÚI¨Á¯Kÿñý÷BžTÓ¹`VXF$ IðÚ–´·$S Ð’Ái`ÃÔ?ù²®çÅ\üòøDÈ‹ân.ÞaˆÕö¡¶˜ÎÀç>/†e~TÝŸ¤¹Ìõtê å^È0O¾ÌkL ¬™ÿ¬hªE=(¡cÇŇYA\£b©ã(o
6OöOŸ¾zrüÏþó3@/Oʺ™÷Çy
´" ÇE3¨ËÙ¼ª ;÷,oy0y¾¸š³|Ò¢–Ê¢ð7åp>nÈ9Ñô-ݯ.ßž¾fݺ«q²£;lëF„ÜK·Ù¯û?‡ýóÓgÐ}t¹¥<ÛUîv Oî§\ïWþúèÇþ¯GlxÒÕ
îèNwtÛûéVûu÷///O^@÷óÓ/+÷;ÊÕý”'û•ÿtyvòö ÊÏŸ]\lªG6m«7ÉWªWÙ~õO.^õ%§ï(÷»ÊÕW:]…®ò~µÀ"ìçrØ ƒc8ªøó®››ÍrÊÞºóõeE^¾¸úo1ˆ"OoP@ÝrÊNÅñßZqÔ'KN¿.òyYMóy! ÿ['J%VüÓ6yÜ2 øGgxy^
?É;‡‹AAœB÷”îit^”óIA=?Öùl\ñb1Ÿ-æw£àm>»>.§£« &Ë_òòàçkGÆÓbr[ÌËAŽ šª!ÄÑÞŶ·?_;WAÓÃiS®;ŽËëëP´¿ ì[ò¦œ.lBþ±¨æŤ¸†¨`+ Ð4¥£:¿…š|° €rPÖƒÅÍõ¤¸r^N†èºÉu5òª.ˆ3óÁ€ü)‡%´5%`éM«ù°¸²æ… “IÞé /¦£¼^ÜLò&V£jZ¼_Î’›Y>(Öá½2Vÿ†ÙýýüÅÏ»¿qý¿áìMÖÃg¥w=|dª´µ¶‡ÐÄõðqlÒžÃW„§Ögè7Πߥ¾§ñYezøhN4æÚ̶Ü.MH~U/öÓ&à¤Ò„×Z¼£Q¯5KöœH²@šS¬5IHF û]B6[ëɯð ú}À ãù,e_2'Ž %Š£‡|Ï9ÍV9DZÆm0†ñ£6zÃ$<ºÇgä›ü†ÈÑ e Y©Eá™vÜ»¨fMu¹º`®C¡ ë,ïkƒ¬j6¥¥Ç6öp %ÖÍòdE ïÈ°4Â
¾ìŽH(‘±ÛYNìã¹tãiÞÊwN•è•z<^‘赡ÛTÖz£Cw¼Ç½>Õk+Ü&wä0¨ÈþŒåE‡tF£‘Ží:&88Ð<…âC[|’[K‰RŒ
ôØÒ;¥69ËÈ æ
˜m¸,Xx‘Ê]€·Ö)§ ²
<¿¤ŽÓÕ§T˜!`ýp¥\7ˆ¨$ŠÝš"ß‘‡Þ2 )VÊë–ŒéjÕ¤§¥v,rHÛŒ…Tü«È …,mù©wi?÷fÔÆžŒ‹;Ï[ñÆqëh 3B»¬Ç"B‘&ÏèåFS!ª-DßÐ’”å“"3è‡ ‹mìI
Ê*V°ôNj9C<•>l ½a.ÚÜ4•9ž©9!àåŒg™¥¼8³•ÍåÒÙ•lâƒPô«ÇÕtmð&µ~#1Ö8Þ/—”θ†¯Æy÷9¤|à÷ Ú¹+®8f8äLàÏôzŒay
6U;‘+9·±‡à3¼£Eì‚3ñ…æ½µ{¡¦½0rò£Z –v¨Æðx+•g·t ýÉ)Úw£†V'·ïªÝ çi/Š«¶.¬,èÒÝž(Ù&z]Z§ôMÒåiçÆ~.»-6©æÂËs»2™Ç`…tßHvÍɣѢH·-kÇç¦Ã†, ŸP²ŸÏžåhŒ×€s ÖéüH>‘§2—9”…,åDÞÈ©¬$*r&k9—ó?+¹w8=Ò¹Ÿä (x2ÉGh/¡Žâ9øÀq B51ñ3ç] ?)qî4íÑ:âñoë~étžOÊÁátfºF˜7¯éf´{&ì\>ÈËÖ"TêåQñS&›®É—2ŸÌƹœä7Wà öÎäüp? & iO°a×°máÖER×Âe—6"×¾d¢Q2ñHöå <û\¾òµ| _Á˃êæ&g_ó4YÜ óf,‹?ùD^Ëkú+%ýU‹ZŽäXŽ?ÌÆÅ¡ñ¾
Ži‰ØXFÈŒî8è| ©š…Ίº¬†ñì {jÙÈfBšPãº(Ú°º•ôò£üXÔÕý<@ñÕº ¡‘ùlÓÙ¶ 6ï÷Ƙÿrˆ=ÿC ÿTþ$‘gò `ôWØ÷dZ³Íí ÁNÀ da7"m²
Çæ}ò~8Ü_
G 0 ÈT“jÊ Œ 2?Ž#?nZë>e‰ÞÏ,¶ »é|«vòoãæ~¯µ¨Y_´6Ù¶–î;ÖòkÇù}.«çHÒíõŸlþfÃkù—x]Ô
4 |¥Yå2íTÏâƒè€Žo¿?zS\é¾ÀY¡—þþX¼ŸåóqSäâv5 ·ãQn+„®Èë|6+†@*Ÿ4|/ÿ?:îfendstream
endobj
34 0 obj
<< /Filter /FlateDecode /Length 2112 >>
stream
xÚåXI¯ãD¾Ï¯ˆæ‚#M<îÅ$͈AÀžà ülç%R?Ùó âÇS«—ÄAâÆ!麺ªú«¥ýÕÝ‹×ïœ[™4´&ö«»-t£0ó«$ñ¡Í’Õ]µú%x³6A³¶IðüêþÓõÆG>ø¡†ÑÃþð+à×îÿ@‚j½qQ4ÝIc¦o‘¢ëàßÇßî¾yýÎä+cÂ<Ž-Ê‘ûÐgnµ±Y˜å–åø ©‹ °²qð#n‡ß‘Ž.«±ažÇк0÷ïû¦n…‰kÜŸïÖ™
ZaòòbÁlò¬Á8pÐN̪d¥ ÏZ6‡€PPëÍå
ñÚ“é©[0“œ{Bn=®óy
^N”ÈiÇBüßL°»ã«£0ŒÅwäüÍ]ð]ƒä‚>ðTË Dî£7ûº.ÞãÞgì{D&ÍÕD:àäÕA9è?B“õ‰m‡Ó`öçuæÈ&.
Þ÷ÌjßñÔ»C·p!Dp>Ñ$H‹¢UjZ ÷jÓj¦·ÄŒ_£8â«ÝX+ò-ŠóÌ3ÿÌ[âwÏêc5ziÜ
¯'‘à_¢¢;¶qÓQá…‚œ:ŠJ”
í{&ׇÎ=ÛM$n¶ÊTÐõû:Ø2$û¹g¤.]ý^á„'ž+=Öe¼ÀÝ^ü_VŽZvá…|…ã²{ZÚ8ÂÌ¥W0ƒ)<¶„³Z¯É£ÈÔ°¼Õ[ƒ~QÊœ áóØj´$ð)
›¡#|@¹jVƒƒÂÚqWs,õÆñá$|yf‰q=· Ó¡éôøÓ¯+°Ï¶µÙ&)T\ ²—[ ^›!Èò5óÙÇ™MéfwóŸE 4ë~\rÊÂQ+É·ç)dÒš3#û•ÂcH–ŽñÖºzdp&øPwêš0¤ßqPnNäj”›Òàýi²A=4ã„Vt¢“1¥lœ†Î8PŽ”zºV;£ªAÖß:’2¦—ª`Åiá„8
o•Áié€Ì
Üsj¹f…K¼’=^³8da’çsô ÉÔš1œ%‰¡FâxÌ–ê¥H"ø:kx†©?¹© Î -˜cfÏ‘ }
^NéU’Ë$¹/Ö›Òµæ×{¹Ÿb¸'£‚¿$ÕXÈ£ÛÙ—L¦çå·ÏûVžÕ…–ò ÷P†eæ´d(×› –Öÿ׊uŠ ‚‹Â†É
^"{rÒT³ ÷Ÿ„ãÃ…Ùzé¿¥Ì%Õ‰•Y EªJÖ{i?B¢brL!N¿%{œpܯ5ÄèSøúÄñ‚§ž…WîýÜöZ9¹ÔC½~âq©¯) hÙ„ýíø– J¼nä,<›¥ ïya3õ0‰U@Ìï^ò ž¸ñÎ¥½d•¥úÉ„PÅj´óݵò0K‡tú9ðói`È^ü¡ê¨¡´Fh…KÞ
I'4._á¤ì&©|ãÓ|4¯çjÝ/ø]êCŸ˜±¬ã;|ÓŠ_}>ºW4ü®´K’0ÊÜhBÏðT¥bµè¦ÑÏÇã‹Á{(ÑèØ#û|ä¹[ïh\£íã`?q‘v^ø‹éŒ xõsÓ Z<ärE?ü}ë)”ûøËTŠB¤žC8À¯ دßiJI}r?3Õã ˜~hä¸Ì‚Ó} bÃG¤<6
#J×ú />™º1Š9°Ôúa¤âIÍøåµl(s6Ä$[YËÛ4Îéy¤ô .†ïZ¥¤$¯OÁñÔ-¿ç{â1ýž‡ãîqÄ Ž›³t
ÙPkM#ñv)¾ÝÖd¨t:œø¤ç ²’TÕb L›÷êýnòÂ…¾àUÁÉ(¥ 7Q »¸²€ÓÈ^\`øÑI«”ì’f ût7×T·³8ë“àvýx‘é„¢Ÿ|¯GênÿÇ’YrÁ@µä®ýè“óRPÔˆ0/ÇÍÿ&4ýcÊ»mÿ….Òt’Ãó:†gJ Æ6Öù@÷âíÝ‹¿ ú
qendstream
endobj
35 0 obj
<< /BBox [ 0 0 432 432 ] /FormType 1 /PTEX.FileName (./Coxnet-003.pdf) /PTEX.InfoDict 14 0 R /PTEX.PageNumber 1 /Resources << /ExtGState << >> /Font << /F1 15 0 R /F2 16 0 R >> /ProcSet [ /PDF /Text ] >> /Subtype /Form /Type /XObject /Length 3004 /Filter /FlateDecode >>
stream
xœ¥šOضÅïû)t´VÅ?ÉkÑ6@M²@Inã¦N½ jí×ï{óF$×uJ=x½o‡”Dr8¿Jaù| ËËß ¾Ä¿mݶm™~õþXöå_ß|·lË÷aù ÿ~|´/¿!ò—’Öœ—§%íaMÅå»åëí.cn/ayØ%‡=¦° µÛ]NöÖ=
»ä°?{ þ|ÃþëÇŒúý¿ú]ÄÄ<þeÑÛh_ÙOý!lÛŽ%—uËãÓòâUZ÷—Ëã¿}äE±ÿaâYÿíBÿ'jê /Ý?–Ɖ|ÖÿÊýSjì9õãþ{[·¼Ô²††9—J{]Sµ9ÿÀl+.5¬¡àÊÃìrØãQÖ6ì.‡ÝowÚ§»Ïƒ›ôjhk9–ºc ™Ove[›÷þèº|²w<à¦ÁzÇë÷¦“ïýÁšø>xÈ›µô}zÊTâÚhŸÃçòlüµÅŠÛ§ kA“ÊŸ|Äw?ÿðí‹/^?ýéû×ß¾¼0Ö`0Öy¡o^üáå’·åÅë÷/—Wüå o_¿[¾xû·—K„zóîí_þùûå7o^. úŸo_ÿôç7/¿[ ?ç 1¦e gÊ‘O›ùùû7Ë —Ÿ0âãHËüódª¸hYÂCa ’¬;;Y€ª°lff7 ûe‡ç·D{Úi—„=·Ç5 ´oÖ_öxÈŽXWq}„ €’5ŸÛ%ÁÕb¢=XIØ‹_Ç£àúpfÔ$ì¾ ÓŽ@¸ÑŽÙ Ý$íIöl¿Ò¾Ñ.I»_?U®c3—…]’÷ßÜŽ‘7Ú›ì&a¯ àcY«žŸÛÝ%Æ¿ùõcäTÀžíù%aO>ÿáXF{5»$ç?¸ ¦ù·õ“„½ùø¶}Ýmþ«_² çüÓ0Õ-¯9 LV۲׆nf7Ép¨ç‹µ®õ ½rü.awÿˆp˜ˆë š—huö‡ÃàV
’°>8 š6L{5»Ixé®ñG8 –š¼‹Ö@ºmkrÃcàz!Žñ´¸Æ‚ù G¸LÞÑ OÆ)t?´D17N~àÔÙ3Jc&Î+ÀioQ×Ín!1‡3K¨ÜO›®*M0AÅÍ$úGimj Q¹ŸÆhSþãF;õf
ÕÁ7‰+ÄÒxìæwód\5î Z¸Æs7ŸMvÿŸüê3««ÇåñýbPDý’dI;c¶Ãy ÈÃúø¾ŸÊªw0®nÍë½±_šN6ëÞ› ÛÊõÞX«jO¾Ý÷ÞìLù8ߺ\ììÄÙ v½7‚G0Ö 7zG#åÍÞØó:ˆ»Þèm -“¿;Þ‚XQíÉnôÞì”Ü^:^ï
Žó?x7õ`ÙO»Õ;1ŠÖz¾ ¼Ô;"IØ,sº³C™ˆe{7nÌÓ´jYW»±¿cµ7¢oÝnô.V| 0ßñÔˆØ,Ÿ»³ÞäÀag‡ûqóäÎä°Üêø¢Ùç×{#/ h˜XÞ7b 3&déΤ#¸ða|ÇÛ]¢ mÝLw{Ý¡QD|a–Â-2Qæ›”`ï¯wg
ž-Yºõð› ³2»c˜ó
ÄrÃé‚N ˜– t˜À„µÝyxž5(EßnL]ÐQD@êo 5褂É|¼áóAÌô;SÇs•é†× ƒð=á ¸’°6Ëw¦N‡(<šÜîŒ]g,<·¼gƒŽ`죬;ÝuBÃ*|¿*>ì Üênç;vjqgÝuüÃC“íŽÓêtˆg6éetvdG@7¼F'KüîŽË*egVoyJñ{‘t£·Î¬âM6ëD+åýãqæÿù¾ U+Jó„DÍSŽZ/|Vˆ¹D>?÷ŽW>ªvù xt/Ç•îˆú|ë=u¿twNŸS÷#\ù"s³‰©û~éƒÐ‰¬©{º2ñÌ3QLÝc¹Ò å ì©{¸r÷êw¿rsž§ Ï–=]鸗žõ¾²è|¹°=ë}eÍù¦ä¹Ãö%ÿ ¿GüÄÛº ¢òö_¾LÔ§zãÓh“çGŸöžÆ¾Uì
$§_>ü8âaßendstream
endobj
36 0 obj
<< /Filter /FlateDecode /Length 1730 >>
stream
xÚÍXYÛ6~ß_¡}˜/ @S I })Ši$Z[>[ÚZvœä×w.ê°l6ÉC±Ðš"‡Ã™áÌÇz:»yü<
U¤&fËHÛBe.²$UÚÑl½Šg뉎«É›Ùb m´Ê2 %,öH áÙÃ󎥇ªW>Ƀð…kø7?À¿
ËUÕâ€l µ8âÙï&ÞÇå–%%*-YGkT‡}
÷as mÙ ª ¿Tï¡}7™š4®æÐDÙjÁc·“Ç Ð·hšeŒ¦Z«Â{öjµE=èV…ÓÔdêÿ ÀUXΦiü:ÑN?‚žÌÅ'´F}ÒØPh±Ž_ÔãÞBìš—m% žØÀq„ú«ÁD0äŠË+lâú¢2P_¶ü‡c0§âÍ^ð˜ P,Ç?X;åf//cøZŽö´¶aü|ÉÍ<ÔÈëîY~ä «ß`JæT¶Ð3λ+á ÁE—®9ø0½éô—˜€^÷™éåp®l¹o|P— z-, ”ÎY¨"8×é< ¶ [hy&™†u½=
RV‹ptbè)€,íOÍãšÇZ)èÁT úu5$ŸFgde›x– ÄÜ1t– ~9ö’စæ-óœí8™xFÈ]z @
Í’(Û‘Q—p-äœç'0¤”ás›ÒŒ·¶]û^¸f@ BàP(™$ë&„¤^A$4q¶ _d%€[Üv‡ ˜òa’;ŒaZØø9¶opæÊëf׿ßvÉ2\;>µ7ƒ$ÔL”J ¶š{›À‘[
;G¼
ÚäœX߇Æ_Å- 9GSkÜfHã¿1¢Ò:´c€ƒo¹÷rN†IóüúK¾Á,fg6ðä°NÓVŸòYÈ ä*¿”ݽøˆù_GWAË‚Øgk%)i3±ìÑÊýšKGiK5øèšN¿ `è°X!Yý9¾™ÈÙ3áš•³eÙžõׂ%-Où šÓžÚ
g ¸
YÞ#–”¹O¾%ì©ïùV‰c¿ˆýþ°™¨æ…¨XHßû ÇNƒ“v.mqî+"ö4ŸôÔô,ÿ¶ízØlM´úaqD
~ÉÖGäZî;ÿ‡Ô9»¤…ò¾—Ð}¸>PÖB¹úCÿòžë
•6Ì¿»\!S^wŸ ¶\ìpX2 ’î{¸ È$ýÕƒ°§»Ð:È$íÆ5ÂÁ ¶¡{ë•b÷œ>>£nù³CC[b¶bÐ94×.øÎ(ãÝç/ø¹òYÜ'W²$S™ë¾Ó¥qd@ dak AÌ”5ÅU¤|X!<äʪ—Å9ÝSi%¼¢ð¸/Fñï_{ÕF›SyðÊË“ŒÅÎoâ™LÉ﹑ ht4îeݹZø/Ç÷î’] ¯º~¢®sPœ ê1¬êü„çÞëgo @pvY)Æç*ÑÝ·Ê®}Kl÷!,Oĸ’\*³Vå™ÿR]àSwð çQEröù«*‰d¹Ò$9Й 1î-w
R´›Qò…/–¨eú&ŸNl‡8´a ùÒLÕHlSæi•6Ê„Ï[ÿÀÎ xendstream
endobj
37 0 obj
<< /Filter /FlateDecode /Length1 991 /Length2 3317 /Length3 0 /Length 3956 >>
stream
xÚ“y<ÔkûÇmIv*r/ÙØkÉ6ö}_
ƒ/†1ØaƲ¦,¡ì$K(ûšd SÙ²„8"Kˆ’ådgê<çœ~çù÷÷šæ}]×ý¹?ßëºn!~sI5WŒ3¨…Aã$¡RP%@ÃPÝz €JIÓ i`A‰Ak"p UT„jxwà’4 •S’‘U‚ÉÒ "éîD5Ä~ÉjÞ é‚@†œèMÖpA sŒ Ä¥ 5
0ûqÂ0ý@¬?è*E…®H à º#Ñô ŽtÑn@þÏ°+Þ篔?ˆõ#›DÉ&Å ²EW E\A7zˆ†|HvòÿaêßâZxÊáýCþg—þ'ðF¢ˆÿÀxûàq 0ĸ‚Xô¿KÁ?Í©cPÿs.Bº¨¡ÝQ ýg駅$€®&Hœ‹à†@ù?ã ÚõßÈmûi b«¦a®kpñ¿óü™4A Ñ8 ¢Ïß²?ª2ô &w‹$ öÒRÒÒPr!ù÷׿ëÿº ŽvÁ¸"Ñä…€É,A¤'o™`@@¢]A ÈŽ!Rh Ž| ·$pÃ`éŒSN€À„~’¼ Ñý›Èó‚ þ!r¥Ëß•– ®¿ €€¿ € A ‚úÉRÞÿ ”,…þÉR˜¿Q–\‹Aÿ"Mn$Äç” Ø_P€àþFY€ù%Mþ$ü?H~DÂOüß ª«cA’0@ò J>)£ÈäCþO¡ ‹Ñ¸Ÿƒ¼±’¼5 H ]è'Æ0.Ê‘ži
Ñ¥7àƒe'Ä)ÕÝ“ŒjÛGž3D¼»K‰*îÑ÷Ÿ®±ÙœyšyñÄ"_À!·_\K°i¿Ö×pߤôÑ£E§ÅÌÀ§Ü6›™†Ä>;«,ÏÞþñFi<<Ý[šj[سNZÍ7Õ´X:9ÅOÑaï_ב)'o£•‰²Œ‰ª–á7cM÷-•y ÀššLóþMÄ
Ϻ¦‹¿o`ZðìûtRŽ ©]ûœvô×î™æíö“úvثî—ÄÍã fØ&” :ÛOñeCíQ›w²ÞŠlIà®eT´úLV¼Í-´Û3¦ã’-Ó¾¥'Ï„Œk~^ÈoIQT) ú¦ñýÃ';£íó}c< úÚÇõ±Ê_>Ô0'4·éŒ ÂçV™+E³Äõdô£&Ç¢àßËëB3Îû%U3/{jUƺµ³
˜Ž~Œ–Ô®÷¹÷Þ„zD7½ÜÂ$_M+ÖõËóN§àlq^íµeðV#ò; ÝÑÓÞ=5~Ã`3H.|»aë‰n\Ž{oŸq|t³IÀä¨Ä)a¤˜Ê]<9¼±FK¡AbßÆ‚Yd ‡_¢õ¯zÏ¿f
³wNÈž b3sPŽqaU]žM¿&5@3Òó ánnéx^vÉaËu?åÆËa[ »îú:Ûôj.3J»2N¬y»£Gå%<áuòß“Âa þî¸ÆGz¿1 oÃ[‚ÓXênJ¼¿x9¦ X¶|’íš-ò싳a´$‰ÃéT(/’¬¬›,ƒöÏjHž2AE'?åÕ}±2YaÌ24è¼Ìv3«Ûsb—ʃ•‚1òÉ
m¸Þ).ø5Oâ¥:s~|\缊)ŒqÚ S˜×i‚à^
J‡SLÕj xôäf’qùÝ {Z m’íÒ5~y‡¯â¨k>Æ+$)Mðù(ÄP?cáÊë3–ŒSH/·¶ÄŒÝkZhí¬ ‹N&o^PŸRªTHyU÷(§òälÇbiì£;(è‚‚^µ¸¡ŸÒ…õ$æ-å/k;ûÚ‡ö~šáX{EzC7Ì
]pJbó\0¹?§øi Vµît÷öĘ\”š3r›9ÆûÃ'»Æ| _yÕ3UÆà¥íg’žš8+Ïz>itŠ†Ý*œ!‰"`êÚÊæõÞç63t*\%A ªw{>˜0<åj؃(é á‘r¥‰:!E¹¨ò™ÂaHñ.Ƥq¨Äk|póyöTy5e,ÎÔáÊ£%ÿªAˆÂªqÚ ôùâÌ6,ù·ÇL>
©
Ú pWDÐ~šeDûÊÀ°3vˆàÌ70¨¶ßð ³1‡´^L1M½$¿¨ŸRq÷È ÜwØFà4#Sün+, éÎÙÖ&±JçXߣÂ×42Ýp]˦3gyÿæV¿åô8M³ᴄ½‰e’e\mÏãüdt)4{y3+hµ"’6<Ógãތę–ï,V€ÎcJ5$Æ>r#ÍÖ÷r y…U{bÌÔ(èóÃå©Éµâî·IŠ–„«"eçi(LÚZ è 8(²«^PGÊs%>8?SÐÀrIz¶´¸ÇWU|¤M@t3IÊO¨Ì ¤½æÌò™©Ð¢»ò
;›é~Ž
xaj5› ×S–œL3ιsTí; ñ.…úþ2>¥CpÐlŠØñá2wÀ$Wg¼É ?i<…££¼ëÌtõC‰p²t.¾åë°Šs/åíkû …Sy\ ¿ßÇû`æMh>é¹[ÈNÏ•%Ï,1´#ç!ãÎ/s¹ÔôŸD¥àáª7™Pþk÷j6
T4'SðêÚîu«•ibòK~5:’NÐ*+´Š oì\ÞÎ@1™$Ÿcª9ËüíøÄ|ØÖòñl$¶% À&óºü7kKbl“¯Á ïÎ5๼Ò('ËË·Wâè>Fe×
uÂg§ìOȨž@ºy™•¥×ð\¯ï0¶Ò“¹Õ¾Îˆ³¾nQuMW/l”ŠJÊÏA²ÝWw<’!€Ø¤›b*Ô²ƒ›
R:uÀ7£Nøh†S
³zAOŠhL Öy•îѼè«Z]Šª
o\Ú
-»—UhÓk™¾zMD0Æ,+ÅŠê1¥@¥à£º÷aø¶.&R£Ó5?›[²~Ý$SJ§6af‡ó¨æýéÑ ¹XÔs ,7«ÅkN¿]P>ª¬Búê9Z©êEœ$MÉrQXÔ.L‡8è°‡ƒ9}¬—Î)q¾GʉªÕ ,Í^GèÛ³g/T™ `–âE}ET‰l‹/=»À¾qΗ xtÄ8¦qW©²;|á[ÎuÃáÁ40—á]hÂôד3*aÔ‹TÔ¼u‡e„úÎvϪ{!Nü‹·nhåQ®‹Í‡…õ 98ÌKY‰ÏºÜåXÑS1Þ™U% ¡Cjñ•ÛV+êëéŠËš>’ºÛ¹‰A0ô•p'Ñ,GP²OYÍ Ò«Ê4šæ²¨(7¬ÍŒRHnÿ\þ´ÿð?>úØðYt *¢SžDgêؾ< –¿>²Y—µç1r¦—ÞnjÈMC±süŠm¶&ß`ncð çv4çR©ŸÃÜ£–K…Árhç½ýí`÷šôÀjÒ1âðâd>Òíƒpe°t*Ásî0w ^Åvlw«íËa·Èîc¥$‰9Ú“b3Ç:G‚êj²…òÞu ݱÕg[åïº?“|)TÁÙ$ö}°gSÈnX4©æüðæ
§r z#ÊyOãäDÿÛ‹ŸÇij'3á:²¡-Zõ¢m˜$ÃÍ2# ›¶çkÛ ,
…JzΗpWþ±6~ÙAxqAq˜øúóUÀ7ÉLPG”"k÷xØ:‰ 9a-<§Ü¿’M¸mÊÜ
¶ã!¹£séjFN„…ÿÖ&ÆÕ6ÖßZ-ßüý©ÞùoÇO¶º9—ü=eŠ§{»ÙÖ¢¶\'¦d kUó¥ºÜµ9!h*Î~-›çSŒM‰E6õLðN1ßZ·ÿ,ÚÜàãþ¢2<32ŒÆtðû®wñkNyþ@éºkéœMÅo2h´õZRK·U5¸uÏ–I0¿£âØȳaO+ÍèIü"ü4øÖMÝó,ûwïŒyÐØÃ¥x ‰ß|o•úЫVî©6oíI;jX£ös&¹ýç*fwL–µé·óšZ‹
E¦ËL#-UîN0μ˜¬ W
¼äÒ[EŸ:Áü>L±“Õh@ŸÀ“+å ü¸Ý”TËQ>¥ÇSYp÷×Ã^d>ñŽ
ŠPyöµ0¢x‚Õø
§„SRŒØ5b–æ#c}7 ÞGÉzhxtžû¸³ÓÀ¥ŒÆ34ÀÏ2*R\ÔìÎdIËýeM°þâ
£5½ÖÍk½qk„ƒÄcR"ƒ«.¡°pO€[âëòƘÿ¤Îí7UÞ”/óë~L°¥êípœöhv+º~Øw{iÕ«iµ0¦†VšÿBZm>Qsvî÷ØÔ;ý¯>׆ÐVW¦&@Î3Ba©‹=åRÑf¶†ÁãQíÓ¦Uô¿çtÖáÆ£WÚôš‚ﵚ«fŠ,ÅSË|?H÷®iÂ5ܾvsWI ¢Ó¦6 žŽ¾:Tv™¡^øÝ}tF…€ÀyNMvq0jnûÓøu³Œþ`hh]2–&[!*þ(ÀÁËh#Œd ¿ žXÀFZ$ç\‰ÆÏßG}¤yëôÀ´öJ™oVì%ÏÌ‚<Öî„ %Çÿ 1šôendstream
endobj
38 0 obj
<< /Filter /FlateDecode /Length1 845 /Length2 2504 /Length3 0 /Length 3086 >>
stream
xÚRy<”k¦Ær¦ã (ËÄ+£BÌb›4&[–QÙ“cæ 3ÌÖš±•J ¥’%ûv"…âd)kY²&ÛPIZ$…ˆˆø&ç;ß9_ß¿ßïùç¹®ûzîçú]÷¥±ßE@aû6lO¥Â–D¢
ˆîH$TKË’ ’xt6ËŠÄq
‹5ìƒ Ú @šàŒ pF&P-À’Ípéþ4 °ÓRû»È 0A.LbD2E=È$àÂ&ÓAž@ 0€ó÷A€3rC@Š>…(t2ðýé,(â»';•
˜üIS‚9•B@nÈ°sͦ6 2Ia³€R¡'¶è7PäåÿaëÇæ6Á †‰ù½ýZRÿS'1é Á¿l&'˜r"›rY?J=À?ÍA
=˜ùcÕŽGbÐÉ–?ôP†úHÃ?yz
RöÓyd@%1‚À5 dQ~t"ÊoÍÂÒÓÓÓfŸî¿G»VÜO¢³x® ÿV¯aÔßX—Î!õ‘H”H(:Ýÿð™5‹Ì¦ÐYþ ÚÈ q¹$T´D"d„¡ :‹ò/rŒÐg±y¢'€(™€ÊæB¿Ïƒžß©5$Z‰Á¡‘þà E ƒÄô£þ!BÖ? @pþ†hQ•ÿ(j XƒÿÒž=l~˜žÐC‰L#
1€‰2â¿„ä`.dñÖQõ_˜JMù :ØÏ&›ž
H¼{ºà˜uÎãÚA°äë§Úk¢Ë’Mõ'†M7örË»\Aµùhù z¡\ÂG‡[
ÕhDù.!¼9Ì-”o–QÐκæ"5Ežðþu]KFSƒ,3-flPý5§\wjHsîEHD±ã8Ê$[ùã9HÁ@ÿ·ÙcÇ;F7$Q4ß$@gÜs¼
º¿{×öõƒ-;‡=˜×²ÅûÚܦ•Aå†çOŸÉ$cÏg×÷%›/ÿ„ÔæS_v@Q)ꃞéX{œ4̸×=3S1×HöÜ]Yá@œ”~Õ}¥‘üó» {=ìÜæUodm$¨2Ìmó”ÅÁ’¯ZænVý™×%Nl ,yŒ6Gqʯj9û.~ÓšÇB> ^Zþ²z^¹vŠ ¦!¦T´ÎC“‚Ù©;¡ÝŽT–è9Û¬qD»2Æ#à÷o°8ßÄìÄ¿ ¯\ žØ¾H…þü³šƒhè8n‘™˜¹‹è¥{§¢ù²·5_ {ð!Ï;õ·–MÖ@kÚ¯wî\})üíp$Çê¤!2ôŒs@LúÒá·Cl½E#߬Ùëï×½ÏH‹sx¿¡ k‹+’/BHÎm
Y¥-V<ÐÍõ' Cññfû#oE\”ytÅÆívbé =uÿlúêÐ^-)ù»wÞÆç7 >¼höÓð¶)ÃT•[¥Oo¿øõ#ð)d–¥ÿˆ ”ÖIÖy*7g4Úþ8ί2ÖÉóÁ_ Lx™4¯jQzn+ÄùzwáÔTT#"ç̲S=’ ßöôuGC ™Ý2TÙöÜÌgÛN§˜gÇ Ï»ÇÊ™ýÒ×íngûÜ¿ž/ÍX~ÿ~«ý³lŒ¢xAÖ ÀÞ6|ynaÂ×ðDNÖÂa€³{‡uu² «¸›=ë½µJn k%yÐh´²×¨-ezi€ñgj@$äãpw›Á¯é‚v9‰Ëe» ¡S¾ˆå·•Ñz͸Î
ELFg@ñ…Ù•:Ó}’ Bð]ç‘2ÙÜ°*?+{ʃƒ¾A9Tƒvû
Ä2ÝåÌ Hg xnoË`ûv먡׼$){W•X½’^6Ï·ÔL®)µ×T4_Ã.ØmÒ}»}ìAªÈdÅǽ19&Ñ]zÊrc•¼ãRòdZ¿ ã¦Þ]sîV¿©éå ê剨-lnŽ#V*´¶ÑvöoØÅgQ[ 7æFh4Øøª4àˆ¦ò19×&CJPÕ|…:Oâ@|¤¦üç\É£¤þìö ê)×÷qÇ?ĤOÏKf–Qáu¹©\ôq/{öýrÝVékòOÝ
3-n-Æ%
ïíy¸óúE ]ÓÑ>R\Ñc˜|5ãÌh”䓽ë.˜üÖ®@³MPŸüzÅúâÂêÑw Ÿ¶¥eÆïpÚ}Ø:¶ÿÔ×’È™ÏÔ(?…ÏtÓ-ž¦‹Ob ÒnñUmK¹þVM-ª%›MõÞÕVW‹¨×8öÛÉěµ…n© ½
Â~*eßuhìm5Ü~;Ñ`rG^ª¤Ìfx|Men×Ý°¶ ›Çõ¥O¥¶Â¾ñ´hrŠ|½ÙuÝå .Ñ;sV!¹§„gZ÷ÑÉ]Ë>å¯(TÎèu-Ë)½˜Á뚘2ÃÏ•J†¼¤…á‡ðìbeGªJ’/$ÂM¥cÚ~}h¿ú±Ê¤ fxþ¼á²GŠá±¾à6¸ÔÆOŸgjc ^ÂxÚžß<à)?e:U_ÜÛj°·ú•ëÃêÂ/e³báOñ’ÞëtÛ°ÛWͦÉGÒ»Ô«¾æZ8„±úë=ñþyPŸðéä?ó‰²ÒX7IÃC~ßÖÕÿ´c¸¬Ãr!Ç•ó)º[Ü_MÓëAjÙ–Ã4%ïZçÌî±]ûbŽä®·äoz*þ{+ívn¹÷èi³³6:×3Ot¬<~1~k^Þ±ÙrPÞ5èÝÚK Èø]gåy $þ!9yñvÊ&ŽLÎñ§åt AÓóұد1¯žºÇèšÙªG)¡ƒÔÊ*ÞÈ©M]
?]ÀÌ[º¡4;áWyÌcè¬c¸lÖ0njûþYvÅÀ(R¬iåAr$?‚ 9P (Ê¦Ì {mœž1;›7Íž~r /8hº+[µCÒ 77Þ9n}eìÎùf×ÅŠ‡TJ³“ K¢ÚP?v²Óýç&ÂæLŽ¨ä[?ëiVÖëöšzýÞÐf¾òê(®¶/Ô0ÅØe4uúæþ¥G)bN˜¾ð‚âÖoÓ*¾³€(å³ug£!
Þž]|Ÿ*fTiå>ƒTQN‹ß0Ò¨ü)q]1BƒXé25rÃùDKÖi7˜>‚óq'¾%‡¾
aŽQm•‡©FAâa—pT Fhí÷ØòåDOßÕ£óLìÁ3À/ÏKSÅsnJˇ—×eÚ(\¿ %%qÁÏZo2iáN‚}â)NEÖv¹vœBðlqcÁÑXïÖ{~ÅMãph>Sôò5«kßüsžV)k۴ĉWéjÆj– ×´Òêv#ýPm«ÇN¿}Ó“e€= ìé.Av V›]'Ž½T)SØ$Ü çCî‘é«VRiEó¸VV>eÕ–Y±1¢^Mö^^ wÇU~}ÕGû>î™ ø¶Œá´6Ÿµï¹ìåÓ®™£ôy&·ƒ–ŸÛ•_û…–f :ÐT‚õ{}_s8?£*O©ÍŒ°S:Z«1 )„·àlNLÜ‹ÑxÒ»Êß ,~´»ËoIfBÅ,R¹>øÉ8¶ºÈ§üáPï ½:Íã$Èׯ“oÑÞ5æ—Ë=Ù7€ÑH© µjQX5÷ |ð1ˆÅ¼¸ÌÕg,XS®Ë™öB¥~çÍßendstream
endobj
39 0 obj
<< /Filter /FlateDecode /Length1 1555 /Length2 10089 /Length3 0 /Length 10986 >>
stream
xÚ—uP\ÍÚíqwwwwîîÁA‚ îîî®Á=¸»C‚Kp $à.Áî¼çÜï$ß¹ÿÞš©©ýë~žÕk¯î½«††BM“EÜd”9º±p°r$•58Ø ¬ììH44’.@37[£”™PÀ! À šƒ/À_A nA N$€$ÈÉÛÅÖÚÆ
@/ÉðO @ÜèbkaæP6s³:€5,Ììš [ ›7+@ÜÞ ñO‡+@è
tñ Z²"qp ,m-Ü æ@k[G$¶,É;Z |ÿ ¶twúŸ) ‹+Ø€þ_6 `“– G{o€%Ð
‰M^
öòÿÃ֋˸ÛÛ«˜9ü#ÿOPÿÏ´™ƒ½÷ÿ- 98¹»] Ê K ‹ã—ê ÿíMhiëîðß³ònfö¶âŽÖö@ û¿‡l]el½€–j¶n6 +3{Wà¿ÆŽ–ÿmœÜ¿,°½—•T—`ú÷žþkNÍÌÖÑMËÛé?ªÿÿ‹9þ08 [/€;8^p!øó?WFÿµ–´£ÈÒÖÑÀÉà 0sq1óFŸ 0ñ |9 ¶Ž–@/ Ð l˜Õän€3ñX\þÙP^^ ›Ä?Cÿ&> ›äˆ À&ó‡¸ lò\©ü‡ lªÿ!~n ›Ö ¯ðþûtþCàÁfö‡ølæ¬iñ âkZ€ þTs°ƒíYþ… 6à Û¿÷÷?œ` ƒ¥™«ÍŸûý§ÇÙ |ÿÈpجþ ¸Àê/äþmÿÒó äùAî.©C³þ ÁýVçæ“·“
Ðñ¯
ðØ_úìà€ìþBpB{Gäð—Wp ¤xÀŽ¶ŽÀ¿æÁ7ú³:¸ô¿¦ÁîþLƒÅœÌ\€Žö@«?!rsüßQ—ÿ•-78m'ð“
ú³ \`ygwøñûß»ðO„ľ_×?:`Ï®öïXúO;x}7à_
öéæ ú«¬áþ‚#óø Á¶<ÿÚ?p·×_–÷þ Á‰øü1Vòºü{©ÿ÷— yù²p‚ ‚Øÿ9ž ^ÿÿUiáîÏí_ïSð{ãØÊü–½€Hk_ABa 2Û"ª¤Kæªa!%¬Û“Tš{‘CW“!íË?+:3n7éþ®ÉÁF;€= ÷|&véñSŸ–9vNÊZ~=ð0=Èñé$Ö½ÊQöþáDy zŒÞݶts©º°ý¥*M¯t4ÿbì¸H^Jë~“bÐÀ£e07Œ—OW&Ç^;2¼‰–‹B#˹*Š;rË#-fk64àC$BÓÆ%¨Ç ëw ÂÇÔWÃ{ÜDL‹ r.=F CÑ“ÊÒH ôÆ·‘‰z±\€µ¦ à v Aã³|hvD}¾‘´ÉüþŒt”²Ç9sg.Å
¯Ñvùw.&›R…Õ2ö‹”Pm<Ý#‚Ïë ãA ¸G15z7èÜ=Õ(þ½!ƒ>,jç*|ýÑœÜ9 ´› ·V°´sžýŠ5J_±ÚÌÊ}=ƒ=–!Ô ePJQ–@ø‡9'¢Ü1`ŸKº]`SöûµÜ+þ ´6ê>g~éL~å*÷š4T¯†à™A´ÜŸ./M|å8ÕVaË2GÒšº‚1á$bR‘¸SüÜô «âU ÇAòÓ:FË{Ãð¨æÈ
‰À±÷ .ˆ]ô®•ÐòPžúdªƒ¬|“±nŽ‚<>^~ɨònj*ǢʿŽ¾ÿ ,«!ÅÚ‘©Ð ’٤鰃¿îª×4<— ÛxòGÇ— \ö¹½'H<×ö ]×gW0€6ݨb‰Í7˜È]¯[:–WGmšÑ?á˜$£ G
2ú1§ZÞËLÝ•çH’¿fqÙÌ
‘›ýÈšxIøÝ©õú(@œàýó/Nž†¸!
‹
œ¸Ë Ç1üs^ßPð~‰ë/âCâ1mù(¤±Uß‹¯(R5™7Üû*÷,JÆ»; Aq'™³Ï,åUÂ…»žW)åþé+±Ç’Ÿ¨)³.à<zÚï•1
ćBQÃYg»X#r ”þ³C³¸Yç¦d”´?ø3Þ¿šö¸’¹÷ñïÿÒaŽ#\ÃÇdð8¯× ÆÃÓ«T¢øNÅ€þàûc°×âÉV–Å‹XèenM~zöék–Iç͹Á° Mtn¨ïgñé rVà†zc€2S¢lÃDè‘á"×Ö=Öý·ìŽH]§" žn]LÔŒKH´8®$ûGŸüUa¹ÜŽ’TæàSîÁ˜ ©':ë@hΉóy:?b{@îDV(¿d‰˜©n²ÌaÁ^YÓ)Àœøc‹mÅÁ+ħ1"cþZ•Õ$Tü5ËÅc I†ŽõfMèIJIÖ…tB‡OnèÎz’qO$o_ª ÙI=Ú¥Q²·KÐrŽCÌÙ1q6`Ô]ô{–Ñ S‚/?›†Ø¿ZÜ9« œÖôœ§“é3C,èk3:9µ‰{?¹X~ [Óî±Gø
©d!´²PWÚÖRà®`nsi!2æ§ ztKÇÏ 2ô–¦3´Ê?yi}’†> ))s a?ýˆà”M\ŽÉ”´(RðSÎÞÌokx°ƒG•X4áŽñ™ÿ¾JÒ¸;Dúb<ƒÐßÝ*†šGo†T¹ë¶¡¸<‹`iKß«mhåo:zýò;ÈåØCÖ
umÆR¬ƒ‹o¾äюߢŠrßë)0
k€]3ÅnÉÏ@Š$³ >©¸S·4µî{Á-PõÝω÷=‹p\Ccîtq_’FŽRÝÛgéRš¸KtíÓb¤°üª 1¨¾œ™Š(¸ÓßPî¼¹Ùü”&œç âz]š ‚¡¡ˆá
VQû¨Èºˆò˹^ò °w׶bÅfAK£ó:·«ŸÄ¹Ç|}7aÀ\ f ¼vØ8
§ñh_N¯[®BK]%àþ¡îÒ&Pw©Ú|vOþDi&åßÑ“Ê…¯&CÃœ”ú0Nö.Q›ûvÏ@?,2B ßÛ”é}uqDYzˆÉkIŽýr7 Ú2V™¸kŠ{KJˆ£:IJ<’Iâ7W¨ÁWË‘ŠWü„¬”WëÈ‚Ð |{²(Ç•šÅ7•Z˜r²ÞåAc*’ØÜÆ.߸„«'vöâ¾”dc³ß¢:4ªÝRÂyg/Æ]bhèKŽ—HS-(©QI¬LÒ¥½ÙÝ}Ü…7¹}Iˆ%>!”J|ÈwÞ@tz9bf™ï7¥#cLcº¾%Eß@&{Jë€ÃpmÝùl ¹‰ÌÚHãw¬Eç)¨4Ãþ̼ÃÅ¢jýÝ5Íðéo±ØKI…˜˜øH<Šíq¸ÈEz9Õ4êòÎÏÑÖË×Õž ø”˜Áxˆõ²> Xfd£MƒoTAcsWZŸt[™ž*ÙäïØ ÖÁâ[zþÀ cXY×à-`óî¢s¦DÀûLS`àX,€CÑPmb%é`øí)¢vƒP¡kþÝ2t{î}QNIÕšÚaÿ”E’to&[ õĘ3õOä¼ k1 #=ô™&ŽÄÚ¬AUA™¯up3¹tûž9yS‰â$Y€`ëžúíjj"nJÛ±X™ ./SV‹ë"¥ââÍù} ÁŒ®71‘Â>-î ¼½ÁƧ CŠÖ 2Áz†kõÀÖž/D P¨TS(íÎcz«îÝ\¼×$Cá‡j7¡íƒUˆ8µ‹OM#%u½!¢²‚8wã²ln¿&Ê”î)´«‘†}¾áà•#߈ø÷Yö'LxLˆ®Ø‡¤{‡¤ã-¾ ÄoH¢›¿ŒfíÔ
nò€EIØWÿfoè.šœa¶ß–P‘¨ˆçEKÖìÆÄ°4tôbD»pýr°3–ù$ºN%ð&¯ ÈTq% Æ‹GjˆsI‘sŒœ ážþ Ë zø¸à¨Ôˆ y ÉÒq>Y‰¥Š;B¶>6 å.AÁüŒÃÒöük’ú~ÏH(Ras…õ Ò‹< ºÐ=úho`¥zÄ\oylUs†1\WûãÑ%I27n¸R«§6´ZûÃ̇Èþ¸Y—¹R#—×'¸ŽúÜ£¯Me?½+„‘£!Ãê07º=‰b¾'\ͼ·^VÐ!úG«2;çVf3Îûª ®fÅÑÀ ¸÷
ÇMZ&u–Ar~Óæi´œuÞÙ qK\_Ò !oõ-qmbÑeOù ÿœÊÎÌ ŽÅóTüu6úhº‰H-…>±Œé.Ì ›»ª“u\ã¦_U2’AhOøÁösSÍFþ Sze1ùÓ" 5’†8”l¹
GàÀ>:Wõi/w›V1\L¡LS¦ËéØ{çŠ;2z†Ú&c¤ÖI†¸ìû¬âÓÓ†D_1 3ö‹ Eè4tβü™[þÔØdùGÀ–¬fpèhŠ”\s£õ75a'ÿ~9>W<ççb¯.•ƒ&¡öÁÔ0B/·¯™gd½ Z
(à
–'~|Š
há¤ÈM+ÔüH+ñ*'›pß‘Ç,û)36¨€œ×e¥‹¨Ü>1y»\ú(WÃ,Ùtã«Ûò¥eNñûÌWRT,;öô>Ë)˜=]ÞØÐêHõEñ£Ærùp}Ç:³ùBÂ:ìîqA $óAŽÌî@~ÑbÕðú¥ñ„wºK¼Õ }¥¸l4/Ø[É8|[¥Ÿ tñWWÅ?é§_ÈZã¸3ö»o ø(ñ èKEg×úwSE+jìy ~ F 5à…%ÆÔïUß
G7€eµdTC÷m‰‚÷Ôï¸g ’FDÄ&ÂZ…-Lœb›a_™/^W+I2´þ\ÐÜ………šUȇ :3Õîfí‰Am“‡³ê|˜ÃM»xËö½ðB;‘Ä þí ÕÅ»¨SßÆ{.þ<|½‚Å^x…2òËVéÏÏÂÝo¡˜¢> uXç”ë¢*€’b¾ÊêÏ0ôI¼º?uüÃ!J]Ü€{qïfjÜêÙÙ«vÛoŸ\|á0û›2w×¥D¬Ü&cjeeuÎRÌøò- WÑ®ÊLD{ÛŠÉ—Þ|r'å_œzý•Õ§É Z{±ˆÃ1U÷b#¼²E´mÖ¦ÉÄ Ðéü
µ:_ŸÞ–ÄÛÄ==Ö, Vˆ ;O¨¦ ¨÷ÓœC»O~wIòEêKX~Y•¾AÛ/ WSx îÌLGé$m
Kr ']iCÀ$q è*þ|öéõ¥Êd&¯”\ÿ"{ŠJÎÕÏô¬°™Gî*INDÁuJùíÁ÷ħB«)Gh¢@vÃS1H«$Mu{YoZ³Í«fr¯ê*4Co`}[=ï'‰Â—>È;;8|&IŠÅ?ž€,1ÁïÔP }‘k©ºA„7ùJµzÖìU¤c[!ÒÓÚúhªm·¾tût Ãæ9èAjàeæh]vç+ xŽµ†&ÖáDj¹ þ¿r-p¯Agôˆd%~ų¢&z©ðhŒ~™0G }ØH ÉLªtíçh p í^~rÉòµ{ àlÖ‰5çLŠ>Â.EÍ©Ù"›ê ôdÄÉJX"'î"½ òie[.~[¯õXÇ*žã»€~¾3±#΋C·Z¿a¯)mÿjo„ þVÚã VC¨ ÈÆ}Y[—ÿ‘þ&Ú†è³Jex½OÓR '?˜"¦?øÂøËýüî]µ@ùÖƹ-Çà ÉœåÞV2§ òZ ÖMˆ8V1 ©˜û/¾\ÖO=6Z—çÔÎ,I&ëÞCmòsy€ Èê€MŠ¬wyÇÕ$,yïþÛñ™N_V9¸†M:£¯ž(å¨ñœÐ–\£
wá ™*^±bÔWùV™Ñ“»6iäGýHº3Ü„QvˆVïá4¢†·c_¡e‰ÂÏÌ×·žÁOìXÞBaËD(ª…·ÜÞ·†,ê;ΦÁ³›RVá§Wø%ÔB¬¥’~k £ÛŸøš·ÈÒ&Úpçc%Ðd× véö}deFuÖÌÑFÑ V8«gçá°ˆtuüé‰Ò h¯æþüÍ ÌYåªJâ;—½÷ËÛæä,^V1Äî„Õ³›j‰ì!ÉôZLÅ_KñâÇX•,<öR7"¤ bB±¤ëÕ|Å™óØ>e9瘈"¹úá§â…ôú3w„²Òå 7ÓÊuÿ[¯Œ$y U»’áâ3Õ=dQbåMHÆ›Gk‡™Y¬)˜ E·QÕHòD{ú¬3Á ;óY1s!šT Ú·Îæà*7¢ƒ©âr?ùŽÔs«œ’"n3A‰±žQøa kž¹ í# ÎRÜí»ÔB{ºˆÉý)c&™Ò¾‰Jª •"˜’Såé‘œ#êiØr; 8ÊÕëk M_m"n‡ö”b?‘#Ò~Sį—ë-ôè/¯Z ¶,¿ˆa(ºÈ|KwZc
“ÍÉ‹G7Š®"FlQ_ îT vòšš™fb‘^¡vߪ^‰ä *#ܱ³@ªPYjŸ—ñâ¼mŸèc' ,Eï™[}~W„
c¡š_Z6«Zš",;1æ<›ü¸PPÕCv×P ¨Â*1ÂúVÑï
ÁçËoFÀO†òšMSu=»ûÓô´žê³á»¨,DÓ†ÝÑB3¨Í\^o# ²ÿžÅŽZ¬¦‰â¬ÝjÉ u‹ôž’Îvêðâ~:€š^²®Îd ã³’Ï¥õ´µ$³jžé;‹B²¡J
ìUø²q=×½Àx7äÐö³VMu´¢ÇW1yû·Xîj,Èg’e
æ©êªˆÀíÅWr1gGˆàl]ru#›”–VÚÞ)¬×ÏÝ7ˆ¿3“]™ÒÐe×ú†~éx@s;_ø¡ãL[ûœZzNx†í2 NÚu¡o’R+^>²ÖmÜÇ4ð§Mdýf=þªë„ðÉU4Å”Òâ¤]TÖ“ß O÷îK "©ý¾zØ¥~`~®D_aðP™ãE¨:¼þ>wg·:³…4»_ÝzVUÉdÓ…Nv—1ØQŠçfuvwªöñ;2Ÿ‰‚ ‹‘Ñ÷dïVjÎŽ i§ðCFÆ €x$ÚW ÔQäk‡³Þ[rÖø0Ÿô\;Ïø¿ÔX
ňÈt +Û Ÿ%ÂiÏä—«QoÄ)h§Þµ0P´«ûzíYøÂÍ>HYºy»ð]Kœtæ__Ç£,ó1™º½¼‡ÇýDßPR˧» -ärzÙO‡Ý
„’§õôïb_ÚÈ: ÎimÚë«eìúíCq
KÔ¨¡ñ^‰œÌ†u³þéM-]•8Ì—§K N$§Ãükˆ–¼y“|Ì
]³Æãõ+ªË²ƒ>ˆÙ¿N˜Ú ì›9=u Apgù´ø”Å)úãï˜àÃÃû½8ÕºõNñ–aMÔôxìT H§³ÈX‰=å‡Ð0ÌÛVLœ$aŽòá1.H!TŒ å
Ã!Öñ¡BLN3ÅDe4öÇq$yY†x÷ü0ZŒ>©uéëEÜRq«uÂÚWD¨Ðœ‡òÁ—+⋸TS±õ‹šýÇPï Œ-ÕuúèÃ
`· ¤/‘3·úë±ÉðªölךÀ ¡:»*ÒÌ…ë¬c¸µ´XêZÞ¦ ]}Ac ãµ âGxÄÏ͆Y/ï½Ó‰¢&‰8ˆš í[x,%%ú÷æC«M a² vN ÷™ ]£º>?,\pøÉbœÈq;/èp07³>X‚²»%§Ô•©€MdJàþôÖ‚ó ‚P,‘’À~Ivòþh^q–ßEï²è}ð½—®nª¼j4Ö÷qD¼å-¢þÝŸ®ÐºÑ6-̼
š¹Â֔Ȯú‘ Â>èÞ'ú«HÚã}mr„+¼•\ù)wOºÑ·o Nñ¸#Hlül®‡+
0e©ÂŽmA
C§Ðe!œ¿ß:czö< =átm]³:¬(„
a…6—)l¢ ë-šútp¬?’p1LJÖ@;Ä´72 v ÛdTƒÚ*ÑùSÉüH³jø÷"ïÌcm¤mi¡vWëR³ýÕóÂ6ý``/j÷;ìë3ņ ±ª î^[« é³Ì#Q¹+†ø´SzüÆ‚"Ò%”7áÚÍt³Þ°§Ç?6÷Ú6šÐÎÜE$„«Wñ÷³™r u´!ø3ÍGÉàƒÒ²3½C§µÂ,óäÐ!¨ïlmŒ¬¿t‡G¤‘1("gTV+*¢Y¡ˆ
—Õã¼j5- 4Ù¨ñJ %šþBö©l—ß0«—«%ãˆg/É’qDÍÄÖS‰Ù·È!äVÏ2j4Þ>ÄEØb?eoâ°ßFÄDÚŸDÂïÚŽ :yi:ø¡w¦Õg7Ël'‡>-·‡ ì±¾'§ËíŽftêubç ¤W‰H":ù{lå]î8O¾M(›Çœö¾: Zð– 5¾W—åÿùHõ£YxzrñÄ Gÿ…ÃoÌ¿ ¦È¿\vcr7A›Êk[“€¦#4î²´±Y§{‚X¯›¼Ÿýg/¥\_O»Ç ’+ôà\ÈÇׯèë•ÏF7DsúÛ¶˜"µÏÑ1á<¹÷ž@sø_Þã”-ê …3ól?M\ÃeÂr0–0Þ<ê±
A¥ò…¯sÌ…Vïney½a2s:¯…|êÑåëyah™y\¦JX"׃ÿ|b™ †ÛxŽ,Ÿr»Ýµ¹ýhÖW¨M縧sÜd—œVëH€£¿ýâœUõ{ƒKÂÿóÎVé”e;¶ÿÕ
¨é=h8Ð$\ýØ}WAbá)äh7¦¢˜¦šàÀÚ'0’T0qáâ5fÂö"–I4%…ÿRûD _LŽõ\Ÿ[ e^ð¬ÇÆ°55WFëV á—}‡ùû¶³J©ØÊú(ñBÏ4 U“ÑÈRlpOqÝsMé°Õá¸éW¹ë÷ôŸr6
òLÎ
_¾f¥g§;`SßÆ%<ØnÒÒb’@!—ûÜŒ’Ôúõ0¶+þ?™Ì‰:à
‰}è~δöqiÖ6ˆ bvÖJ© ÖBk¸qÉìPz`p(€“ì÷!ŒSp3ÝwÅßX7Ì(Ÿ})Ne0ÜÊ ƒWqCOcV%w¾WI3Ô ,cI»¦úi¶·[Jë|-O…Àæc÷ˆê6ãG+™ ßÛâ”ä)”ó#—´ d‚ÀÆ:D%&Ï΢
˪f^Ü*îË$Ü·PÔ…@
î½ý†W¹B†ÿ4YTªäUM¦óÉ¥íai Çâh¿%5GPㆉH^Íõ^3- r2½2<Ù\p<Žâ±Öp¥TŠ¹hz‚Ýh— Û|'þ~ ¯P0/n‡-¬;ÄèîXn›0T®¯ÑMu¨Vo^7 4N²æx4y>ó$“ 97j9̨–\ýAyi
ðµ[Õ4#Ž2U>IP*§ãvc§ÛŸ ‘ƒÿîºæµ6Ÿ4ü}9AÍ™ã1µ7gZçxòr'C†¸žk©J×e öÙô¤l\¥&£Â‚N#öl@rsgO;þ¾ãäêô6i“ýµÑ]…M 3c+Ò Åò/—ÖŠøƒwÁÍõEvg²µä¦†W´S] #wfÓ)b‹—ŸÑº›ßbnüJzóO%;µèÉ6dÔµà}(N s2
µ*ŸÀJˆ>8Œ«°r7,]ò×—@ÂpNúV]í®²*~i€Zv¨°À¦ mGoñ渰Eö› ·ó(†×ä˜øر®`ñ"€&Ìv"Œ$ă¾‹¼»9hÑ¿xT;1i@W€)‚“º@»º‡M7s;CVÆWÎÀÖî µƒiÐäœp‚jÞq“O{JÜ ¥Dü’cWæ*‘UhÃKölìÔKv¤5vT‰ºjp]º¦mOZ£CÁžñ$âü7ÌÐ&®Igþ òöÇÏL®B
4êKùùÂ2
Éñ"Ú\ô éÌccÀæŠ
Éç"B%Ü™’ £¹Î 9 4B¼šð jš_ §J»›ë¢Ÿ³qroPvœwmÌ B×â@¯¾QNÁ–F´ûžõä JÓ¥ß×Ö‘ +%V5
—ï4ÙTÝ´ÜßLÍŸ¤ ™ëéw™iÅ
N€ìÛí‚"‹S0ï>foçR†9Íl†&œoÅ+ýÒ:Í}¿n ¯±Yå>Ÿ}W:—ÖÏÑìM×¹M¬Ð¯èF§‡ŒÛŽ¨-W ÿ
½VVÉBötG‹¶ ý«ª¢¼x‚®Íã…Ñ^GmY¹—IïÓœck<—ý©ù”dvag¿ç×ÛÛò B'm\ÑëMÔÊü$rÐ&<á1ÂäåqD?ïõ@¹¥ûT>ô·–ï‚[
˜² ð²Žø†ô>¢?ò¸§IÈ
qÑǨfáסz¯ˆßFµb©ÊÞ=ÂÌ£33ð `M:—Å›p´’ °¨J׈–Þ+ä¹K1ÏEBTß0VHìÕ–V»ê¨º‘N¿œ>:_“Zn>IN ) X]B"ê@™|*ñžc-ÚQW\©µ·Y¿¹æø kÅæÞBÕ³ zØ6A˜Þ,úÒ
{g¼Ü‡Rà¡Æ}ßÐB=ÞÝ4Ðk‰¼bª>Lá_:,çÙš”Á'DŽb;üX• âßÖ¢rJy,â"¬±<# ûî£)¹ƒ3áA«ûiO7çRÔàð ˜2mlµü€
vÏßoô|›SÌëöo
ÞÑtHжdP¿‡âžæ}$…Ý"z–ÊßÃ3Q¡Ûq`4j0õ²þjÏuB Ž uîbxîE$_gþóeø$¢Ï|l« ³Dª¶‘²ð½‹x¶êG4YÍÊB¬hs«¤®»µ¥ióÁŒAÒY= 4¶Êˆ°”À.<ö$j„g+¢_*»_Xr(ìì&£cè›Äs1Ö‰{œ÷
¾qõRÂGeÜ.¹ùµw:V=)Œ3iû…^Bœæ5®5½û± YTšGT?º¬˜‘)¦NŒx˘˜·¡¯O°àðä;ÃÏPXXd·aŸÎåà,áos{¸d}ahÈœ”Æ1ŠçÏ什Váà}€â&0JG4˜¬*OñÃÙ@‹7ÂÔ¡Î\åç;KåB õ_y § »“s!áùp, »ô¬ç’mAS$È ‹Ÿ9ÇŒèã®gy‰§–Nðr’o¶ ¶<ª¯äõv Ä6y4ªÞ÷ÙtIÖuW<N|n!+;9ÿN°÷»ð(Y‡Åœ‰ª±® þ¸ô+zóºL«w\Ûö‹vÈír•,:‚Ð]\áV2
·yòŒß†§v£HW ï™l¹•ˆÉ
ðÒÜ£«U)è|]ÞÄ]TÕñB…\~`ïÁ9ÿ.æ?·zGç«0ø„ ¿§‘‘=ûeƒÐ Ÿ_Š÷6dä S
˦þ9)EwîóÑQDq H¨ÓþÞŸPE}¾äCÙ 3Æôç[/æsîºÖ9ºwškòe‹À˜
®Cëá 0I ãø¾Ü¹Ú|•¡
…R±Ñ¢ÁzrÛSF"¬£¦³©'j’é5”mÆG¢”I™.('{v"éñ“Ç•÷ÎÕ<DÕFTý«ð¦e«Š_bºÆðœ/v5vÏà jU›òÐÞE$åÄ}¡03½7ª|ÝDèËP_ñÇÔ7Ž’qa÷yqx 2ó¶’`ç h^”ѵ ð•K²T‰)ˆ@&¿fÿÚ²mgÍ Œ³‰sï2+²
?Œ~Éœ-§ZØ úueSXÅ–åj6<røZÎÇb‡Î»GDAºyʬf(jà#ãê ”š\^ì‘7_vg-„>ßµ¹¦65UoEžIkE Ñi#ÏÁaz6&cÉþ°wVwâ}û+ë¾™ùb…1fn0*Tçx`(ûæKëTê–ãMY±=––ê'©ÊO‰?gä¯Ý•°ÝûUE_Ò¯
¥”O ÝF§f¯oH©‚I ‘Xôáwb K(10ÎûbË,W¥S<2üõí‘ü.Ìm^ílM =úW®º“µ² YÝÃçmFænx|':´ð[?,* -‡orµÝ Åç_òÐùuSä`¼“¶+£SW¬!| G³qgWlìÙÓ–ó¬ø[×Þ¥ŒS ÃötÃìe§"«4ª7«ë3ú¼¦~~©^ÁŠtÌaÖv¡m¬Ìhbâ«ÛÃƨ¾ä¬evy2à
Ñ áî—Æ
W æ#¶¼$q\“…S=‡ fèÏ (¶’Õì%¾¾—Ô[Y¸*d—kÞ’Lõ®`
…¯‰…ƒ6s%Ü;û^–¤Òv5ô5§ÓE¶j¬÷Áï#×»èB^)ؤ3 ‹>>¦ÊîÁrY?³‘ømZu&y[Ò¢Eühã_lW呶öÏ»ÄÆIÑó¾ì™O•"¹Æ=ƒ¾\ï¾\Æ=ÔË|× Rg›[Ta-™ kékˆä|ƒþLvÔv¡Oó+’ièvµÇØ3…U\„¦|…"¹¼í÷˾\:ƒŽiyƒóÀõÛ;2}Wî £!‘9䡱"¸Ã;ëÀ )ß5=¥½Ø½k>7ÜøA,”·D=Ò#T7¹ ý>fB²ÂÖnáÍ‚±Æ‡YêÃhdÿàO¢EûgÁ±ä7M[eKˆ ê_ä—?Y\}ÃVy; 浇)1‰vF¥P¸µ‡³/52…G#yÄ&¸Ìg&6·Ï5Hsé¿ÛÉk-ô?ôÔ„ë%ÿ¶ÚâZv ^|ÝÖ²k¬ÂÔŸóQÏ—¦<*íeÚµZvCºÑ"]©I(ó:øÌCÐ,xÕñÂ×õ䙺ûÛ< {¢ëÖ¸j}¾Èx'i3û§ëT= $û#ßö!üøU/ ‹æ'½Ö¦í)Ìë„Ôú%ãþµN©ê±Ôš]ö)Î)ÃðӬ׃‘åþ™ç›®=)yq; ñ÷Îh—¼`µJ ‹¿Š÷}R "I"sn{^pî˜Ñ‡xè›áý •oÛw¾\^Š5/QFÙ‘ ï¼¥óöJXÈOg¿~v¿ iº }†‡ˆé!՜߃[˜-áçú˜B#£d“ž§˜˜€¹ íyqý êÕùPæ¢]ªˆ»Sàé…vyôCrtn*ô*-…wc´ìðò]Çä阑½"‘•@”ˆ9¨["Ä4o«ÏY¡§N÷YãßæMOï–9¡ŠfÝ ï Ô8R£7rX®É:6ë3Èà!çVôz†LgQM$!7Ê>©™8FÃÛâÎÁß;ŽÏñ¸ P™Î£Ü<ÍÍî¯#³|Ëœx‘AyÖW$”~¾„ë¡.HÍÝÓÆ,sñå ? ÉA½L\ ׫ r÷íVNÑÕ e?Ô¨¶?¼E’¤›¢ –WäæŒ:ß¹ nN7‚ׄ¬Hò2Ÿ{ÞF×)Äí°Ö ¹TÚ_
É¿7"RýýUöÈÊÿ¬Dendstream
endobj
40 0 obj
<< /Filter /FlateDecode /Length1 1141 /Length2 5408 /Length3 0 /Length 6113 >>
stream
xÚ“e\Z·Æé‘” J:†n…¡»»a(‡b@º¤»S@J¤ énDQBéîxÎ}Þó~½¿ù2ÿ½žõ¬gö^ÃL¯©Ã%m ³Ëà p. 7P RÓò€Ü¼8ÌÌ 7°Ü•µ‚ƒÅ @QQ @Úà ÀÇ
‰ñ‹ˆñ
â0@0o7G{8€ÄöK$ v»9ÚXAjVp°3ÂÃÆ
ÐÙ8‚áÞÜ i ý«Ã
v»y‚m¹q€@€£
`
¶w„âðü
¤µƒ„ÿ>¶õpùOÉìæŽ`E„d "Ú o€-؇G †˜F$ùÿõosyDÝÊù—ý¯Kú¯²•³#Äû0g8Ø
³»Aÿ-5 ÿM
lëèáüïªÜ
âh#
µ‡€¼ 9ºË;zm5 á6 ;+ˆ;ø¯s0Ôöß!÷öW =]ic%}Ž¿ß󯚦•#®ëíòë/ñ_ ü͈Ûqsô˜ðróòBÄç?ßÌþ5Kj³u„"BP`åæfåƒØ | G¨-Ø öBæá†Âàˆ âNüv07œ_Ï)$à‘þuô óxä €Gñ7 x”“€Gý A(µ?€Gç7!útÿ!ÄðXý&„‹õ?$€PÚÀœ×¼ˆ@¶ Àþ
â·8þsÈ (
àqþ@„3ôD8Ã~'AhaÐ?&!^ÇåD vûƒÝÿ@! ü D8à ÜÀ¿Ý£áÏa4 ’zþ Ž °Ûßõÿ^,˜—/¿ €‹a-*"äõÿ?:770þ×±ÿa;GÄ.ƒÁ^`œÅ9˜x˜SzsDe€\ñÄ+tvdû–DõÆžé.ÜÐ…—ȲaWöµÃëª,b‚/è_èžßR¹ÇtúiÉÿvM̘½ûâiù%˧Êð8KÍû³ ÃEèÞƒŽæ™Ó}Ad©µ‘Ê£’ÁÜÃw{…š¬²º;˜+ôH½&žoz³Ã„„
å³ z‘á
ùéµ 3\+£"WŸ¦$¡Ž‡8Ebµr, Á:=ˆ®‹±
’ïLO¿M=5!Ë™2ƒ~œw˜s†8XÅÆ)rÐÅÇg·µß^×äð<™ào~œ˜‘ÌI¶’keLš Þ¤ô{“À ²2>R úœ»áKy®û
…ºQ/ h©#û‚›óXÚ¤¾ßnŒ\µÈßä÷µ¸`ò«Ö¼’†;¶ó™øŠ]ÇeZSäzèoa…·ˆ•¡Dc\Ýà½hx¾š‰nû0çuN&iÌÀ¦æØ-§ÝAÒÞ›QlöW˜q›Ö
…üÆÓ|K•šS¨©å"ªü÷n»Å”&ÑžP ¹ ‡š€¯ö¢ ¨Áu~mŒµSä*û ó!o‹ h?컩–%p¦c”‘S„ãC6ʸڂ EíÇaúÔÛü˜B
Ñœ7àÔº@~¿31÷l+ÆÆÛM$»‹æ‚Ή½"C©îÕcâ”ÚDÊÄêW£Ä?æ¤mq²ç§Ÿ9ÓQµÄù”ôñ’¥R³
£H|\jßN„<¹gs9À¯˜g›ÿº|å íg›b¼D/dU9%ÑÍœ&Í@I–½
XÙÆí3-¤µ@{õÅ‘“å62o&«ñóCܲ}£o
¾£õÁ3C~þþÄŽ'ë§ëËÁñ|b’·‚n#ñƒ²‘ÎsÚ!]q¡±'T:°M 9ÌÒB‡ŠvËjªÇ„©\)²OSšðªT G’Y‰Š/˜íiòç¸\YK÷“* Ðô²ûè¸íc‘0MÉàÌÖÇQ çjH™F ˆWT|Û–+@çRöð
á(foþ$9—ׇâ¼ÆÇW¤¼ÚGÌ,&.Û•}Þ£Â,ÇLÑë,¡—±2íWiQLÛ”lê²ÝäpHñDw`CcÜá–…XOË|;T$, cõX 981 ¼1ÌBÔ îcîÞ°leÅ#@V[ Ø( ‡µ-kM™¾>K³ >.¢DËŸTü”,7 -´$]ù±ÛŠ74IOÿ>¤Ã[ÄGÛ.jgíeo l¨N·Ö
ü^›'þãWU™>ÁuK<âÛuÚ Õ³èÊ`Z£YŽQú£’—lN¸çB´Í˜ƒfQ”çLW«ÃŸ€eݤó“Æm'í* ËÖ •|¾XÀy¡ú©hõ@³ÓF‚¶Ù&¸ã½)vΨñÇÊ’Âw K,ôÅÝoÛU†ïßš›a´fB3–…Þ²V«a„,ïž|Ç~û) -v~_œ¯ww¢$j_0/naã/Ÿ²2$©ùÜZ0)”óüf/3þ¥ëä÷,žRrf>µ¬M¡Fdw‘A’ ŽpÔ7{u‹Ây꣦ÃÅd‰~ N’yãƒü ÅÌ"G ÞGŽýýÓÏ6Fg°SvIíJï¨_R Rž¾'<}#nÍ ‹³\Tœ;Že\a·Cí0‹µv Úê™A)iSM —9~úCìùÝÝÇ™JLhˆ=°å•¿!Š*ͺ·GWÕ• ¨2 cQg3–;zÛ§oü-ÚDð~7Ò‹7 Ez´¹êv^±ËÄ:=4Ñl•
ßc ¥Ã@?³fäµw .Ê°êõû”œê.fû‘ªrV ì ,Eî¾ ôµ!¼U$TZYâMì„%U[Ó ”s—é¿ZÁ͆IÞ`µ›+`“Eno|ì9ÏÞšk×#©ã?±lÍƸÄKÈ
4`Ž·,Ù»€å’áF>«NZß)EPÏbG¤TwQûË“wsS¶®9S¬¸{&1F–xI£î7E wt k‘WðÈUoÞÓ0ù·´.jð-ª†šRدåu M`ÿÆÜÇŸ„Pø+wx Ð;eI0E®üàZÓFYÂ1rTW&£M¢ l™Æ¤nÔ6u¥>¢@,ûåã
‡¶Ç›µõR’[_A¯Cö pŸðWùVˆVpù@‚ø†7÷Êœ…‡etŠ—LØÍåõ$mqiIìÑ÷ ’‘¨{F¬ô ²z9Ÿ¸…ú9©(© øÕE7uÒ8@™
7Ê'Ý*×#Û.ëª~ÄPzœÜÏ?p÷ìfêp²?ŽB†¯Í^o•‘{5<¡·ŽU8#ÒF—<ï;Íì‡ ¥°œ?‹Ç>ªhYɵ®YU¾zS!E×ÅAªIPK»±9„ºC¨&DÛû2ºBDm„Ð= ytò¨‡‡ 90oDéD\ÓøC²d¤Àuð®‚&~P8){yÉÄV¾zR2y²}½0²û• æNg±Æ`;߈ú žõ»2ÖãŸyüúÛ•‚•»ÁÞ%cQ¥7þ/bT’;ÇÎl¤C²+ßt6Û&F †àÈÕ'¾¨dôE=*Ý”uØvÑ£àß:vF¶K…º÷9™ZRÃ…]Üëõ.'²59ø\1úÉèλ“X%›Q@¦ ‰JÕq ·y(Þx
õ„
RpŸ£µ´[ ){Òü‰j,íë¦x^"G²>äìm¡Gœ’ iuÆ]gß+[+ßп¢¯£5‘OP£õã5Á¸~0Ý+œÕEsåü¶^ÑÛ·ÿgƒ-©áH¤tGÌlé•ía$]#{_à–¾‰‡{mjy?Ž³ÌÚ½ <‘ÿ|ä
OôE¿ò§¦q¯šzÇ…+ô…¨OhÇ ¥)óÄó%Ä oüÒ— ìãýhŠeØúLctœ•11D¨c¾ÞÂÓó¡:¹¤<ØO‰ÝϺôõÕZ™4 ¸0°‡„|´¼ÏT S…‹¡ºÐŒ~B×»_×-SÚD9ˆ÷â8£5|½sQ ŸkO2V€v€Ô¡«z¬Z¢\÷(è@~œFð%æ*¨†E+(¦AAx?B'VJ¹‘òJyxªu`tg,>á¨~.ŸJE™}ë~Nà3MhUáä:î ·“
GÞi܂ȧl.A“¬àÏ0NÇa·5®V ùÞ.«ÃýNµÁÅóൡóÒK=£°~Ú‘˜£‚@ý6ikO–€ùÅ(¿¡fÇÄ÷„2/–?ëØßt0 )5ø3ZVYŽÉ•¶²×“©$/ú>33üΠ%šõ¼À©§GYú*úXSaÌÉÄ ÃD#% ⪱ïâ*žA®ôs©bDè֘摮§g–g ‰–èí›]ªSù!Ò¾ËñæZ:¿»%#Ægéjo<Ð0[µVŠ¸ðHMa=ò5 ³eÑU7$¶ŸžØ;„5›Næ*Æ&-ÑN! çwTo}>¿‰ŽVû"®t¹Ö«.»—#*EjA¤G` ¶óƒÑW¨ÎY*¸³Øh…&v|\þE¨Nj|ùH!§4U@üzë‘Ï•H&Kcož;Ö)1û8ú¶©œ…µ
™È>‰ØüJù2Ÿˆ˜®´ú¥
Á¥U åH• Úf‰ÔÓf”.Õf8¶yèùÖ°S\y6WSxÛµ[µ Ê”ŽÀÚjÆÃÔÞìoñyžø_gWXèUo!šäÆ©’uÑ–¯FÝ G?+‹¹dî¡+†*GžÉ §rŠ`82¶
»§Ï~Ä3,ämµÆjýœý¬3ÏÑÎ…›ëyFiñÅ;²QþX»„à†²hÍ×S¢Äs$TrÙÁ¹ÊK¤ W—D&±÷öJ|HÊ ÓuO†á«Âü AÃ9U{ÁòÃîÔä¥ÔCMft”ƒ{Ü
ªÎ¾Ïoª\ Z;—ß”€h˜±V´Ç뾯µû:Í¿^zÁ”ºÞ¨½ÃÛå`‚¼Z¶¾Hœ_Ï@»Û‡Ü1ìOé+VÔóY¢¶/2¶ÊÂ(f Ò@u÷t ´G÷pU9LÛ1©äØŠ´-þƒò m4íøáz‹‘Q‘èeàŒ~ù[ž¾âõYƒ^ŠIhÌFJ\ä%ÞN.ÖÊKG´ï»Ùk]¹ ÏC¤gn,ÇXqN„öÌñâ*°Ã®_iÎœ¨ÍÕŽàÙ¡×õ׉š¯ ÓEFA™ÚïÊ’áþá"X Utç6̺À¼·¨„¯ŸVÈ.RtH’ 5väµOÊ0ùçËAöó[b‰BžÔˆd%’<4Œ}Y*Ù ¯Àº+yk›ÈH
½/ÍÅ9dWÇÜË%öìw±"]á‘u2:2rçLg¨ŠI§é2ˆ¢Ð|°ú6?qø"ëðÔž@ù;› «€ÑãtÆ-5§x7AwCê;6»@/¥Ow@ß@µË‚¹n³¦'ÂEÄÂÈ‹XKO•Êt;ŒÔÅv_£¿ñŠîÈî»Z^ù"ǵ^QîH΢Ë=\OÌ–¨+d“êâCʬxr®¡f³·N…ÂÛ¢8¶×•Qã]¤óØ °nìVMeó¾5V‰nK㡤ŽŽG*/ Áe¢¯]ä dùhIH1ÈÕ'[9¯ÕqQð¤„H¸noëàD¾ïG–‘€PR±Ñn0û¢}›F¤e¶;J¯XÝÎ…äâV}Úeécõ:è©™•úµ¸9œ˜á6ÕˆÝ
[‘|å@ú@¶–æãLá\ó Ö^nò—¶téÄÌ„q2)4óïp fãPðàÞÃY9C³šî!ÌôÂMI=¡—ûÚÙ•œ¾V÷
¶ÚhÝ9< ^;¾Âá`Üa>ê³V{âBžÚ&ìµ[Þ…[Þm°÷.’øþýu¾ÉçE€9 ‡R81ÆêÇÞ'§¡|¼¦Ý«*ã6L ‚O²^u7F?wò]w#M¶õ{ö‡i°_äfÎ T¼xš™±æ[J!Çæ8
ÛËS;8U"%>.ä•?ÔNyÇÿä5™[6¡¯n†Då¾cwŠV ÏäÙay)L ^úÁÂ5R M`¬€h7¨'{w/Ÿ½àÊoY®æÝOÕgu_+.’uud<;{2i8L°vß+ù Ðw¾ðÊ4É«\I/;ÒááÜWº4ä¦}ñ
¹vù3M£ÊªÃuóR"\FWžÏ¾M
`PÇ}÷þcûÒ*ÙO„ù}냩Ý?Öž)[%ñV{ ç$SòÀdsŠ#PTdôõ²õ„eº?ùñïìvE)΃‘ýÙ3k×_ZÆeÀ19³Y(ð©æc¯µ%(Ø»g¬ÛÜ—D\ã .'V(ÚªJíJøWÃ_ƒ2 „ýkï´ÚšÍƒZÔ¤Rô›ZC…'ú
»çšDr÷¯`ynDxÔäë´`õLù$± ö“¯õæáâõÁ…™©´OÕÛ Â÷è:·:
¬ú–³
ui ˜C+TKGÓÔ÷à \GøÁál®ß¿¤‚„Ž"ZÞâåëïÒÇyØ–TèlãÉ>¿y/ñãÓš@0í³œ¯ ÕP´éÃs&u”ÀËKÞƒ1-Yv·+Çš¦
™'µ%¤¢ Ü`Ž †óºW‡?í±fͦQ¤Õ¥ý€m™ã Ä@îÌ„êÁôü\¹”ÁÑÈ(N– -†;^´æÁþUé2˜¤ ºº&$\ÀÔh‡îy•ƒÔTùÞ[U“ÓmëÁÈ—ø‰òuì±þÔÔt gõ„lø‰7þv
œäÖ ~z-Š“Zõ
jëcÞù'°/É‚k Îܾ;ldzsÙÈäãëW©–´tO"p§ŽDªõȦœü}¢d;8Ø6|ù²¥HæµJd-Ú^mW~“) ÄÅ6–vø§ëôáI±?ø¸i8òD pÖ%ÌRK59ûÉùõ ìGB•Ô±)\š“¯ 5Ö ñd%/ÖØ×V’•ŒÑʉYô$çžê§lQÕ¹¾ ߯.Žê$R9K|ÖËîjºÄØ’w§"¾W$áÉò>¯ø)íÜ+·áÓëß@çÒ8JþÜï+ÈLÈTªçÜ@Y:ö,ÓÍæ“Ì»„¨b´=öc’UýõæÓÖ¥÷:Ëåáã l>œI9ÄÑi ¢’R7ù¼KŠk@XœDVA— Æ›sþpÿ® Þ&îöò
REëüÀ§ ³SÝÑú+œ…cb ìÚ†ƒO0=ߥsp&5Ôv¿ºÐJ4Œc7ÏKøN¼¦ó¨ŠZ—-º?lNEÔ:¨MC›Ã%SšÀzã+>?JUâ‘!p®¨íg‹¢ãº‹œ@&Ê ¨AaÅ
ÍDX’Ÿ4ë¸ sH<\(¨ŒoK°3u̽0$Ÿë7D½t+]¼ka èyXbìŸ ¯Yž'› E¬°¥¿e*X BµúÔ¶_l³(ªDFMk¿+ù€>v×CÑçË’ÏÚv²©|ã?HFabýÎáLÞþ××–¾" Ù7ò®ÜGö˜>܇N3öeJØI㌙a·?Õ„‰ ß’UûVöÖˤ÷ î’gQ
J’cÎo«ÿö©ÀØ‚á©ìâT´m?Ò1MÊk©™ÏþgÁ‹R÷áwv’v?à»/WÔpMy‚C¾Ý“ïEµÂ¾Õk5þ ¡Žç>>.‹ïu!´QÆ>L=}û?ÓÛendstream
endobj
41 0 obj
<< /Filter /FlateDecode /Length1 976 /Length2 3609 /Length3 0 /Length 4248 >>
stream
xÚ“gXSk³†Azob¡.z'….½J
]) „Õ ˆt¤I‹¸)"¤7Q@ª¢4Þ¥©'ºÏÞŸgϵþ¬{fÖó>kf^1!+[9]8Æa„Aãä ò €¾…
D€ÈƒÅÄô±w ƒ6pÇ!. 55 ‹÷ ` ¢|AAõT‘Q ÐÇø°Ho ©/õ³HÐõC`‘žîhÀ çƒð#kxº£ [Œ'#Ⱥ(`óó‹@ÀˆÀ!àòŒ Gzâ „7ÍúiÈí…Tþ
Ãñþ§‚Ø@²)@’lR
[„cÐ( Gx1‚`òY²“ÿSÿ7£P0w¿Ÿò?›ô_iw?$Šð¿?<,0pýïRÄ_Þ,p$ÞïßYœ;
é©‹öF! ð_!d 2·Bâ<} /wT âW †ÿÛ¹o¿,€ôMŒí
dþšç¯œ•;³#øÿ£ú³øCþÃäî`‘!À°< !’Ÿ¿ß\þu–!ÚG¢É ¡¤ ¸c±îFòfI ƒ H4 BȆAòh Žü @îIà…Á2þ §²
ÙäÏÐ/R… ›ˆ#í!„ÍIWù]«ª.¾c0í¶·£©OTÔ=Ú©°@=ÚèJž
ƒ™ÝÜN=í ¯žxÔV{ÀËã
žh' ɶقï¶>€ñ ¸š5’ÑÄy¶Ñ‘.·]ò$úÞ\Üuથ›»CƒE´ñI|pØÌ žóYZ^ýQΤ)šFÉ€T“Ow–«jŠ?ÆçNï;¿pnŽÑöÄ cÖ‰ƒ®§†ÐÅÉ»ð;aÜ$Àϟاí9èÉ/ F’²ãÂ]XªYZŸj¬³Öé¬ÛÃ6”ìì%’„Ð7;bëÌ¿ ú|xêØðšú2,³ÉÀ¤À3VÚk¼ýå̚ᑇ»ûôµ)&¡—lCÄöåÙÃ[….gv}ÎòdEw¶‰QP ¸òæ÷_'Žd”Á̬á»üc¬2X÷¹t&Ò[8–&¸ªzñEª]bÅʽ’ EQu§¾v:%~%^Ë‚ P“¸Ã ¬ýë"óúµ¹}$è;þgóW{˜›ð÷æ&g{y³«Î•p ÜW-º/–¶ñîŽëñ ã×Þ‰[³§r§n5{¿ Šsxø]ý¸×*Ü-$wa¹‡j=CæyÜšHWúG(*¯ëCÌ+–O|Íw£Žg±»Ûmšã ÌõÏÆf(³ºI}¹4÷Ôa’I`XÈy§Ä”ä32ìC§ûlÁØF'Vñ[èK5ç7ûÄ+CM¥ÿ9eËQóýãõÕ§Ïöš>Ö'ð2ÚŒž÷ß¼¬gÛ1z:k2÷ဃÆy6p9![™kŒ"›ÇÆjõÆÍw .Bz’íUFÝDóщª0FÒHyÚè nr î€ÒZö‹Åë®éB~¨ªG¨´_5ÚÌP‘ØÔ·ïìzë¨ÛkÕ¢í;°|DŠâ?`õ’OØŽ»'tÜ« }×ÂòX(±dÖ¤í›Ë9a9¶“až2ì}SÙ3éE(ÛаøIuÎkDEŸP6U;Vj*QŽÞWÛ F«ý‡i Ž^1RUm1ì^‡…qrôµA+˜ÇuS¹aß1£3ra7c£üÂãÕæ,lÅ|u؇;‚©ÍéCîŽÊ*v5p ˜g½€^ﻕ]½
ªy"
Ùrvšr´ù fÌU´°y²ª—hóAÎÚÄÚä²…£hÿòà€õÄèý[…Ìm‚:5»_ŒŠ /Ê’$&Òý”÷_Å•ézãÛô‰å¤¼{ç\¤tÀ Ç ˜*i¾
Û½
ׯW·s ò 3ù«4å
.ê™LB.ä˜2w4"&Ðcü¨<@ÅfG‚*ríLÓðMá|–;ƒ,C «OJ4®ÙëMg^õ«d׆:z͇ÖD_(ea ±X4ÚDA8G¦J#ÖÔÆ©¹•ÄR›¿D¢
ßÜë¾ïJgœ‹§.IH;öaÛÝéJ
g Þçõׯp”8§I
OÙ0zœž7蟎ëzX§0×r°ê[ ðHljտºû9íA缧¶nÞüS Â)ƒI Ç•Ó3÷ÓÌŸã¨åbßÑ(Ë{)s5¤¸—.9%[?ÔkßUq»n|pöà ì<–Æn©«îK"ѱZƒ–>Ã2p9Ôì±À×jªþÝû=–'òªqe‹Ô,MÏꜿhzŒOÆ^0’ò\„jsû aÏØ–ê_U ÚÏ0ZdÛ’²¾Óµì=.Ôã˜ï?LAupGòöÇð[WÙ
«Nñ< ¹³?©OW3¦7Û È2kê±UÍ!2çøLÖ˜ÒWd¼(A…O`Öók_-• ¯„/uÍ·O…:ªQIëÜÊÕØ”ÓÉÕ€Þ†¯/,8SßÝGþt'¾·a$”RÐÚn”?¢áÓé«Ê«°6Nææuëì„~¥=c<€ÉÓàS‰±ˆìŽ‚~|
ÌKqYÔ‰K/G¢óëÙÓçï–ؽ¡Mn²h 3 .~™ôÊŠíÒcÛÕ2}5𦦳ÍlxôG‹ÜͳçÞí+R㣢ô™£}º©tµºv’YÿdUsÜ_¢-¼ÍKäÝië;i|™Vt‹Ý(6?¿îéßØÃíá˜ö±ŒùÀ¨9¬–†öÝùdA¢¿”Q|·–ªä
ˆ`gž-¥:Õî2” ±®$û }Ä©„‡ ?§–þ¡£Ñfº\ž›í`5Œ°$ðñFÉÓÈçªõBD3
¶¡_?4Áô){SŽÂTÛµÀÉóËücð‹¦üçN¨g÷Õ¥’îiíÍƉD¡ÈÆ÷o¯mÐ WAØÆËO.Õ¿Rg0mÓ^H1 ;Öé[šÜIÂÏ[¶ ÷Új¯ÎäfhÌÙü,·:Î+åyÔž´V3JPØÐÕM=g°)7vU9ëËÇé¡Ü›ëˆõ
±Ó«9†“#ù’ ꎤWõì×y+ÌV·hžàISµbUºŒµA¯©ù[c›Ão¶€Û¹ÙD¤¿-‹Ì~÷N6¯ûñ Ñô[TyGGJ<¢ù¯Û QxIÄÀŽŽögj®ÂƒnPTŒ¼XšnÈ~´s¬#%DGúÊßé>òû8fÏÕqå›@‹z¶IxlÎÔ C·âÄœhž÷åHuREÏz× }ç”…M£¨³O„4ó,»A·D:'a_kQ_†c;QIÿˆ1Å=“ ¥Õ8ï íwÏ6ádNû¤lÛ
ekI5YñXWy¬uÒ}äk(ýfâÔØÝüØ &ÐVoÎù§¡Ý¬ \žŸr‚_ÄŒGšß¡¸[ Yæ1‚É«nfÁ…› h·?/4ø€¤qræz’ $uÆä&0†n90 5¶=•×ÉI™°;y+>圚‚ì‹DéÈÍÓß9'³,êˆwŒ˜XUâ֨凜Á¯E°:tYóÅäз–â{×íh“40«×£Çé¨- Y]äÛĈ )—™Ag¯©P.¢: ”ƒdE±wTfêÕ`Ÿj
%EÌ(Æõ¼x³Štê?$¦§€M
oAÙ|A4ìêB£¶1Òò$bdåǞ˩qíª‚Î¾çÉþïVï¶cÞoq:Ë.çKORßn.Z(Ý6.-«¿üõËžd€”\®j?c»Yˆ0AD°mõ¥o§i.4l¦U#¨ Ç.®¤‹ž!$Ì$}ö*e“¢(~›ý6…8pÂC1ö‘Òëhfƒ¹¥(2á™·ÃÈ!²N]»…4ú¦6cÃ…±=àT˜™_~¯•“1WiÏ¥Ìl§c‹ ¤Tñ¾ ´«ëã_xócg‘'bàµÚf•ñôÛf®×Þð̪ ›¹Ø×\Kæ‡?ž0M}Q`]R?^t0õL|ÁL×”Kwn™zj2 ý
3d_+}YíìÓ¯~YpÎ#mqãÝû @_4ÓA‘V÷ô½7µ‹ï‡ø×>4‰µÉ{/[KÛÎÊ°¨±£r¤?ÂieˆÚ3eÄ!*ë”pÒH¢£ÕöÞpjvHßø¶¥ÈêóA±Jס¨÷Ew?cµ‚Ç^7u‹…jÕßÑŒÌTÑRhé÷7â8^ܶ»P\Ëî®=üv9 ã%eØé^ê
€u·´«ëI˜o=PJ÷®ß–CÓ}%5[J]*Iãèr[éV¾\‘‹—âú£‘~uÕ.¨;ïžÔívu†÷ ÷¶j…ÅÊyÀ¿aý>ï«u>üñ,Ãad]7׉®ë9~e6Øîþ§àÛ°Š›«êkE±sYö‘¼-ŠQAãý[Çಸ©†hßãrŠ®#£±VöÆÙ€eAr8½o:Ó5×£hºwÃÅå´©æhË^¿8E·àÒõ9RÅõ? O;"endstream
endobj
42 0 obj
<< /Filter /FlateDecode /Length1 1441 /Length2 6981 /Length3 0 /Length 7825 >>
stream
xÚ”w<Üíò÷#úŠ-D[A”`N¬ ½÷ÅîbukuÑ¢—èD„Dï½÷.ˆÞkB”D ‚ ÏÞç>çNÎùýû¼öŸ}ÏÌ5ó™¹æú²1kêðÈÀ
ÎNh 0/X QÓQÕÕóÁ¼| 66
a‰F:;ÉY¢â@°˜(ãnäÇø…ÅÁâ‚ 6 ÄÙÅ…´±E9 œ‰ e ($ÌÒ ¨f‰¶E8brÀ, €:Î0$íÍ ”qp jÿuÂ
¨pC <p^ „#ah Âé ý¥IÙÉÚ(ò·îîò —å†äÀˆäb$ ¼p„5 ¤îŒ©…À(ùÿ!ê“+¸;8¨[:þ•þßsú?–ŽHïÇ8;º¸£( š3 rúßPÄßòÔp¤»ãÿz•Ñ–H˜Œ“È#Æ+(ü·é¦€ôBÀ5‘h˜-rGüËŒp‚ÿ¯Ìôþ¥¤b¨`¬ý蟋ý—[Óé„ÖõvA ù~Çÿ‹Á¿3&Ò hÂÇËÇÆb~ÿùgö?åä`Îp¤f3„„–(”¥7 ³"ú‚H'8 ˆðÂHñ:9£1G€˜É<Z;£ Ý«°$ó—éo‚ ¿I ’ÿ‡D€ åHC:¿IÒÿ‡0²üM¢@Õoce ³G ÖèßvìïÎ?ŒØ?$(ˆ!gGÇßéÁ||@ü À(;;8X¢þˆÀÔDüîê/ruÇìß?g0µ1#r°tüã ?dýbBl~§À8mþzŸˆ?«`¤ÙþŠ™«·‹-ÂéŒ
ùbó[„0¦
„›Û ~Lç(Â,èd˜7 rþ]ëì„øÑèòÛ9ëb‰B8ý×ÌÁÿ¶þ÷Ä1 rÁ¼Wç?¦ŠéÍÕÝ€[9üQcýc `L{ÈcÒ ÿ@ÌR¸ÿ˜æ=þ@Œ~Ï߈ùʼþ@ŒPïßú0G}¨¿›ÿ¿oOVÖÙË—“‡ s±Â˜ºÂbÂÏþ+æŽÂtþ×Ç
ó€ÿÃÖHÌ‹G ¼0ÀüŒ3ìq¨]JÝ‹BùÜñ"\Ž<ÜUªCÉÊî…Èç8às¨Û ›<1ã¸6®
SÿöPøÃûÅtÓ&Raµ 23<ö†åÓûÍÔiÀ1ÙÌ…«'žáƒæ>Õ$¡çÒQ_µw¢kÆŽ,'¶ s Gx}¢¹oyë4ýäÓŸ±ð„'9±=w
þ z«9ÿŽßG=~Dyç§>MÕYüTÍ{
!&µ™ •Y« âäÏäò¡9Î%>RUr…‚¿p³¿¢R*4¡œ¯Js‡ÌØ9*W^Õ©D\~—± ›Z44 LD6ö¦6$ãžQtàÓ¿
˜×¶[6Ùº^BwkßÚí™ú ñ6à§6† ½>(À"|bE`W:Þ@TgHÊ}FGw©¿šó¦ŽÞpåÂ>ߦ ×?O‘ÏK ÏÂÁB}[½0Ã4†m 4@ ç'¡$u™zÖC]WÕ
Y¹h¯m;?†ÌÞ¨¿(÷Å펫þ¡°íÏ2궖3¼Úã—&ô
·A>Ή]¦á° ®²y\J4™\0bgP{ùS+#>·rß/‹s`ÏʜܥOZ?ÌÙÙ:¾Ïºû@:D!Møð"¿77B¬ëk¿˜±‹’µ^$`‹tdÀ°ùÈ°~ŽŸÏlB •Q^Æn'Až_Ÿ„÷Ç
saùóŸËŠ˜å`SÀÎño¾Ìº¨Šêã\áêp˜dXåe?»ç˜Y- ºÑkp±í’ñÔÒø¨/ ;0ßVÀú8Qj;h¢Oߘ¢p‚U\_À·€jbû)E³F@ÙVûm‡¨ì({Ÿl$Ÿ+s¸ë…Pàè´ï·ÊåµÆËûXæWA<ˆÕêâú¹T¤ƒ© >¢’&hõšæãZ‰"~È!ØUV¼&Cƒn ŠÐñyêqõN3~_ýQ ãí ?§(Û™[F^âzK\`Þ¶}Ø’/¬Ü&~yøc3é÷LË£Z£(¥.å^žeÂè=ú ‰+=M8qÕOh¨— î`•g åÏ”®áæoAz§7ågu |욆fîÑ"ÛŽ·˜ža@Y“%1´HFµlRÖ Åsa»ØzJ‰Êo¨;kvS<=äJ%zßÁæ©ÍŠÅžîéêAúÕÀ“弃¯4éê7rÆÙgØIEµ0ø®€’3 Ñ}w-Lóp¾ˆétŒM Å”……€ žÁ‘5ƒ%åt[ýßÕg=>
V*çò+£•4JÎF®ƒ•t™ ëLD+×þ’fWe.¿r¨2ì O‘êCk< IäˆÚZ„–ÞÐ
8MjzŽ£{ʦè¨*o¨3uÅ–*pÉ7qs/-æRu+µ³_&‹Ôáaþ*?·= ¯»Š× Ýÿd–Ãón·8ùbØÃÍ•_XæIÊ(?ù¤`~Üúyåf…#öDΆƒì“&3œtZC‚ñàì™5¿–n¥OGÚpp‹]Ð;Z¡b Qå ÅÂ\h¼§2¡%¤ïXúGí
ÑSU
뜾+ÙãªÚzC<‘ ¬—LSâLI ¼ž¢®hCŸ“ó/>*Ô ¿ ‰uäǵï±Ãz9cZÞyA¹³ßÎ Îõ‰f"!دužUÚÏÔóœ tö):RÍg[ó~•dgY« Ǽ"¿Po;L¯:´}øÖ¨à¬èÔ™Iµ 1?#*‘A,‘\aÀ±×Éêcc˜Çž>¿ß#µûb곕ñmEª"Ò€ûDºÃñÆe}Á8 µòºÆ³¡Ò m5hצޘ g|Ó"þ4.kÔvꯄåwÎzo‰ò¥SWòu‹s‘žQèǤD Ôjƒ‡I ‚•es»š}MëybÐÚÿüåfl’©_ä‡<ž íµÛ§(~»h»-Jdç§QªØ©<±ÖÔ|è»gKÖŽ£{ÇÙNCÄUY| tÒÅ?iªŸ,–=§ ½™Ÿw<‡e¯æà8F¾ë£Áo'íkÜ.÷p|™O=³a{ãÌhHùÄ?ì*ë»õ{>ø«¸f«J„c‰ô•úÏPœ»Ëvsú9 ‚9/ÞuòiÅm›© Õó‰j Î3þ Ë”DmÀ4yK78m[fö= .¸ŠýR‹\EóýL&ZoàÍ Æ7"oÜŒÁDÙc1ÛÆèº3Ái3è=dCxI*åÉv˜Ï‰ã¨Æ!®˜êà ¥=àéN.áîW~fy#h®/©Ê‹(™`iK,ñ÷¬§\¹Z©7û ~žs|¡?º'¤š‰0|wSW©Ðé2ƒŒb'á$ò¶6…o³yYsSi;ªÜ²›œÛÇÕ0•:ÿ¸Vÿlg%>Âæv¾:ß)2b[‘2¥xÔ1[>¶m\§¦,U¦<¤_Ïܲ1È
á˜|ã|?t}ã{Ð…÷nßõÕÎS9 G_ÿvY^ãó 3Ç‹ÊM¢? a•ÙÜý,¸MÆ‘çöèˆ|Íuå;¶‡GOo—gÆuŽ¿L’\€Ç·ø¥ ^¨v—ÎÅà¤Eõd’Lí ¡}ºâÚ<Ù$pr8bQj%? ×þN°Îi×bå6;åU2«1]zY˜[»‡÷üÕäX€}&çä¬;¹ï`ìSÛû”.Ý¢˜#¹¤ŽLµ¸GÖÎ
yKdC†'Ï’–×É ˜f\¡ë:ŠÚ/âRÆ}Ô8ý¶ÈÙƒó¤Óµ5,MêE£ µµýFî©ÐAeA7†™,£§è0ïþƨ ÖƒÖ!šše~éš~N\†BϬ ›á9k|{9e²6Þ%i™‰bóØë ÕBè½ ]_$¾V†Ë˜j'V®q½Q{iõôB=ƒüà@…¹®AÓýÆ8ÁÄ?”iòËóIû5{{åd
4ø„7Å\>œiW/YÅïŠnnì
u$;±Ì ^ÒIëe2|ÐgE%†ö±Ñ+—Š'0QÇabµ;¿¤ÎŽª¼[vRsÏÄØm‹Vª¼ÀNžþ<©
_Tõ\VæªÂÎu7\êþ’83ˆYè Ûš ¿ÃæFR¶Îæf ã‰wYšóг Ò¢yO|nñŽÊIMw A G3†™XQy¹=.Ub°:ƒÈ™Ð§d¥8£ˆ©ò[é’dŸhCðƒbko=¹C‚n Oq‡ô¨¾}ý}¿Æ—À79!5²Ò/½H-è:Ÿÿ¹„)”„ª½m5“ ȲÂ6EÊ?ðź¦V é^~.Èrö8ô ç*ÞÝýUo ½¾XP¬C©@áHñŽIðî¢)Ì7¹å~v7Å:ˆ§öBÂân iÏâ±ñ.pÜ(?æÅî#ÿ‰kÚ6ÐØñƒ¾æ‘SÁóûù[å!ôKa6õ*û[÷9(†.ò?y8¨x¼Îʇåßœ 9\¥@|·° îOb°ô4D);ØÝO×S±s¡¯Ù"‘IûÊ ùÕ
síÞŠñÞ_^ÐKÞÐ[®Ý=5‘ L…*f=9¹0bM‡ ÞúÙ†ÿâıùív´%ì¨LQZÑøªÆ¹€|“Û«ÿsA+bkxþN
oê?¡ØKÿ5_:ßFÅÚi²ÓUt?¶˜ ä d6W'h«Œl•Ñ˜ˆÆe D¬Á”<}xûjr£øòT¢
F2ÑE5 a¬í…iÒ௚b%;e H ý×¼uÉkM„;h}'Kîñv‚IQÎ~Á8 Oȳ2‚ÈÇ §™íÈzc9¼ó;ïùô¦£âX±óïN¸8<¥Ñ]‰jhfÊÇ"šÝþð“Nç1¯-•îWj<Ê’çO8”©ÏÄÏ;øLÄÀ9kâ#éºýiŒwºêbÆ$…m56lY=¹ q:z(æ
dö Û÷UÇêDÎÄ–râéŤED›f
NÂÍ*qÛ›ÓX|v<¥
Öº‘.jç`¹Uêùá±Bj20C$ø"iE¼ÝëÜ Ý›On<;nöÅí}˜bÜ:6„µÖç_稓pÎF-?VsudÚÑY™®/Û ´|œ‚ÿùõ²*RŠME:S¶hµ?óÚØàágóÑ$è…ÛÙ«Îì¦4œ{-é:¿êNøq¢è&:1Oi
þ
ÞDQÅØÀqv1êKPC–'”C¥‹x«’@)í‡û•Ï…‘‡U€³Óñ‹bé)
Ê;âµî|éf‘F©PoõâCu§ÝZGgïÓSòI&~:¢ä‰¥ÑAÝв+²¶Ö®äÍoc5‰=’Ò̺© õDoÒ%P]s¿ Ãw¥˜s~ê†p_>Ì8öÔå¨ãŒ¥I8[ÁÂ+·!?SÓõ(\R!:Úæh_-Òé¦.w@•åaçT¢;t¯1 þàíѺÄÁÑÙ/†mÕïÀ_ÁJŠÂ*¸d Þ\•uJenom9D=m“ó êãû6BŒ»u-šnüp¸,Û—æbùrWXf ‰’øåuDhJ&#óÕ\nŸ7ÎÜrèe²sÅ6Y®½ÉÞe«“É76º„^Í£Vï
°Ò érŸ
gs‘$
<Ä/W708ÿÑnÒ–&KŒÔó'R±î/^¬¦Î㢕¾eñ„ÙºìsÊ qoq醯(—m¡ßò›ášz%ÙΞ í½vùs;î*Õ(>Ã;N3r5?ú=
H÷£A ûçÄZ|†çgBÝ›‚¢1 ÇBÙÁDRvæ_ÅõÛ,„ >b±—6·œüÖ$%ß‘~?’¤ŸV¡êNäìJo&÷æÅ7âyÕh‹j† ¼Ív“1¦Õ^Xõ²ÁÒ !Õ–Êw»ð@G‚j¸WR²Ô°¥ äÑ0c\>"C¬¥©dÒ§wˆ†‡y‡›m?7b·<˜¡}yß!|$½¸
ýëgjô,‡6T…•uLQÚÇ‘çו§e'kN–Ϧ¯.sØáV³2rñ“¿v*£‘ôYísŸh¾Â²Ïá]ˆÑýÎow;£ˆùn=g‰ˆûû² s…-Z…É ýÍÞëà˜ ‡ª˜–¯
Ø´q)ÝÖN…i+edN +C¿blî ܶÎq¸ÇŠšþuvÇ+©Ëìqõ¹W¼Ø»zW´ó!¥…Z9—ÚþЧӃ^;ÞUú®B¤¥dâÔîИ#y¿<Ú²_|Gé N†ï#áEËEôþZì>è Nq %~ó÷ó
¤Åjbƒ^ÝC’”©e&F;OÙ"ìØpz¹„3›©×óDo;´ãt~0»XR
Xo8´‰(ëáðQºdÒQ.Œ†³ÌqÆ-‰™¶Ãú¼ X‰ÝÁú
Ng¢¸ßÊÈVºÚgX7yFqfŸEÀèï-ÃÓ6Š‡åÒ^P}þP8'Ô'tûRÊÐ5ê›O>@Þ;ËÐ¥Û©|5—RgÛË°F•×pÔ?Û¯R3{PýÈ—;“ýrÐ A”È‚ço`xÓs‡vt˜?€òŠ„š ?x訸?£Žé=YÁV܇oe{®ÁZ“bkú-ÛŸJ&VÇG75³BOQ«`{Æ…õÎO¬#Q*¢R] m ~vî® ’í¡ê—óµi`A*ï9ò·Çø^)…ø¤âá‰$†¼²Üt‹Ý«$Áù-úax@µAã¯Jf„ÞÜ=õ·JÔÒ.5/\ë¦é¦ö™r†.Ëï6õ Ç 4«½iÛ7Om\šY9k„~¸‚Iâ$nL³|´âØ}EB?CAÌã.V\púá€Â“0 è¿ÒÅ+"1 •èËÒÄ=çäMc»ôJ‹\‚îÉ4%¦¹Íe€uR®æÉ+×-²·æ}°”{µQih|ðöóu¿F•€²»yp wF1öø›S7 ç)vƒ¤Á[ }Ÿñžg³Ôãs`%
øî˜ÆÙÄ`w›µp1Ç'žõËÅÉ°Ëظ2§6Ôöö3Õ„Á ÒMø>G+{M›T2ÐËÚƤèdxl38rb”•4@ÍÝ Å¡e¸ºf~¬<à y~–dUs ± ÖÂ’a¿vi(ÞÆjmFÑìØXdÚjqÃ+ˆ^¼³¢4K· EE
ÒâyÞòŠ¼ú›ð&‰L³–R~YbÍgœó#ÖanäykÙÄ/ß
î:{9áyÔ!¾úü|ŒŠH©ÿ{`'jžVž‡óÒRÉ ©ÔEÜƼi£Ësà£Eë5âµµà}7̲=. §õ¦_þ¡]†“'Ñ)rÖý—9 i»’¹Á ¨uk"ð¥§øéÒ,1b/ŸU/„aE{óHœ* ™eŸäÐïªý¤'—¿7à $’» |€:† A#ôv²ïŸ5Êç}ïžßÜ+´x-K·5üÜbq“@ä;#^b!×ÿ21,Æ ¯Ñ£\׌8+¿óšÛä—ûžyÚÞi06€ºÛ±0l.\ Mó¹xo€þçêãQ–ì²0^Š¢‡úå¨ËT£i±XÕö“üŠÈé›à´ÆáAj*^ (Á»`ée
çÊ Ð
&ŠoÌû;3©‚>¾¸Ëï˜tÌŒyÚqü¾âF< #}Ý bÊc“Ëyg4vùÎí{ – cÉýý 8R£÷(³S·s¾•Á’ ØF3Î/õ½Y–Bú$‹9^¡(-ÇZ¶Ìß<[êmx´T`þEY†ÂH±j2 ¦Ä{ÍñàîêÓï+™‹®‹u^ëy ix–q …ðíPŸ÷â§Ûéosñ™frB’ÝË9{(Ì˯=šjXóü5
ëæ§ÑS)yš³ÉE=©<à.afñ6‡ÂË×µÕOðpH_¿ï>èf#,ØsÚw²&*«J§of¶“Rm\däÍ8]°[µ!rhÓôzf±½zwZVáYÞûO‚z¸žC-펕ջÍðÌ
§8ôá7·AÔ~r4M™áB$œ› ;Qp FòEõ¥¶ÉA=k {ß« éŸÑ&ÇËWË$
ñ3YÛÉ&ð£'úz}ME eb•-îñÿ>WäÇ‘m~ðYÕù Í›‹àéè±CãÐûB'z³gid-ÐÕoÙ*ê_ë#ã'bÏZçLà8ÁxOúÝ‹
gÏÇ= ôyû™Ü#¸†i¦QÑpÏI6Joû£Ÿ¨qU+ f)” «ÇòÆÿ”°LÅï7ðâ®Ê°,+FPäí w…µôêý†ýjÇט”åö
{Uïj±Ÿ
EŸK|¤"ñ|¨øù=åáÄ5¹wìñ6¯3ô+Æt³Vœ ÎúìélÔø›w?±‘ddºcœ\Â`ÍŠ/kSžüèý‡—„Á6¯FÑÞ²$ ·F*±ºW:ÞE—†ÓÑ4
L!‹¶)ÚfÈd9 |¢HòÁÜŽR7É|P€QúT*©lJ/›éz} |¡f€X|ÃwYŽlsˆ+¢Â½—Ý: é|à5˜žÅí^õâ‘UîîxUJ¥”¯cl÷q*h ;‹àò‹è‡ê
ÀF
heE»¸p]ݬ67¦-u/ïwWYÒ€ýDݪ N2ì¿Ø#ªäÉDÂÃxHsã vÏO#žâÕ ×(JĈÅ<Ÿ‹.·¶ð#—… ݪ»h%?^M\Ÿ}Q¬“Êﾓ¹ÈÉ,ìh¾çû.2Œwz¡¦‡¨8–q¼€ß®—ã*nöÏüܸ]v¤ÆM¢À¦G(òµÇ[ã= ô?º-
ï~>àcÞ@«@äˆDgŒ%_ïoßå¨~˜”ÿ ÜøÌ×ÓÅ ü¤+â0—ÁÜ&dä‡k_rù=.ðv„ã(5i%ÑçªušíEµòÖX/Üç¬ ¯}M!gŠkðP„‡¶WgŽ´v\ÂvYaþ²á/5f<©Ø:!]Ÿv?·*ñÜ`ì I„²[};¸ÕÿŠ
íÁÕv^œJòÕDöZ)ì™@‘ùƒÐDr¤çñ5šÀ»<ìK4ÿVeË †žþ€ÎXgül°ºƒ:¬\6—š"Ç\ÄN—‚QëjN š j·{ÎMÎE0ûëÜ»[ìÆ$-?ÅjZ ‹‡Õ‚k‹Ó¤TËØÓû¾øU¿Oª¸‚A¾5†ÙG’…ŒÁì{ñÁA¦Ø_Užæˆ Lh`!Ðâ³´Š˜¦š^ÂcclŸïøPF¯ÛЋIånºf£QÛÎõ£wkß ¸Î#ÎjFu¯zì`fèdåX:ïEõ¹þ@½Ë4ÂtÅ^¬²ñQQž§sÖíBá›o?8Ûå†|šÿ£,ÿ5y)$Öë€eãFIŸ5ñ¯^G£¿”—rju§âÚ¼ÑÖñáô2…ùúÅ°€¢!õs{V‘ç™îG®#=9÷_-,|#òèɵØ_çI×pÛ¶3<ÿ€uGõÐÄO¦©¹X„ž²ð.Qgj‡
°®©D>S03þö„æ@±{Š<„Ù9¬ahÄozxKmÝ w¿?ÆÛŒ¨Ÿ‹_íJ÷Ócõ¤‰¸{qW·Ì¿›9kÉÕe®;ÌÄ}‘7ó·“ ¦io[6Ô2Úu¤ß§Ä»Gæ®t« 7˜n ÁÎhÁ©´ BÝßW•&Þs¹GÞøõÿ Pàg endstream
endobj
43 0 obj
<< /Filter /FlateDecode /Length1 1359 /Length2 7058 /Length3 0 /Length 7871 >>
stream
xÚte\Úú5Ý-J ÒÝ) Ò
Ò1À3ÀCwIH§4H— Ò)!Hƒ4Ò9Ò%¼sν÷è½ÿ¯ïoæì'Ö³žµ÷ F:M i'+ ‚㇓G «¦«ËÃ
àáäæ–Áad”…-Ý@N9K7 (€GD„ í ð
x¸Eùy_ F€¬“³dgï`–eù«H
BAÖ–€š¥›=Œà°¶tè8Yƒ€n^œ iGG€ö_ ® m +ê´áÄááØ€¬Ý V@;‡ë/MÊ['€Ð¿Â6îÎÿIy ¡®Q æ¿e² "mœ Ž^ -—ºb¡åÿ‡¬ÿ%WpwtT·ÿEÿ·Sÿ'o 9zý»Â ììî„Ôœl€PÈÿ– ÿ%N
hrÿoVÙÍÒd-
±s¸ÿ¹*€<6š 7k{€Ô øw±ù_
çþVÀ%¯«'«¥ÊöïCý;©i ‚¸éz9ÿÃúWõߘç7FØyŒ¹þò
Ÿÿü2ýŸaòk'ÄÀ+ °„B-½p× >< Äè z"sqBœÜ- „'~ ['(Î_'*$àRý+ô7æpéüƒÆeù ¸¬~# ‚ZZ; Ý ¶n¿ã|ÿÄÿuöÿ$D \Öÿ ~~rƒÓóps¸lþ|]6NŽŽ–Ð?*3ÿ@ĺ\Àÿ!øWÞÅ q£~·ð¸l· ÄÙ‚<þàø+íäþç D‰Ý P Àú"<ø“ ±ø7D 1ä77Â=ü#ÐçôÛD³Ó i~D·³%ù/?ùyþ ýo7ùË;#Þ’“Í ôOÿØ„!Ýõ·š¿Ðø‡>D¹+â’ýn@pþž €ìfþaB ÌéĆî@„9 ¿!/¢ü7;?"é
„þ«ûÿ>'O ~ /Ÿ @ˆ ÌÍí÷_eÖîP„ nÿ!žÜ°-ñ>@O 5ÎüŒ“µXèë´Oáeþò…ãåè¬È2vñêõ]Sí¸!s ÈŽÅC*.¬ßë o*2 l¡oÑÂFµùj( ¹Ä§»ßò°ØÊôn~jx’©æµéL²OØúiúì@ Ycòûײd£¢¾lxÿ~¾&³œî6æ R·±ÇÇîw¡‚B†
™ŽzauÏùè´‰Ò]Ê"ù#–aDɉhËc!þ¯#°šØ:µ¹“Übå%Ý›#][cDóÊäÌ5Ž½–ˆÈä ^6‘³»^Ž1}@ª/îäb´zWÝQ =Ý™o½\è _ÓiùÐí…4}·ŽmÅ:B¹= ÊŒ9.âjpˆY¡€¯¶/*ßX©FäàŠZ¢ÆÍ["FžüâÆ!-äUQSéGbïj•Ñš¢Oaä©ã àF[ÀWñÕ öÊeù¤¹nõ‚W¾/{¯¨Ûd îIjøÎ:÷° ˆÇÒHÕfœ z»7N™ó÷7oiȺü™‡²/he ü¤÷^“½}ÏÿFès»,9÷Ô¸ÕJ¼ûÝ!ˆ®Ë´fÖÑ ‡&YºœÿŠBB»;q .Gsž ?ÉÚäNGË>$¼D¥&òe’™"J©š´ïUúaZ/ûéÛÂvE-ÜÍžLñ|ëæH @êx+ƒtBCÛ&zÙiŸ%L¬½–¨•ÓÃXTyÎ^îË-ãÍß+“.ûh[Š Œ£œ©‘¾`²ÑZ'%ð«‹Ùº¸-ÖÈ…ë„,DìfÿÂÁd—Ba/'1PN‘ºköc‹éY3÷~‚Y½¯ Ù¶ 9f›Üø`ÈÙ—S{wÖUPÍ!…žI€7§b@ur8·†Y±¿tOìÒ‚Ù~*oePëSÉ*aþ¹ú˜#‰;ÚLÇàHÀŸpp?fï¸îÂH° Ÿ˜XÂ<¾B#Io̯”ÀáY{G.åëµè7cûµôªÂ´%þ‘×ü¡å´ÆA²XP>ª~‰]QþrÛLýkŽyÝK£Tñƒo·2Wù=•jKþp÷+¤ñ穇þ‹™ÈÙ[²Ûïô›mö¹Þ3Ÿ6
îö-©¦GVaiÞµEЕ÷ôNö· 6Ñâ<_–©JM3pѺºŒ¸Le5êoSŸc. ê-©2Ú³[R,»‚ ÔÛ¾ÖÁ3NÝF _´F¶ØSY¾êµÝ/M#iDè”Eo¢¢?¡æxð‚³ãtˆÄiÛ‰mç™nˆøö²‡ŒÖ³n–ÖÔ‚;¬ûWäC“'¸ÀfyðÛ
Ût&ä·eÚÓÌá SqÂr6‚î´ß¸ýŠÆb=Ù(ÙÌR‘«i%Ú²Ž öBê“<É·ƒ7 ‹íì y Ñù :r¹F""Òðê ªÜä!j³ë‹wí`³©}eO¸kÀvË-p;zÏÓ?ïsRKâÝe
«s εŠ¢~‡A×s´øJ‹®J—÷ùš> ˆ2géÄ÷u‡ªÕbÆ,ø*½Yò¢¼Êcf‹‰wç ø–˜â'Ìë‘»§KÍ É‹éÉ‹Z¡_æʳÖ÷¾‘¼˜v¶¶o ky–‚ú ‡æ7Ó¢,Pw
£{tÎ,U¸ŽØ]dóð£Äêt
&?QàêÙÏ{ƒÜn¸—ΊΔ0â›Zcö9Ÿ D%‰¬F&í²[o JW‹
^«åb1ºå:Aˆß²G”V_kÂá‘øºN/gŒ ×fŽÛ®t7ší#‹ûõŒsâ"à†öô6úHžzîÞ©[ªðĈÙø¨Ôâ§#O…ÒBpíDçùÐSTJ²}‹>èKµo‡ ÇÁ—*ÛÛ^#k¼™¢n©(– 0Ö;wÌ—S÷!½5äX™n&)
¾`%ÿµÇI9‘µºÒIL"Yu•ÄÅoÿÖƒ)[/–’øSü#@Õ$öqìG§,ßÍÙ L¸(Ê¿ÁŠ…rp¬¶ 9SdžÆìÛ&‚Îõa#Íç%%O³{›a4Žc£“—WP?¨ÒæœÝ«ÏÔ7&æƒwº ŽGÅdCpEá‰0ýBR¥ÏGüWÄî¬6Þåü Í!£CâìÃAdÌL¸oµ¢i
¸¹o‘‡|¬‹5N“%â(öf¬³2'ß÷×b<È/-ëk¿¶ôXr X6Ÿ_YÛî1G¥8åpªñ)óéhÕ¯0ÅxÛh• $°-ÝØ÷||fâÖìŒ@N–ÙòÇ„¯ui ÏA`Áò•Ã%
Ò¦ sYÍvÔ$l_Zî"ÝgÜ•»ZÉhòúŽX¸iM é»òU¢f½0»2}[h˜ø’öy!ô ¢»˜%¶¯‡Ó• ^œ¼¦Í|‰õÒ]™uƒÈD>ðÓÐæ6[¿Ìö&ˆ‘õ [x¸BØn¸§ý„A¸²üÂ꧱_rÇ0&jüh ×Ì@«Æ¼ ˜„cOËÔ%¢µY¶rl¹ýåLf±¡ŸØųv{LLµSLŽ…‘5œd³oFáúÖæ&Y¸âA)4r§g±Ÿ¸\12Z;¶ ÁÜlþŸ_oñ>íÇ»¹»§ƒÃò÷"ÏÏ›?€°ÅIj¶Ù(›E5šk0œ¶ VÏ.‚±¥ý˜DôÞ:[®=®0ŸÕV¿ý|‹±šµ~@ßiî+I¾·uÉš‚߂ͳäò»@3âÅç´7i'ß›òÂpD“,7T ¦“ ¼Ö”Åì5Þ95c·Ù;§2µ½jžÏa ØHR£¨Ÿ…ãéêé¤4 8‰‰sËŽo%†ØÙ—46m¹²„´Åã*ãÛ%úR”a*ÇŠGaÃìSj:¯U ¶l›çÄ'öEÍÀ1y¦œœÕ‹,Ó%¾Á¹TûU6F‘‹êûj
\¾ ±bò!âŸÂ )D/Šsxa‹ÉPý6 !cÎ4-sÇÃÊýõþÃ|“”•è;ß tqÐù‡ÃHÈÊ&Þ…!ÔwO“Cø†cÔ•l п*÷ÙjKãVS6Éþá}Þ–?š ~(!êqØKúç]ö¶0 ‹…ÃÌĉ§vj¯2š%"©-ß®Î˾&¶°Iƒ5Îzƒª Љ\ïCË°$u¢hop»tý4ßs|®)r÷}õýÐ÷ñØÙjsÝù+Â$jbƒ , ’1è¢bäåUòbÆ„§TIi\V
¬®’ÖRNÔ~(YwéþÇŠÞ[,¥n *>JÔ§•½ÿå¶_•7tmÓã4ª¬1× ÞV¥O)Oo+‰×M!M®>
k÷g•ÃÆâõº¥Aç©%¿h¶r§˜æc\!ÃO\µt4{Zñd"Ä™×áÅH ‡@€ï…4kÜš …äî‘O-þûŒÉ Ç·|-Pªû–bL >,œÌ ÁÅïƒj)ê)ò¹’bV?â;]xщŒ||â*bYƒaIˆ5È4à''¬+«ÇYú|.>maÀdÂb?à 6ú2¾³
;gÏÁèmHèRo£R³Œ³ºó =ïšÏáB9“Ò·ÙKGû4i ¨>á“€›–„S?0"ßß 1Œøð+©’ºHÿi>I0¬ìºí¾w®ñÑU1à³_:ò·ì/*m¤VGÕê)ÛÅŸd”Ñ3ÓÝu†SÍxH»ÞõQ8 Õ^‹‚=~uvg¡¹èŒ Voë%zÔØôÚ
ÊÅ܌ĤYÆŠ¬ùózÌwÝ Š8Ö˜‰¯üª1ý8r²ÛJÍ3?µý
¥’¤©.|ä‹]·È÷l í(›n}¤ÎõÒx¾Ss…” Ã¥ÔópŤåéºÀ®iw û8Œ¾6)´ëJ¼HÛ®/Bc@H¦Êm÷üÔƒ´7[ÔE`9½Yg5Xâ½Ìb
ExoŸU"|x”Kv+l¤K=ÔZƒùH„ÁÊôÂO ÄãXѨ‡ÆÂP’|mòsåo?Mz;†tÐP¼R&sÔeКÅZ0bœIxÕš9 ³¨o¥¬Tià™K4“êîOÁÅÐZ
Cgáèùµù¬3l2ç ×µó1PѮ֊ØlÇqå6 <öÊÈ¢Dæp1ð`WØn,#tØ\‡Åñ„ÏSmú²RÃL¬kÒ ×Ë^_"2§ˆ6mˆº+r (§âÉߤ³¢Æ„AÏZ*ËÈÕI €Cë•VÉyƒì¶²U%˱¹Ôíl‡o$Õ°e…ÚG΃7k+Ÿ/gY§®nêc‘¬³éùñ–Õ—¦gýå©Äd‘o ?”W‡YïÞŠ[ew
}äûØEÆÍM÷
¬Ä{ªŠÜqÉjâ`ªˆ‰äÃG»ª}ì:›ªûU‹UØw2[6vG¡0}¨]šŸšé§jÝŽ‹íÄÔÌ«TÆ,\ ÿÓ‰¨=aŧ“¶¹4©™ã厷vvÎɱbD>ÝŽ³ò{ÁÞ1mǯK©ñ0WtdiMÇš ÆÃT?–*&]dŽ©ŽÍŽƒEzøôoqÄ|·J¡puÚÒÈ®œoÈŽ·Óê ’3‘¶\ñk!*\ ‘©%6/ÇìHÂãSù÷íž½.{ßO„"_ëîV–Öz ËiÌUÎ{ywO&^ðvÔ()ŒÜÞk Ê£•F_ðIÃøÑi“ÝrWð3y3Ž€åM+%?Ec©E3ÚŸh„à1»Ì]DPwÑáXCe¸ñTï •LT
à™žˆ«?^]Ò65–†u:$iÚn?°ÀqsÞj Z?%zX¨îbþÜ´†QŠW«¨%%MSIÙ~Oyug d™x® 4eé! e¡ÑÉÉȯeî+½ ›
«Ö÷*|±÷s#‹ãÕž£€3n’Þ@ "QGçk§Öz´®P3y"É÷¬«…G Ói¸v5Ï·ü¬qËåºV¦'Ìã5Dw’b›5>*PÝèáq¤æös¼½— ÏÝñy¡<ösÿÛˆ¼–tªŽ·(‘v7VnjvÕºR;ëæ,)›Qµå–ËŽ®>å¸4µyhþþÍ’¦yÅ¡¦MHc2$£‡¤
œ@UÚXß™&Ì>%;»¿ÿ!,Fé›M è8÷ͳg‰ÏZù%½„Šòa6ª¥ò}cc9ü´Ó§ixíŪ .y˜›§‚ÅeU¦[E½7c€#•„V9ÿøB¦Õnö@ÿ‚G;ŸQf÷) *¿0I‡T~«ò5#m®¸¾[)fž7‘< 'Cîï\ºé
e˜ëý9l/P‘ìªä{W™Â†ÔäáÆ×2 Qw~Wæüêà9°suGٶł.YåûàÒKÉ@åQd:žøÕ ™…Õs
Yú{‡ìíòC9¨˜yŒ[©åŽY…¿¬“o•ŒëtÞ*0HCêجgSVZVõ{6dðPˆÿzMêá¤ÇJ¸ `üèÑ« Ñ/oiåÇ:À¢C{¼{HgI<8ã2ÔãÒAw?º|o–5Æá–/—ùM¯¡]Ì÷¸GM¹êíó ±ËïXÅ—=®IBÜÇKPH’¹¥ÚÚ(Ê»|s ‹ê–7ÚÙÿ!íH9,gÜ턬BS·ø4|°
Ü(¥ aù ï"—qô]¾DíÃq`eq%üE°ÕúY¥ kdÀ·jd/ø²éÀŽý{ÀÞý¤NuËõ-©ëW×õsPÚ"ñš½Ûµ’-*ˆ²ÛùÆ@ uV¶¢úÔ@mØ5÷œ9ÄP†Ð;äm|¹y©¦àñ™")iq¬+-¶²¤( ¯,ìt¶%¹QEäÖ“ßÿ$ +8êÆÈ\ñÀó…RtC<ƒ£Î4ÖÅ-Äôx6gÝGõ1ùžöÑ$kˆÓöÅ:_§Ô¼ñ”êžÓàÙ†ñÞ;ÀcóÏÕûÒLÉÎ×a/÷¢]"8rÚ#¿x:’‘ W Ìm)B ˆ#"×^4š¾H#ʬ¤Ý8 Ÿ×L.ÌqÝd
õ {É-ÿ2þñStû“.Ÿ\,•ðç‹\Ê/ô ‰už&}ýþîçñ›™ÙØGôÌ6w¾²Ûvë9Ï9šN‚Ÿ† ¦{V‘¶t%ÈP^Š1f¦“QÙA¹g¦;DÍ—}侑 z&,-µÄŽNéT4Ö +Ym“÷¨Ì;”Ïe^l×¾ôFyrV”owqLSª¾“0ž˜DÒuzüUU¦°dlÁˆ}c‘sØdJšä£‚¡Ø»íªþ©‰î5Œ+Jxi+zzÿíÊۨĈó úU¥ø]ûÓõVTÝqLµ;RÎrbT ;ô“] *@t,ºæûct?æ‰$©æ ŸÉÃKNÛùLã¶Ú–J¼]µUr_k¤¤~œ¦ÎX‹¦HMp¹ˆ†&ÓUw¤NeàceíXªå3uKë+´cuÆej%Y† þ)'ó,ÅA•“psiÄA§59±(N¿,¨
Í*äÁÌ‘„h¾, ‡J>žP—|ì[ís|ŽKi8”×ÉOŒ¢4ûœ]_I”‡iH•›Ü¾ƒ ©iªQR\ƒª‡½ ´¬`çžèòÚO•Þ±Î©²¿Êû VèQ—Öÿ…¢¤yšë ¡Cu”W_ÝèëixEl%„QqÖn%„ž«¦ÛfŠâ½pöñzyР™-Œ-I³XãrMik-;K¾S¡þµŽ%f;¿ˆTõéwµ)ѶE¡A‡eM¥8¯Ð’Jêqg¶ãÀ† j?lSòJVŽ}Oç* ç(s¡©¾Ð~cŒ6,.†‘\Z\ýBƒÝ¥NðÜF>Þ˜ ûÙKþ6 å`vu¢K…t#ì"ýóÕg“É©_`ðõÒĦïZm½o%Ø:GÙ;ó'Õë\š(ôG¿ê£ô”B»›Z –oCÛ¬– †áYïåFq®Žª&±ºdÎ$Ê5¾â4¼ó$B÷½Å*ÀûJ&šf¦˜û2of<\0Ò4=SLûƒ±¡ÍWTúª""0×ÃD¢ó,óF^t&‡Ö˜¬ß
~’¶ÕW²âî—韄q)4ä gæîPˆìÍQ
‰ßawÛºhÑž˜È•¤-Ê×®Éû:I·Q‹Ž[@a(”>_?«{0§¦•~TEÓbd{+ýIY§·KnÛ`Ký‰´ö*
Ûð&PâXg TðŠ%DJPCL¯ÐÄÜzoMiÅðvM¢‚v9—^ÏLÙ°Vüý;ÃJŠHe CÔÛØMjß*× §vΰ“r›@F|vò\øÕγ笋¬n½†ôKy²š»Wð¨¦q5ÞeŠÍýšìrØø5Ý5Ù€ýþvÕ%`Xö‹»d{·ä,K¬q·v(s¯’¿£•úÒwƒY2ÿáÒi”É‚Üûê¶yMï”çy¶±Uk¼Ó¹¹³ ¾¡ dz—ü
O4 }ò\o_½C>¹G MõìêÄiNïëÒ åÏæšF8Å=Îx¨¢(‹6ì£.cÆçX1v¥Ñ¥·Ø²
ݧ”–¾kÉ:¼âTaPÙ‹˜¤™º$§u)· CE/Èßئðz/~?ØQ´èd'ïOrññíß uîs:ò>Ò4?8
ÿÕtfoü“ ¥QhÑ‹y—Ve4¼\:ƒÔî¢ ºŒcaF ¼"ŽnBÞ)‰vx…2Õëó Ù\l3n¼û“¡“mU_S¤w/°Á}.†Q³(?㪠CʵŸßâi œ4“ŒÑ"f« ©2ï‘m€Ê
–ŸŒ”›#kܺÏ^ ÐR€½y‘ÏPôÓ>ëQãeà²ÅT0|¤l7WüCQ<ì&îål”BdzûHÑÖRS66Iü[Ï5!gŸô¼?Å÷8ßD± VæµèááuœIª'6Å¢ öïv±ßÖš¶ABÒœŽö˜ïláL‰m¢/ö¶Ñ>bŽ ‡» ÂðpÖ êåžx¼oô £¼
£ÇÖÊêñ~¾þœt£³vâÞ?¨îŠG-¾„avQº³/· ¦°óÄ}Ê~.1K¢© ´[ô^ „.ˆåþ øÜJK¢†ZG@|k ‹,²è)e$NÏœàmQÞåA«Ÿ=Ö‹òÔ§['+¢äÝþ²Ãó|vÜÍ-¯øuà¾Ù:ó0dÖ†`súÿ
ö•øendstream
endobj
44 0 obj
<< /Type /XRef /Length 67 /Filter /FlateDecode /DecodeParms << /Columns 4 /Predictor 12 >> /W [ 1 2 1 ] /Info 33 0 R /Root 32 0 R /Size 45 /ID [] >>
stream
xœcb &F~ñ‰ Èc ¤ÄÎ7™8;Žä}$ع„Àc ÁS $tb€„D dòŒ BŸ -ªâ
endstream
endobj
startxref
54705
%%EOF
glmnet/inst/doc/Coxnet.rnw 0000644 0001762 0000144 00000006437 12672345374 015301 0 ustar ligges users \documentclass[a4paper]{article}
\usepackage{graphicx}
% \VignetteIndexEntry{Fitting the Penalized Cox Model}
\title{Coxnet: Regularized Cox Regression}
\author{Noah Simon\\
Jerome Friedman\\
Trevor Hastie\\
Rob Tibshirani}
\begin{document}
\maketitle
\section{Introduction}
We will give a short tutorial on using coxnet. Coxnet is a function which fits the Cox Model regularized by an elastic net penalty. It is used for underdetermined (or nearly underdetermined systems) and chooses a small number of covariates to include in the model. Because the Cox Model
is rarely used for actual prediction, we will rather focus on finding
and interpretating an appropriate model.
We give a simple example of how to format data and run the Cox Model in
glmnet with cross validation.
\section{Example}
We first load our data and set up the response. In this case $x$ must
be an $n$ by $p$ matrix of covariate values --- each row corresponds to a patient and each column a covariate. $y$ is an $n$ length vector of failure/censoring times,
and status is an $n$ length vector with each entry, a $1$ or a $0$,
indicating whether the corresponding entry in $y$ is indicative
of a failure time or right censoring time ($1$ for failure, $0$ for censoring)
<<>>=
library("glmnet")
library("survival")
load("VignetteExample.rdata")
attach(patient.data)
@
We then call our functions to fit with the lasso penalty ($\alpha=1$),
and cross validate. We set {\tt maxit = 1000} (increasing the maximum
number of iterations to $1000$) because our data is relatively high
dimensional, so more iterations are needed for convergence. In
practice, the function will spit out an error if convergence isn't
reached by the maximum number of iterations.
<<>>=
cv.fit <- cv.glmnet(x, Surv(time, status), family="cox", maxit = 1000)
fit <- glmnet(x, Surv(time,status), family = "cox", maxit = 1000)
@
The {\tt Surv} function packages the survival data into the form
expected by glmnet. Once fit, we can view the optimal $\lambda$ value and a cross
validated error plot to help evaluate our
model.
\begin{center}
\setkeys{Gin}{width = 3 in}
<>=
plot(cv.fit)
@
\end{center}
<<>>=
cv.fit$lambda.min
@
The left vertical line in our plot shows us where the CV-error curve
hits its minimum. The right vertical line shows us the most
regularized model with CV-error within $1$ standard deviation of the
minimum. In this case, we see that the minimum was achieved by a
fairly regularized model, however the right line indicates that the
null model (no coefficients included) is within $1$ sd of the
minimum. This might lead us to believe that in actuality the
covariates are not explaining any variability. For the time being we
will concern ourselves with the minimum CV-error model. We can check which covariates our model chose to be active, and see the coefficients of those covariates.
<<>>=
Coefficients <- coef(fit, s = cv.fit$lambda.min)
Active.Index <- which(Coefficients != 0)
Active.Coefficients <- Coefficients[Active.Index]
@
{\tt coef(fit, s = cv.fit\$lambda.min)} returns the $p$ length coefficient vector of the solution corresponding to $\lambda = ${\tt cv.fit\$lambda.min}.
<<>>=
Active.Index
Active.Coefficients
@
We see that our optimal model chose 2 active covariates ($X80$ and $X394$) each with a small positive effect on hazard.
\end{document}
glmnet/inst/doc/Coxnet.R 0000644 0001762 0000144 00000002526 12672345374 014667 0 ustar ligges users ### R code from vignette source 'Coxnet.rnw'
###################################################
### code chunk number 1: Coxnet.rnw:30-34
###################################################
library("glmnet")
library("survival")
load("VignetteExample.rdata")
attach(patient.data)
###################################################
### code chunk number 2: Coxnet.rnw:44-46
###################################################
cv.fit <- cv.glmnet(x, Surv(time, status), family="cox", maxit = 1000)
fit <- glmnet(x, Surv(time,status), family = "cox", maxit = 1000)
###################################################
### code chunk number 3: Coxnet.rnw:56-57
###################################################
plot(cv.fit)
###################################################
### code chunk number 4: Coxnet.rnw:60-61
###################################################
cv.fit$lambda.min
###################################################
### code chunk number 5: Coxnet.rnw:74-77
###################################################
Coefficients <- coef(fit, s = cv.fit$lambda.min)
Active.Index <- which(Coefficients != 0)
Active.Coefficients <- Coefficients[Active.Index]
###################################################
### code chunk number 6: Coxnet.rnw:82-84
###################################################
Active.Index
Active.Coefficients
glmnet/inst/doc/glmnet_beta.R 0000644 0001762 0000144 00000027637 12672345374 015722 0 ustar ligges users ## ---- eval=FALSE---------------------------------------------------------
# install.packages("glmnet", repos = "http://cran.us.r-project.org")
## ------------------------------------------------------------------------
library(glmnet)
## ------------------------------------------------------------------------
data(QuickStartExample)
## ------------------------------------------------------------------------
fit = glmnet(x, y)
## ------------------------------------------------------------------------
plot(fit)
## ----height = 4----------------------------------------------------------
print(fit)
## ------------------------------------------------------------------------
coef(fit,s=0.1)
## ------------------------------------------------------------------------
nx = matrix(rnorm(10*20),10,20)
predict(fit,newx=nx,s=c(0.1,0.05))
## ------------------------------------------------------------------------
cvfit = cv.glmnet(x, y)
## ------------------------------------------------------------------------
plot(cvfit)
## ------------------------------------------------------------------------
cvfit$lambda.min
## ------------------------------------------------------------------------
coef(cvfit, s = "lambda.min")
## ------------------------------------------------------------------------
predict(cvfit, newx = x[1:5,], s = "lambda.min")
## ------------------------------------------------------------------------
fit = glmnet(x, y, alpha = 0.2, weights = c(rep(1,50),rep(2,50)), nlambda = 20)
## ------------------------------------------------------------------------
print(fit)
## ------------------------------------------------------------------------
plot(fit, xvar = "lambda", label = TRUE)
## ------------------------------------------------------------------------
plot(fit, xvar = "dev", label = TRUE)
## ------------------------------------------------------------------------
any(fit$lambda == 0.5)
coef.exact = coef(fit, s = 0.5, exact = TRUE)
coef.apprx = coef(fit, s = 0.5, exact = FALSE)
cbind2(coef.exact, coef.apprx)
## ------------------------------------------------------------------------
predict(fit, newx = x[1:5,], type = "response", s = 0.05)
## ------------------------------------------------------------------------
cvfit = cv.glmnet(x, y, type.measure = "mse", nfolds = 20)
## ---- eval=FALSE---------------------------------------------------------
# require(doMC)
# registerDoMC(cores=2)
# X = matrix(rnorm(1e4 * 200), 1e4, 200)
# Y = rnorm(1e4)
## ---- eval=FALSE---------------------------------------------------------
# system.time(cv.glmnet(X, Y))
## ---- echo=FALSE---------------------------------------------------------
structure(c(2.44, 0.08, 2.518, 0, 0), class = "proc_time", .Names = c("user.self",
"sys.self", "elapsed", "user.child", "sys.child"))
## ---- eval=FALSE---------------------------------------------------------
# system.time(cv.glmnet(X, Y, parallel = TRUE))
## ---- echo=FALSE---------------------------------------------------------
structure(c(0.508999999999999, 0.057, 1.56699999999999, 1.941,
0.1), class = "proc_time", .Names = c("user.self", "sys.self",
"elapsed", "user.child", "sys.child"))
## ------------------------------------------------------------------------
cvfit$lambda.min
coef(cvfit, s = "lambda.min")
predict(cvfit, newx = x[1:5,], s = "lambda.min")
## ------------------------------------------------------------------------
foldid=sample(1:10,size=length(y),replace=TRUE)
cv1=cv.glmnet(x,y,foldid=foldid,alpha=1)
cv.5=cv.glmnet(x,y,foldid=foldid,alpha=.5)
cv0=cv.glmnet(x,y,foldid=foldid,alpha=0)
## ------------------------------------------------------------------------
par(mfrow=c(2,2))
plot(cv1);plot(cv.5);plot(cv0)
plot(log(cv1$lambda),cv1$cvm,pch=19,col="red",xlab="log(Lambda)",ylab=cv1$name)
points(log(cv.5$lambda),cv.5$cvm,pch=19,col="grey")
points(log(cv0$lambda),cv0$cvm,pch=19,col="blue")
legend("topleft",legend=c("alpha= 1","alpha= .5","alpha 0"),pch=19,col=c("red","grey","blue"))
## ------------------------------------------------------------------------
tfit=glmnet(x,y,lower=-.7,upper=.5)
plot(tfit)
## ------------------------------------------------------------------------
p.fac = rep(1, 20)
p.fac[c(5, 10, 15)] = 0
pfit = glmnet(x, y, penalty.factor = p.fac)
plot(pfit, label = TRUE)
## ------------------------------------------------------------------------
set.seed(101)
x=matrix(rnorm(1000),100,10)
y=rnorm(100)
vn=paste("var",1:10)
fit=glmnet(x,y)
plot(fit)
## ------------------------------------------------------------------------
par(mar=c(4.5,4.5,1,4))
plot(fit)
vnat=coef(fit)
vnat=vnat[-1,ncol(vnat)] # remove the intercept, and get the coefficients at the end of the path
axis(4, at=vnat,line=-.5,label=vn,las=1,tick=FALSE, cex.axis=0.5)
## ------------------------------------------------------------------------
data(MultiGaussianExample)
## ------------------------------------------------------------------------
mfit = glmnet(x, y, family = "mgaussian")
## ------------------------------------------------------------------------
plot(mfit, xvar = "lambda", label = TRUE, type.coef = "2norm")
## ------------------------------------------------------------------------
predict(mfit, newx = x[1:5,], s = c(0.1, 0.01))
## ------------------------------------------------------------------------
cvmfit = cv.glmnet(x, y, family = "mgaussian")
## ------------------------------------------------------------------------
plot(cvmfit)
## ------------------------------------------------------------------------
cvmfit$lambda.min
cvmfit$lambda.1se
## ------------------------------------------------------------------------
data(BinomialExample)
## ------------------------------------------------------------------------
fit = glmnet(x, y, family = "binomial")
## ------------------------------------------------------------------------
plot(fit, xvar = "dev", label = TRUE)
## ------------------------------------------------------------------------
predict(fit, newx = x[1:5,], type = "class", s = c(0.05, 0.01))
## ------------------------------------------------------------------------
cvfit = cv.glmnet(x, y, family = "binomial", type.measure = "class")
## ------------------------------------------------------------------------
plot(cvfit)
## ------------------------------------------------------------------------
cvfit$lambda.min
cvfit$lambda.1se
## ------------------------------------------------------------------------
coef(cvfit, s = "lambda.min")
## ------------------------------------------------------------------------
predict(cvfit, newx = x[1:10,], s = "lambda.min", type = "class")
## ------------------------------------------------------------------------
data(MultinomialExample)
## ------------------------------------------------------------------------
fit = glmnet(x, y, family = "multinomial", type.multinomial = "grouped")
## ------------------------------------------------------------------------
plot(fit, xvar = "lambda", label = TRUE, type.coef = "2norm")
## ------------------------------------------------------------------------
cvfit=cv.glmnet(x, y, family="multinomial", type.multinomial = "grouped", parallel = TRUE)
plot(cvfit)
## ------------------------------------------------------------------------
predict(cvfit, newx = x[1:10,], s = "lambda.min", type = "class")
## ------------------------------------------------------------------------
data(PoissonExample)
## ------------------------------------------------------------------------
fit = glmnet(x, y, family = "poisson")
## ------------------------------------------------------------------------
plot(fit)
## ------------------------------------------------------------------------
coef(fit, s = 1)
predict(fit, newx = x[1:5,], type = "response", s = c(0.1,1))
## ------------------------------------------------------------------------
cvfit = cv.glmnet(x, y, family = "poisson")
## ------------------------------------------------------------------------
plot(cvfit)
## ------------------------------------------------------------------------
opt.lam = c(cvfit$lambda.min, cvfit$lambda.1se)
coef(cvfit, s = opt.lam)
## ------------------------------------------------------------------------
data(CoxExample)
y[1:5,]
## ------------------------------------------------------------------------
fit = glmnet(x, y, family = "cox")
## ------------------------------------------------------------------------
plot(fit)
## ------------------------------------------------------------------------
coef(fit, s = 0.05)
## ------------------------------------------------------------------------
cvfit = cv.glmnet(x, y, family = "cox")
## ------------------------------------------------------------------------
plot(cvfit)
## ------------------------------------------------------------------------
cvfit$lambda.min
cvfit$lambda.1se
## ------------------------------------------------------------------------
coef.min = coef(cvfit, s = "lambda.min")
active.min = which(coef.min != 0)
index.min = coef.min[active.min]
## ------------------------------------------------------------------------
index.min
coef.min
## ------------------------------------------------------------------------
data(SparseExample)
## ------------------------------------------------------------------------
class(x)
## ------------------------------------------------------------------------
fit = glmnet(x, y)
## ------------------------------------------------------------------------
cvfit = cv.glmnet(x, y)
plot(cvfit)
## ------------------------------------------------------------------------
i = sample(1:5, size = 25, replace = TRUE)
j = sample(1:20, size = 25, replace = TRUE)
x = rnorm(25)
nx = sparseMatrix(i = i, j = j, x = x, dims = c(5, 20))
predict(cvfit, newx = nx, s = "lambda.min")
## ------------------------------------------------------------------------
data(QuickStartExample)
fit = glmnet(x, y)
print(fit)
## ------------------------------------------------------------------------
glmnet.control(fdev = 0)
fit = glmnet(x, y)
print(fit)
## ------------------------------------------------------------------------
glmnet.control(factory = TRUE)
## ------------------------------------------------------------------------
glmnet.control()
## ---- echo=FALSE---------------------------------------------------------
data(QuickStartExample)
## ----eval=FALSE----------------------------------------------------------
# fit = glmnet(x, y, intercept = F, standardize = F, lambda = 8/(2*dim(x)[1]), thresh = 1e-20)
## ----eval=FALSE----------------------------------------------------------
# beta_glmnet = as.matrix(predict(fit, type = "coefficients")[-1,])
## ------------------------------------------------------------------------
fit = glmnet(x, y, intercept = F, standardize = F, thresh = 1e-20)
beta_glmnet = as.matrix(predict(fit, s = 8/(2*dim(x)[1]), type = "coefficients", exact = T)[-1,])
## ---- eval=FALSE---------------------------------------------------------
# library(CVXfromR)
# setup.dir = "change/this/to/your/cvx/directory"
# n = dim(x)[1]; p = dim(x)[2]
# cvxcode = paste("variables beta(p)",
# "minimize(square_pos(norm(y - x * beta, 2)) + lambda * norm(beta, 1))",
# sep = ";")
# Lasso = CallCVX(cvxcode, const.var = list(p = p, x = x, y = y, lambda = 8), opt.var.names = "beta", setup.dir = setup.dir, matlab.call = "change/this/to/path/to/matlab")
# beta_CVX = Lasso$beta
## ------------------------------------------------------------------------
data(CVXResults)
## ----message=FALSE-------------------------------------------------------
require(lars)
## ------------------------------------------------------------------------
fit_lars = lars(x, y, type = "lasso", intercept = F, normalize = F)
beta_lars = predict(fit_lars, s = 8/2, type = "coefficients", mode = "lambda")$coefficients
## ------------------------------------------------------------------------
cmp = round(cbind(beta_glmnet, beta_lars, beta_CVX), digits = 6)
colnames(cmp) = c("beta_glmnet", "beta_lars", "beta_CVX")
cmp
glmnet/inst/doc/glmnet_beta.Rmd 0000644 0001762 0000144 00000145504 12672345374 016235 0 ustar ligges users ---
output: html_document
---
# Glmnet Vignette
### Trevor Hastie and Junyang Qian
#### Stanford June 26, 2014
> [Introduction](#intro)
> [Installation](#install)
> [Quick Start](#qs)
> [Linear Regression](#lin)
> [Logistic Regression](#log)
> [Poisson Models](#poi)
> [Cox Models](#cox)
> [Sparse Matrices](#spa)
> [Appendix 1: Internal Parameters](#int)
> [Appendix 2: Comparison with Otherther Packages](#cmp)
## Introduction
Glmnet is a package that fits a generalized linear model via penalized maximum likelihood. The regularization path is computed for the lasso or elasticnet penalty at a grid of values for the regularization parameter lambda. The algorithm is extremely fast, and can exploit sparsity in the input matrix `x`. It fits linear, logistic and multinomial, poisson, and Cox regression models. A variety of predictions can be made from the fitted models. It can also fit multi-response linear regression.
The authors of glmnet are Jerome Friedman, Trevor Hastie, Rob Tibshirani and Noah Simon, and the R package is maintained by Trevor Hastie. The matlab version of glmnet is maintained by Junyang Qian. This vignette describes the usage of glmnet in R.
`glmnet` solves the following problem
$$
\min_{\beta_0,\beta} \frac{1}{N} \sum_{i=1}^{N} w_i l(y_i,\beta_0+\beta^T x_i) + \lambda\left[(1-\alpha)||\beta||_2^2/2 + \alpha ||\beta||_1\right],
$$
over a grid of values of $\lambda$ covering the entire range. Here $l(y,\eta)$ is the negative log-likelihood contribution for observation $i$; e.g. for the Gaussian case it is $\frac{1}{2}(y-\eta)^2$. The _elastic-net_ penalty is controlled by $\alpha$, and bridges the gap between lasso ($\alpha=1$, the default) and ridge ($\alpha=0$). The tuning parameter $\lambda$ controls the overall strength of the penalty.
It is known that the ridge penalty shrinks the coefficients of correlated predictors towards each other while the lasso tends to pick one of them and discard the others. The elastic-net penalty mixes these two; if predictors are correlated in groups, an $\alpha=0.5$ tends to select the groups in or out together. This is a higher level parameter, and users might pick a value upfront, else experiment with a few different values. One use of $\alpha$ is for numerical stability; for example, the elastic net with $\alpha = 1 - \epsilon$ for some small $\epsilon > 0$ performs much like the lasso, but removes any degeneracies and wild behavior caused by extreme correlations.
The `glmnet` algorithms use cyclical coordinate descent, which successively optimizes the objective function over each parameter with others fixed, and cycles repeatedly until convergence. The package also makes use of the strong rules for efficient restriction of the active set. Due to highly efficient updates and techniques such as warm starts and active-set convergence, our algorithms can compute the solution path very fast.
The code can handle sparse input-matrix formats, as well as range constraints on coefficients. The core of `glmnet` is a set of fortran subroutines, which make for very fast execution.
The package also includes methods for prediction and plotting, and a function that performs K-fold cross-validation.
## Installation
Like many other R packages, the simplest way to obtain `glmnet` is to install it directly from CRAN. Type the following command in R console:
```{r, eval=FALSE}
install.packages("glmnet", repos = "http://cran.us.r-project.org")
```
Users may change the `repos` options depending on their locations and preferences. Other options such as the directories where to install the packages can be altered in the command. For more details, see `help(install.packages)`.
Here the R package has been downloaded and installed to the default directories.
Alternatively, users can download the package source at http://cran.r-project.org/package=glmnet and type Unix commands to install it to the desired location.
[Back to Top](#top)
## Quick Start
The purpose of this section is to give users a general sense of the package, including the components, what they do and some basic usage. We will briefly go over the main functions, see the basic operations and have a look at the outputs. Users may have a better idea after this section what functions are available, which one to choose, or at least where to seek help. More details are given in later sections.
First, we load the `glmnet` package:
```{r}
library(glmnet)
```
The default model used in the package is the Guassian linear model or "least squares", which we will demonstrate in this section. We load a set of data created beforehand for illustration. Users can either load their own data or use those saved in the workspace.
```{r}
data(QuickStartExample)
```
The command loads an input matrix `x` and a response vector `y` from this saved R data archive.
We fit the model using the most basic call to `glmnet`.
```{r}
fit = glmnet(x, y)
```
"fit" is an object of class `glmnet` that contains all the relevant information of the fitted model for further use. We do not encourage users to extract the components directly. Instead, various methods are provided for the object such as `plot`, `print`, `coef` and `predict` that enable us to execute those tasks more elegantly.
We can visualize the coefficients by executing the `plot` function:
```{r}
plot(fit)
```
Each curve corresponds to a variable. It shows the path of its coefficient against the $\ell_1$-norm of the whole coefficient vector at as $\lambda$ varies. The axis above indicates the number of nonzero coefficients at the current $\lambda$, which is the effective degrees of freedom (_df_) for the lasso. Users may also wish to annotate the curves; this can be done by setting `label = TRUE` in the plot command.
A summary of the `glmnet` path at each step is displayed if we just enter the object name or use the `print` function:
```{r height = 4}
print(fit)
```
It shows from left to right the number of nonzero coefficients (`Df`), the percent (of null) deviance explained (`%dev`) and the value of $\lambda$ (`Lambda`). Although by default `glmnet` calls for 100 values of `lambda` the program stops early if `%dev% does not change sufficently from one lambda to the next (typically near the end of the path.)
We can obtain the actual coefficients at one or more $\lambda$'s within the range of the sequence:
```{r}
coef(fit,s=0.1)
```
(why `s` and not `lambda`? In case later we want to allow one to specify the model size in other ways.)
Users can also make predictions at specific $\lambda$'s with new input data:
```{r}
nx = matrix(rnorm(10*20),10,20)
predict(fit,newx=nx,s=c(0.1,0.05))
```
The function `glmnet` returns a sequence of models for the users to choose from. In many cases, users may prefer the software to select one of them. Cross-validation is perhaps the simplest and most widely used method for that task.
`cv.glmnet` is the main function to do cross-validation here, along with various supporting methods such as plotting and prediction. We still act on the sample data loaded before.
```{r}
cvfit = cv.glmnet(x, y)
```
`cv.glmnet` returns a `cv.glmnet` object, which is "cvfit" here, a list with all the ingredients of the cross-validation fit. As for `glmnet`, we do not encourage users to extract the components directly except for viewing the selected values of $\lambda$. The package provides well-designed functions for potential tasks.
We can plot the object.
```{r}
plot(cvfit)
```
It includes the cross-validation curve (red dotted line), and upper and lower standard deviation curves along the $\lambda$ sequence (error bars). Two selected $\lambda$'s are indicated by the vertical dotted lines (see below).
We can view the selected $\lambda$'s and the corresponding coefficients. For example,
```{r}
cvfit$lambda.min
```
`lambda.min` is the value of $\lambda$ that gives minimum mean cross-validated error. The other $\lambda$ saved is `lambda.1se`, which gives the most regularized model such that error is within one standard error of the minimum. To use that, we only need to replace `lambda.min` with `lambda.1se` above.
```{r}
coef(cvfit, s = "lambda.min")
```
Note that the coefficients are represented in the sparse matrix format. The reason is that the solutions along the regularization path are often sparse, and hence it is more efficient in time and space to use a sparse format. If you prefer non-sparse format, pipe the output through `as.matrix()`.
Predictions can be made based on the fitted `cv.glmnet` object. Let's see a toy example.
```{r}
predict(cvfit, newx = x[1:5,], s = "lambda.min")
```
`newx` is for the new input matrix and `s`, as before, is the value(s) of $\lambda$ at which predictions are made.
That is the end of `glmnet` 101. With the tools introduced so far, users are able to fit the entire elastic net family, including ridge regression, using squared-error loss. In the package, there are many more options that give users a great deal of flexibility. To learn more, move on to later sections.
[Back to Top](#top)
## Linear Regression
Linear regression here refers to two families of models. One is `gaussian`, the Gaussian family, and the other is `mgaussian`, the multiresponse Gaussian family. We first discuss the ordinary Gaussian and the multiresponse one after that.
### Gaussian Family
`gaussian ` is the default family option in the function `glmnet`. Suppose we have observations $x_i \in \mathbb{R}^p$ and the responses $y_i \in \mathbb{R}, i = 1, \ldots, N$. The objective function for the Gaussian family is
$$
\min_{(\beta_0, \beta) \in \mathbb{R}^{p+1}}\frac{1}{2N} \sum_{i=1}^N (y_i -\beta_0-x_i^T \beta)^2+\lambda \left[ (1-\alpha)||\beta||_2^2/2 + \alpha||\beta||_1\right],
$$
where $\lambda \geq 0$ is a complexity parameter and $0 \leq \alpha \leq 1$ is a compromise between ridge ($\alpha = 0$) and lasso ($\alpha = 1$).
Coordinate descent is applied to solve the problem. Specifically, suppose we have current estimates $\tilde{\beta_0}$ and $\tilde{\beta}_\ell$ $\forall j\in 1,]\ldots,p$. By computing the gradient at $\beta_j = \tilde{\beta}_j$ and simple calculus, the update is
$$
\tilde{\beta}_j \leftarrow \frac{S(\frac{1}{N}\sum_{i=1}^N x_{ij}(y_i-\tilde{y}_i^{(j)}),\lambda \alpha)}{1+\lambda(1-\alpha)},
$$
where $\tilde{y}_i^{(j)} = \tilde{\beta}_0 + \sum_{\ell \neq j} x_{i\ell} \tilde{\beta}_\ell$, and $S(z, \gamma)$ is the soft-thresholding operator with value $\text{sign}(z)(|z|-\gamma)_+$.
This formula above applies when the `x` variables are standardized to have unit variance (the default); it is slightly more complicated when they are not. Note that for "family=gaussian", `glmnet` standardizes $y$ to have unit variance before computing its lambda sequence (and then unstandardizes the resulting coefficients); if you wish to reproduce/compare results with other software, best to supply a standardized $y$ first (Using the "1/N" variance formula).
`glmnet` provides various options for users to customize the fit. We introduce some commonly used options here and they can be specified in the `glmnet` function.
* `alpha` is for the elastic-net mixing parameter $\alpha$, with range $\alpha \in [0,1]$. $\alpha = 1$ is the lasso (default) and $\alpha = 0$ is the ridge.
* `weights` is for the observation weights. Default is 1 for each observation. (Note: `glmnet` rescales the weights to sum to N, the sample size.)
* `nlambda` is the number of $\lambda$ values in the sequence. Default is 100.
* `lambda` can be provided, but is typically not and the program constructs a sequence. When automatically generated, the $\lambda$ sequence is determined by `lambda.max` and `lambda.min.ratio`. The latter is the ratio of smallest value of the generated $\lambda$ sequence (say `lambda.min`) to `lambda.max`. The program then generated `nlambda` values linear on the log scale from `lambda.max` down to `lambda.min`. `lambda.max` is not given, but easily computed from the input $x$ and $y$; it is the smallest value for `lambda` such that all the coefficients are zero. For `alpha=0` (ridge) `lambda.max` would be $\infty$; hence for this case we pick a value corresponding to a small value for `alpha` close to zero.)
* `standardize` is a logical flag for `x` variable standardization, prior to fitting the model sequence. The coefficients are always returned on the original scale. Default is `standardize=TRUE`.
For more information, type `help(glmnet)` or simply `?glmnet`.
As an example, we set $\alpha = 0.2$ (more like a ridge regression), and give double weights to the latter half of the observations. To avoid too long a display here, we set `nlambda` to 20. In practice, however, the number of values of $\lambda$ is recommended to be 100 (default) or more. In most cases, it does not come with extra cost because of the warm-starts used in the algorithm, and for nonlinear models leads to better convergence properties.
```{r}
fit = glmnet(x, y, alpha = 0.2, weights = c(rep(1,50),rep(2,50)), nlambda = 20)
```
We can then print the `glmnet` object.
```{r}
print(fit)
```
This displays the call that produced the object `fit` and a three-column matrix with columns `Df` (the number of nonzero coefficients), `%dev` (the percent deviance explained) and `Lambda` (the corresponding value of $\lambda$).
(Note that the `digits` option can used to specify significant digits in the printout.)
Here the actual number of $\lambda$'s here is less than specified in the call. The reason lies in the stopping criteria of the algorithm. According to the default internal settings, the computations stop if either the fractional change in deviance down the path is less than $10^{-5}$ or the fraction of explained deviance reaches $0.999$. From the last few lines , we see the fraction of deviance does not change much and therefore the computation ends when meeting the stopping criteria. We can change such internal parameters. For details, see the Appendix section or type `help(glmnet.control)`.
We can plot the fitted object as in the previous section. There are more options in the `plot` function.
Users can decide what is on the X-axis. `xvar` allows three measures: "norm" for the $\ell_1$-norm of the coefficients (default), "lambda" for the log-lambda value and "dev" for %deviance explained.
Users can also label the curves with variable sequence numbers simply by setting `label = TRUE`.
Let's plot "fit" against the log-lambda value and with each curve labeled.
```{r}
plot(fit, xvar = "lambda", label = TRUE)
```
Now when we plot against %deviance we get a very different picture. This is percent deviance explained on the training data. What we see here is that toward the end of the path this value are not changing much, but the coefficients are "blowing up" a bit. This lets us focus attention on the parts of the fit that matter. This will especially be true for other models, such as logistic regression.
```{r}
plot(fit, xvar = "dev", label = TRUE)
```
We can extract the coefficients and make predictions at certain values of $\lambda$. Two commonly used options are:
* `s` specifies the value(s) of $\lambda$ at which extraction is made.
* `exact` indicates whether the exact values of coefficients are desired or not. That is, if `exact = TRUE`, and predictions are to be made at values of s not included in the original fit, these values of s are merged with `object$lambda`, and the model is refit before predictions are made. If `exact=FALSE` (default), then the predict function uses linear interpolation to make predictions for values of s that do not coincide with lambdas used in the fitting algorithm.
A simple example is:
```{r}
any(fit$lambda == 0.5)
coef.exact = coef(fit, s = 0.5, exact = TRUE)
coef.apprx = coef(fit, s = 0.5, exact = FALSE)
cbind2(coef.exact, coef.apprx)
```
The left column is for `exact = TRUE` and the right for `FALSE`. We see from the above that 0.01 is not in the sequence and that hence there are some difference, though not much. Linear interpolation is mostly enough if there are no special requirements.
Users can make predictions from the fitted object. In addition to the options in `coef`, the primary argument is `newx`, a matrix of new values for `x`. The `type` option allows users to choose the type of prediction:
* "link" gives the fitted values
* "response" the sames as "link" for "gaussian" family.
* "coefficients" computes the coefficients at values of `s`
* "nonzero" retuns a list of the indices of the nonzero coefficients for each value of `s`.
For example,
```{r}
predict(fit, newx = x[1:5,], type = "response", s = 0.05)
```
gives the fitted values for the first 5 observations at $\lambda = 0.05$. If multiple values of `s` are supplied, a matrix of predictions is produced.
Users can customize K-fold cross-validation. In addition to all the `glmnet` parameters, `cv.glmnet` has its special parameters including `nfolds` (the number of folds), `foldid` (user-supplied folds), `type.measure`(the loss used for cross-validation):
* "deviance" or "mse" uses squared loss
* "mae" uses mean absolute error
As an example,
```{r}
cvfit = cv.glmnet(x, y, type.measure = "mse", nfolds = 20)
```
does 20-fold cross-validation, based on mean squared error criterion (default though).
Parallel computing is also supported by `cv.glmnet`. To make it work, users must register parallel beforehand. We give a simple example of comparison here. Unfortunately, the package `doMC` is not available on Windows platforms (it is on others), so we cannot run the code here, but we make it looks as if we have.
```{r, eval=FALSE}
require(doMC)
registerDoMC(cores=2)
X = matrix(rnorm(1e4 * 200), 1e4, 200)
Y = rnorm(1e4)
```
```{r, eval=FALSE}
system.time(cv.glmnet(X, Y))
```
```{r, echo=FALSE}
structure(c(2.44, 0.08, 2.518, 0, 0), class = "proc_time", .Names = c("user.self",
"sys.self", "elapsed", "user.child", "sys.child"))
```
```{r, eval=FALSE}
system.time(cv.glmnet(X, Y, parallel = TRUE))
```
```{r, echo=FALSE}
structure(c(0.508999999999999, 0.057, 1.56699999999999, 1.941,
0.1), class = "proc_time", .Names = c("user.self", "sys.self",
"elapsed", "user.child", "sys.child"))
```
As suggested from the above, parallel computing can significantly speed up the computation process especially for large-scale problems.
Functions `coef` and `predict` on cv.glmnet object are similar to those for a `glmnet` object, except that two special strings are also supported by `s` (the values of $\lambda$ requested):
* "lambda.1se": the largest $\lambda$ at which the MSE is within one standard error of the minimal MSE.
* "lambda.min": the $\lambda$ at which the minimal MSE is achieved.
```{r}
cvfit$lambda.min
coef(cvfit, s = "lambda.min")
predict(cvfit, newx = x[1:5,], s = "lambda.min")
```
Users can control the folds used. Here we use the same folds so we can also select a value for $\alpha$.
```{r}
foldid=sample(1:10,size=length(y),replace=TRUE)
cv1=cv.glmnet(x,y,foldid=foldid,alpha=1)
cv.5=cv.glmnet(x,y,foldid=foldid,alpha=.5)
cv0=cv.glmnet(x,y,foldid=foldid,alpha=0)
```
There are no built-in plot functions to put them all on the same plot, so we are on our own here:
```{r}
par(mfrow=c(2,2))
plot(cv1);plot(cv.5);plot(cv0)
plot(log(cv1$lambda),cv1$cvm,pch=19,col="red",xlab="log(Lambda)",ylab=cv1$name)
points(log(cv.5$lambda),cv.5$cvm,pch=19,col="grey")
points(log(cv0$lambda),cv0$cvm,pch=19,col="blue")
legend("topleft",legend=c("alpha= 1","alpha= .5","alpha 0"),pch=19,col=c("red","grey","blue"))
```
We see that lasso (`alpha=1`) does about the best here. We also see that the range of lambdas used differs with alpha.
#### Coefficient upper and lower bounds
These are recently added features that enhance the scope of the models. Suppose we want to fit our model, but limit the coefficients to be bigger than -0.7 and less than 0.5. This is easily achieved via the `upper.limits` and `lower.limits` arguments:
```{r}
tfit=glmnet(x,y,lower=-.7,upper=.5)
plot(tfit)
```
These are rather arbitrary limits; often we want the coefficients to be positive, so we can set only `lower.limit` to be 0.
(Note, the lower limit must be no bigger than zero, and the upper limit no smaller than zero.)
These bounds can be a vector, with different values for each coefficient. If given as a scalar, the same number gets recycled for all.
#### Penalty factors
This argument allows users to apply separate penalty factors to each coefficient. Its default is 1 for each parameter, but other values can be specified. In particular, any variable with `penalty.factor` equal to zero is not penalized at all! Let $v_j$ denote the penalty factor for $j$ th variable. The penalty term becomes
$$
\lambda \sum_{j=1}^p \boldsymbol{v_j} P_\alpha(\beta_j) = \lambda \sum_{j=1}^p \boldsymbol{v_j} \left[ (1-\alpha)\frac{1}{2} \beta_j^2 + \alpha |\beta_j| \right].
$$
Note the penalty factors are internally rescaled to sum to nvars.
This is very useful when people have prior knowledge or preference over the variables. In many cases, some variables may be so important that one wants to keep them all the time, which can be achieved by setting corresponding penalty factors to 0:
```{r}
p.fac = rep(1, 20)
p.fac[c(5, 10, 15)] = 0
pfit = glmnet(x, y, penalty.factor = p.fac)
plot(pfit, label = TRUE)
```
We see from the labels that the three variables with 0 penalty factors always stay in the model, while the others follow typical regularization paths and shrunken to 0 eventually.
Some other useful arguments. `exclude` allows one to block certain variables from being the model at all. Of course, one could simply subset these out of `x`, but sometimes `exclude` is more useful, since it returns a full vector of coefficients, just with the excluded ones set to zero. There is also an `intercept` argument which defaults to `TRUE`; if `FALSE` the intercept is forced to be zero.
#### Customizing plots
Sometimes, especially when the number of variables is small, we want to add variable labels to a plot. Since `glmnet` is intended primarily for wide data, this is not supprted in `plot.glmnet`. However, it is easy to do, as the following little toy example shows.
We first generate some data, with 10 variables, and for lack of imagination and ease we give them simple character names.
We then fit a glmnet model, and make the standard plot.
```{r}
set.seed(101)
x=matrix(rnorm(1000),100,10)
y=rnorm(100)
vn=paste("var",1:10)
fit=glmnet(x,y)
plot(fit)
```
We wish to label the curves with the variable names. Here s a simple way to do this, using the `axis` command in R (and a little research into how to customize it). We need to have the positions of the coefficients at the end of the path, and we need to make some space using the `par` command, so that our labels will fit in.
This requires knowing how long your labels are, but here they are all quite short.
```{r}
par(mar=c(4.5,4.5,1,4))
plot(fit)
vnat=coef(fit)
vnat=vnat[-1,ncol(vnat)] # remove the intercept, and get the coefficients at the end of the path
axis(4, at=vnat,line=-.5,label=vn,las=1,tick=FALSE, cex.axis=0.5)
```
We have done nothing here to avoid overwriting of labels, in the event that they are close together. This would be a bit more work, but perhaps best left alone, anyway.
### Multiresponse Gaussian Family
The multiresponse Gaussian family is obtained using `family = "mgaussian"` option in `glmnet`. It is very similar to the single-response case above. This is useful when there are a number of (correlated) responses - the so-called "multi-task learning" problem. Here the sharing involves which variables are selected, since when a variable is selected, a coefficient is fit for each response. Most of the options are the same, so we focus here on the differences with the single response model.
Obviously, as the name suggests, $y$ is not a vector, but a matrix of quantitative responses in this section. The coefficients at each value of lambda are also a matrix as a result.
Here we solve the following problem:
$$
\min_{(\beta_0, \beta) \in \mathbb{R}^{(p+1)\times K}}\frac{1}{2N} \sum_{i=1}^N ||y_i -\beta_0-\beta^T x_i||^2_F+\lambda \left[ (1-\alpha)||\beta||_F^2/2 + \alpha\sum_{j=1}^p||\beta_j||_2\right].
$$
Here $\beta_j$ is the jth row of the $p\times K$ coefficient matrix $\beta$, and we replace the absolute penalty on each single coefficient by a group-lasso penalty on each coefficient K-vector $\beta_j$ for a single predictor $x_j$.
We use a set of data generated beforehand for illustration.
```{r}
data(MultiGaussianExample)
```
We fit the data, with an object "mfit" returned.
```{r}
mfit = glmnet(x, y, family = "mgaussian")
```
For multiresponse Gaussian, the options in `glmnet` are almost the same as the single-response case, such as `alpha`, `weights`, `nlambda`, `standardize`. A exception to be noticed is that `standardize.response` is only for `mgaussian` family. The default value is `FALSE`. If `standardize.response = TRUE`, it standardizes the response variables.
To visualize the coefficients, we use the `plot` function.
```{r}
plot(mfit, xvar = "lambda", label = TRUE, type.coef = "2norm")
```
Note that we set `type.coef = "2norm"`. Under this setting, a single curve is plotted per variable, with value equal to the $\ell_2$ norm. The default setting is `type.coef = "coef"`, where a coefficient plot is created for each response (multiple figures).
`xvar` and `label` are two other options besides ordinary graphical parameters. They are the same as the single-response case.
We can extract the coefficients at requested values of $\lambda$ by using the function `coef` and make predictions by `predict`. The usage is similar and we only provide an example of `predict` here.
```{r}
predict(mfit, newx = x[1:5,], s = c(0.1, 0.01))
```
The prediction result is saved in a three-dimensional array with the first two dimensions being the prediction matrix for each response variable and the third indicating the response variables.
We can also do k-fold cross-validation. The options are almost the same as the ordinary Gaussian family and we do not expand here.
```{r}
cvmfit = cv.glmnet(x, y, family = "mgaussian")
```
We plot the resulting `cv.glmnet` object "cvmfit".
```{r}
plot(cvmfit)
```
To show explicitly the selected optimal values of $\lambda$, type
```{r}
cvmfit$lambda.min
cvmfit$lambda.1se
```
As before, the first one is the value at which the minimal mean squared error is achieved and the second is for the most regularized model whose mean squared error is within one standard error of the minimal.
Prediction for `cv.glmnet` object works almost the same as for `glmnet` object. We omit the details here.
[Back to Top](#top)
## Logistic Regression
Logistic regression is another widely-used model when the response is categorical. If there are two possible outcomes, we use the binomial distribution, else we use the multinomial.
### Binomial Models
For the binomial model, suppose the response variable takes value in $\mathcal{G}=\{1,2\}$. Denote $y_i = I(g_i=1)$. We model
$$\mbox{Pr}(G=2|X=x)+\frac{e^{\beta_0+\beta^Tx}}{1+e^{\beta_0+\beta^Tx}},$$
which can be written in the following form
$$\log\frac{\mbox{Pr}(G=2|X=x)}{\mbox{Pr}(G=1|X=x)}=\beta_0+\beta^Tx,$$
the so-called "logistic" or log-odds transformation.
The objective function for the penalized logistic regression uses the negative binomial log-likelihood, and is
$$
\min_{(\beta_0, \beta) \in \mathbb{R}^{p+1}} -\left[\frac{1}{N} \sum_{i=1}^N y_i \cdot (\beta_0 + x_i^T \beta) - \log (1+e^{(\beta_0+x_i^T \beta)})\right] + \lambda \big[ (1-\alpha)||\beta||_2^2/2 + \alpha||\beta||_1\big].
$$
Logistic regression is often plagued with degeneracies when $p > N$ and exhibits wild behavior even when $N$ is close to $p$;
the elastic-net penalty alleviates these issues, and regularizes and selects variables as well.
Our algorithm uses a quadratic approximation to the log-likelihood, and then coordinate descent on the resulting penalized weighted least-squares problem. These constitute an outer and inner loop.
For illustration purpose, we load pre-generated input matrix `x` and the response vector `y` from the data file.
```{r}
data(BinomialExample)
```
The input matrix $x$ is the same as other families. For binomial logistic regression, the response variable $y$ should be either a factor with two levels, or a two-column matrix of counts or proportions.
Other optional arguments of `glmnet` for binomial regression are almost same as those for Gaussian family. Don't forget to set `family` option to "binomial".
```{r}
fit = glmnet(x, y, family = "binomial")
```
Like before, we can print and plot the fitted object, extract the coefficients at specific $\lambda$'s and also make predictions. For plotting, the optional arguments such as `xvar` and `label` are similar to the Gaussian. We plot against the deviance explained and show the labels.
```{r}
plot(fit, xvar = "dev", label = TRUE)
```
Prediction is a little different for logistic from Gaussian, mainly in the option `type`. "link" and "response" are never equivalent and "class" is only available for logistic regression. In summary,
* "link" gives the linear predictors
* "response" gives the fitted probabilities
* "class" produces the class label corresponding to the maximum probability.
* "coefficients" computes the coefficients at values of `s`
* "nonzero" retuns a list of the indices of the nonzero coefficients for each value of `s`.
For "binomial" models, results ("link", "response", "coefficients", "nonzero") are returned only for the class corresponding to the second level of the factor response.
In the following example, we make prediction of the class labels at $\lambda = 0.05, 0.01$.
```{r}
predict(fit, newx = x[1:5,], type = "class", s = c(0.05, 0.01))
```
For logistic regression, `cv.glmnet` has similar arguments and usage as Gaussian. `nfolds`, `weights`, `lambda`, `parallel` are all available to users. There are some differences in `type.measure`: "deviance" and "mse" do not both mean squared loss and "class" is enabled. Hence,
* "mse" uses squared loss.
* "deviance" uses actual deviance.
* "mae" uses mean absolute error.
* "class" gives misclassification error.
* "auc" (for two-class logistic regression ONLY) gives area under the ROC curve.
For example,
```{r}
cvfit = cv.glmnet(x, y, family = "binomial", type.measure = "class")
```
It uses misclassification error as the criterion for 10-fold cross-validation.
We plot the object and show the optimal values of $\lambda$.
```{r}
plot(cvfit)
```
```{r}
cvfit$lambda.min
cvfit$lambda.1se
```
`coef` and `predict` are simliar to the Gaussian case and we omit the details. We review by some examples.
```{r}
coef(cvfit, s = "lambda.min")
```
As mentioned previously, the results returned here are only for the second level of the factor response.
```{r}
predict(cvfit, newx = x[1:10,], s = "lambda.min", type = "class")
```
Like other GLMs, glmnet allows for an "offset". This is a fixed vector of N numbers that is added into the linear predictor.
For example, you may have fitted some other logistic regression using other variables (and data), and now you want to see if the present variables can add anything. So you use the predicted logit from the other model as an offset in.
### Multinomial Models
For the multinomial model, suppose the response variable has $K$ levels ${\cal G}=\{1,2,\ldots,K\}$. Here we model
$$\mbox{Pr}(G=k|X=x)=\frac{e^{\beta_{0k}+\beta_k^Tx}}{\sum_{\ell=1}^Ke^{\beta_{0\ell}+\beta_\ell^Tx}}.$$
Let ${Y}$ be the $N \times K$ indicator response matrix, with elements $y_{i\ell} = I(g_i=\ell)$. Then the elastic-net penalized negative log-likelihood function becomes
$$
\ell(\{\beta_{0k},\beta_{k}\}_1^K) = -\left[\frac{1}{N} \sum_{i=1}^N \Big(\sum_{k=1}^Ky_{il} (\beta_{0k} + x_i^T \beta_k)- \log \big(\sum_{k=1}^K e^{\beta_{0k}+x_i^T \beta_k}\big)\Big)\right] +\lambda \left[ (1-\alpha)||\beta||_F^2/2 + \alpha\sum_{j=1}^p||\beta_j||_q\right].
$$
Here we really abuse notation! $\beta$ is a $p\times K$ matrix of coefficients. $\beta_k$ refers to the kth column (for outcome category k), and $\beta_j$ the jth row (vector of K coefficients for variable j).
The last penalty term is $||\beta_j||_q$, we have two options for q: $q\in \{1,2\}$.
When q=1, this is a lasso penalty on each of the parameters. When q=2, this is a grouped-lasso penalty on all the K coefficients for a particular variables, which makes them all be zero or nonzero together.
The standard Newton algorithm can be tedious here. Instead, we use a so-called partial Newton algorithm by making a partial quadratic approximation to the log-likelihood, allowing only $(\beta_{0k}, \beta_k)$ to vary for a single class at a time.
For each value of $\lambda$, we first cycle over all classes indexed by $k$, computing each time a partial quadratic approximation about the parameters of the current class. Then the inner procedure is almost the same as for the binomial case.
This is the case for lasso (q=1). When q=2, we use a different approach, which we wont dwell on here.
For the multinomial case, the usage is similar to logistic regression, and we mainly illustrate by examples and address any differences. We load a set of generated data.
```{r}
data(MultinomialExample)
```
The optional arguments in `glmnet` for multinomial logistic regression are mostly similar to binomial regression except for a few cases.
The response variable can be a `nc >= 2` level factor, or a `nc`-column matrix of counts or proportions.
Internally glmnet will make the rows of this matrix sum to 1, and absorb the total mass into the weight for that observation.
`offset` should be a `nobs x nc` matrix if there is one.
A special option for multinomial regression is `type.multinomial`, which allows the usage of a grouped lasso penalty if `type.multinomial = "grouped"`. This will ensure that the multinomial coefficients for a variable are all in or out together, just like for the multi-response Gaussian.
```{r}
fit = glmnet(x, y, family = "multinomial", type.multinomial = "grouped")
```
We plot the resulting object "fit".
```{r}
plot(fit, xvar = "lambda", label = TRUE, type.coef = "2norm")
```
The options are `xvar`, `label` and `type.coef`, in addition to other ordinary graphical parameters.
`xvar` and `label` are the same as other families while `type.coef` is only for multinomial regression and multiresponse Gaussian model. It can produce a figure of coefficients for each response variable if `type.coef = "coef"` or a figure showing the $\ell_2$-norm in one figure if `type.coef = "2norm"`
We can also do cross-validation and plot the returned object.
```{r}
cvfit=cv.glmnet(x, y, family="multinomial", type.multinomial = "grouped", parallel = TRUE)
plot(cvfit)
```
Note that although `type.multinomial` is not a typical argument in `cv.glmnet`, in fact any argument that can be passed to `glmnet` is valid in the argument list of `cv.glmnet`. We also use parallel computing to accelerate the calculation.
Users may wish to predict at the optimally selected $\lambda$:
```{r}
predict(cvfit, newx = x[1:10,], s = "lambda.min", type = "class")
```
[Back to Top](#top)
## Poisson Models
Poisson regression is used to model count data under the assumption of Poisson error, or otherwise non-negative data where the mean and variance are proportional. Like the Gaussian and binomial model, the Poisson is a member of the exponential family of distributions. We usually model its positive mean on the log scale: $\log \mu(x) = \beta_0+\beta' x$.
The log-likelihood for observations $\{x_i,y_i\}_1^N$ is given my
$$
l(\beta|X, Y) = \sum_{i=1}^N (y_i (\beta_0+\beta' x_i) - e^{\beta_0+\beta^Tx_i}.
$$
As before, we optimize the penalized log-lielihood:
$$
\min_{\beta_0,\beta} -\frac1N l(\beta|X, Y) + \lambda \left((1-\alpha) \sum_{i=1}^N \beta_i^2/2) +\alpha \sum_{i=1}^N |\beta_i|\right).
$$
Glmnet uses an outer Newton loop, and an inner weighted least-squares loop (as in logistic regression) to optimize this criterion.
First, we load a pre-generated set of Poisson data.
```{r}
data(PoissonExample)
```
We apply the function `glmnet` with the `"poisson"` option.
```{r}
fit = glmnet(x, y, family = "poisson")
```
The optional input arguments of `glmnet` for `"poisson"` family are similar to those for others.
`offset` is a useful argument particularly in Poisson models.
When dealing with rate data in Poisson models, the counts collected are often based on different exposures, such as length of time observed, area and years. A poisson rate $\mu(x)$ is relative to a unit exposure time, so if an observation $y_i$ was exposed for $E_i$ units of time, then the expected count would be $E_i\mu(x)$, and the log mean would be $\log(E_i)+\log(\mu(x)$. In a case like this, we would supply an *offset* $\log(E_i)$ for each observation.
Hence `offset` is a vector of length `nobs` that is included in the linear predictor. Other families can also use options, typically for different reasons.
(Warning: if `offset` is supplied in `glmnet`, offsets must also also be supplied to `predict` to make reasonable predictions.)
Again, we plot the coefficients to have a first sense of the result.
```{r}
plot(fit)
```
Like before, we can extract the coefficients and make predictions at certain $\lambda$'s by using `coef` and `predict` respectively. The optional input arguments are similar to those for other families. In function `predict`, the option `type`, which is the type of prediction required, has its own specialties for Poisson family. That is,
* "link" (default) gives the linear predictors like others
* "response" gives the fitted mean
* "coefficients" computes the coefficients at the requested values for `s`, which can also be realized by `coef` function
* "nonzero" returns a a list of the indices of the nonzero coefficients for each value of `s`.
For example, we can do as follows.
```{r}
coef(fit, s = 1)
predict(fit, newx = x[1:5,], type = "response", s = c(0.1,1))
```
We may also use cross-validation to find the optimal $\lambda$'s and thus make inferences.
```{r}
cvfit = cv.glmnet(x, y, family = "poisson")
```
Options are almost the same as the Gaussian family except that for `type.measure`,
* "deviance" (default) gives the deviance
* "mse" stands for mean squared error
* "mae" is for mean absolute error.
We can plot the `cv.glmnet` object.
```{r}
plot(cvfit)
```
We can also show the optimal $\lambda$'s and the corresponding coefficients.
```{r}
opt.lam = c(cvfit$lambda.min, cvfit$lambda.1se)
coef(cvfit, s = opt.lam)
```
The `predict` method is similar and we do not repeat it here.
[Back to Top](#top)
## Cox Models
The Cox proportional hazards model is commonly used for the study of the relationship beteween predictor variables and survival time. In the usual survival analysis framework, we have data of the form $(y_1, x_1, \delta_1), \ldots, (y_n, x_n, \delta_n)$ where $y_i$, the observed time, is a time of failure if $\delta_i$ is 1 or right-censoring if $\delta_i$ is 0. We also let $t_1 < t_2 < \ldots < t_m$ be the increasing list of unique failure times, and $j(i)$ denote the index of the observation failing at time $t_i$.
The Cox model assumes a semi-parametric form for the hazard
$$
h_i(t) = h_0(t) e^{x_i^T \beta},
$$
where $h_i(t)$ is the hazard for patient $i$ at time $t$, $h_0(t)$ is a shared baseline hazard, and $\beta$ is a fixed, length $p$ vector. In the classic setting $n \geq p$, inference is made via the partial likelihood
$$
L(\beta) = \prod_{i=1}^m \frac{e^{x_{j(i)}^T \beta}}{\sum_{j \in R_i} e^{x_j^T \beta}},
$$
where $R_i$ is the set of indices $j$ with $y_j \geq t_i$ (those at risk at time $t_i$).
Note there is no intercept in the Cox mode (its built into the baseline hazard, and like it, would cancel in the partial likelihood.)
We penalize the negative log of the partial likelihood, just like the other models, with an elastic-net penalty.
We use a pre-generated set of sample data and response. Users can load their own data and follow a similar procedure. In this case $x$ must be an $n\times p$ matrix of covariate values - each row corresponds to a patient and each column a covariate. $y$ is an $n \times 2$ matrix, with a column "time" of failure/censoring times, and "status" a 0/1 indicator, with 1 meaning the time is a failure time, and zero a censoring time.
```{r}
data(CoxExample)
y[1:5,]
```
The `Surv` function in the package `survival` can create such a matrix. Note, however, that the `coxph` and related linear models can handle interval and other fors of censoring, while glmnet can only handle right censoring in its present form.
We apply the `glmnet` function to compute the solution path under default settings.
```{r}
fit = glmnet(x, y, family = "cox")
```
All the standard options are available such as `alpha`, `weights`, `nlambda` and `standardize`. Their usage is similar as in the Gaussian case and we omit the details here. Users can also refer to the help file `help(glmnet)`.
We can plot the coefficients.
```{r}
plot(fit)
```
As before, we can extract the coefficients at certain values of $\lambda$.
```{r}
coef(fit, s = 0.05)
```
Since the Cox Model is not commonly used for prediction, we do not give an illustrative example on prediction. If needed, users can refer to the help file by typing `help(predict.glmnet)`.
Also, the function `cv.glmnet` can be used to compute $k$-fold cross-validation for the Cox model. The usage is similar to that for other families except for two main differences.
One is that `type.measure` only supports "deviance"(also default), which gives the partial-likelihood.
The other is in the option `grouped`. `grouped = TRUE` obtains the CV partial likelihood for the Kth fold by subtraction; by subtracting the log partial likelihood evaluated on the full dataset from that evaluated on the on the (K-1)/K dataset. This makes more efficient use of risk sets. With `grouped=FALSE` the log partial likelihood is computed only on the Kth fold, which is only reasonable if each fold has a large number of observations.
```{r}
cvfit = cv.glmnet(x, y, family = "cox")
```
Once fit, we can view the optimal $\lambda$ value and a cross validated error plot to help evaluate our model.
```{r}
plot(cvfit)
```
As previously, the left vertical line in our plot shows us where the CV-error curve hits its minimum. The right vertical line shows us the most regularized model with CV-error within 1 standard deviation of the minimum. We also extract such optimal $\lambda$'s.
```{r}
cvfit$lambda.min
cvfit$lambda.1se
```
We can check the active covariates in our model and see their coefficients.
```{r}
coef.min = coef(cvfit, s = "lambda.min")
active.min = which(coef.min != 0)
index.min = coef.min[active.min]
```
```{r}
index.min
coef.min
```
[Back to Top](#top)
## Sparse Matrices
Our package supports sparse input matrices, which allow efficient storage and operations of large matrices but with only a few nonzero entries. It is available for all families except for the `cox` family. The usage of sparse matrices (inherit from class `"sparseMatrix"` as in package `Matrix`) in `glmnet ` is the same as if a regular matrix is provided.
We load a set of sample data created beforehand.
```{r}
data(SparseExample)
```
It loads `x`, a 100*20 sparse input matrix and `y`, the response vector.
```{r}
class(x)
```
Users can create a sparse matrix with the function `sparseMatrix` by providing the locations and values of the nonzero entries. Alternatively, `Matrix` function can also be used to contruct a sparse matrix by setting `sparse = TRUE`, but this defeats the purpose somewhat.
We can fit the model the same way as before.
```{r}
fit = glmnet(x, y)
```
We also do the cross-validation and plot the resulting object.
```{r}
cvfit = cv.glmnet(x, y)
plot(cvfit)
```
The usage of other functions are similar and we do not expand here.
Note that sparse matrices can also be used for `newx`, the new input matrix in the `predict` function. For example,
```{r}
i = sample(1:5, size = 25, replace = TRUE)
j = sample(1:20, size = 25, replace = TRUE)
x = rnorm(25)
nx = sparseMatrix(i = i, j = j, x = x, dims = c(5, 20))
predict(cvfit, newx = nx, s = "lambda.min")
```
[Back to Top](#top)
## Appendix 1: Internal Parameters
Our package has a set of internal parameters which control some aspects of the computation of the path. The *factory default* settings are expected to serve in most cases, and users do not need to make changes unless there are special requirements.
There are several parameters that users can change:
`fdev` - minimum fractional change in deviance for stopping path; factory default = 1.0e-5
`devmax` - maximum fraction of explained deviance for stopping path; factory default = 0.999
* `eps` - minimum value of lambda.min.ratio (see glmnet); factory default= 1.0e-6
* `big` - large floating point number; factory default = 9.9e35. Inf in definition of upper.limit is set to big
* `mnlam` - minimum number of path points (lambda values) allowed; factory default = 5
* `pmin` - minimum null probability for any class; factory default = 1.0e-5
* `exmx` - maximum allowed exponent; factory default = 250.0
* `prec` - convergence threshold for multi-response bounds adjustment solution; factory default = 1.0e-10
* `mxit` - maximum iterations for multiresponse bounds adjustment solution; factory default = 100
* `factory` - If `TRUE`, reset all the parameters to the factory default; default is `FALSE`
We illustrate the usage by an example. Note that any changes made hold for the duration of the R session, or unless they are changed by the user with a subsequent call to `glmnet.control`.
```{r}
data(QuickStartExample)
fit = glmnet(x, y)
print(fit)
```
We can change the minimum fractional change in deviance for stopping path and compare the results.
```{r}
glmnet.control(fdev = 0)
fit = glmnet(x, y)
print(fit)
```
We set `fdev = 0` to continue all along the path, even without much change. The length of the sequence becomes 100, which is the default of `nlambda`.
Users can also reset to the default settings.
```{r}
glmnet.control(factory = TRUE)
```
The current settings are obtained as follows.
```{r}
glmnet.control()
```
[Back to Top](#top)
## Appendix 2: Comparison with Other Packages
Some people may want to use `glmnet` to solve the Lasso or elastic-net problem at a single $\lambda$. We compare here the solution by `glmnet` with other packages (such as CVX), and also as an illustration of parameter settings in this situation.
__Warning__: Though such problems can be solved by `glmnet`, it is __not recommended__ and is not the spirit of the package. `glmnet` fits the __entire__ solution path for Lasso or elastic-net problems efficiently with various techniques such as warm start. Those advantages will disappear if the $\lambda$ sequence is forced to be only one value.
Nevertheless, we still illustrate with a typical example in linear model in the following for the purpose of comparison. Given $X, Y$ and $\lambda_0 > 0$, we want to find $\beta$ such that
$$
\min_{\beta} ||Y - X\beta||_2^2 + \lambda_0 ||\beta||_1,
$$
where, say, $\lambda_0 = 8$.
We first solve using `glmnet`. Notice that there is no intercept term in the objective function, and the columns of $X$ are not necessarily standardized. Corresponding parameters have to be set to make it work correctly. In addition, there is a $1/(2n)$ factor before the quadratic term by default, we need to adjust $\lambda$ accordingly. For the purpose of comparison, the `thresh` option is specified to be 1e-20. However, this is not necessary in many practical applications.
```{r, echo=FALSE}
data(QuickStartExample)
```
```{r,eval=FALSE}
fit = glmnet(x, y, intercept = F, standardize = F, lambda = 8/(2*dim(x)[1]), thresh = 1e-20)
```
We then extract the coefficients (with no intercept).
```{r,eval=FALSE}
beta_glmnet = as.matrix(predict(fit, type = "coefficients")[-1,])
```
In linear model as here this approach worked because we were using squared error loss, but with any nonlinear family, it will probably fail. The reason is we are not using step length optimization, and so rely on very good warm starts to put us in the quadratic region of the loss function.
Alternatively, a more stable and __strongly recommended__ way to perform this task is to first fit the entire Lasso or elastic-net path without specifying `lambda`, but then provide the requested $\lambda_0$ to `predict` function to extract the corresponding coefficients. In fact, if $\lambda_0$ is not in the $\lambda$ sequence generated by `glmnet`, the path will be refitted along a new $\lambda$ sequence that includes the requested value $\lambda_0$ and the old sequence, and the coefficients will be returned at $\lambda_0$ based on the new fit. Remember to set `exact = TRUE` in `predict` function to get the exact solution. Otherwise, it will be approximated by linear interpolation.
```{r}
fit = glmnet(x, y, intercept = F, standardize = F, thresh = 1e-20)
beta_glmnet = as.matrix(predict(fit, s = 8/(2*dim(x)[1]), type = "coefficients", exact = T)[-1,])
```
We also use CVX, a general convex optimization solver, to solve this specific Lasso problem. Users could also call CVX from R using the `CVXfromR` package and solve the problem as follows.
```{r, eval=FALSE}
library(CVXfromR)
setup.dir = "change/this/to/your/cvx/directory"
n = dim(x)[1]; p = dim(x)[2]
cvxcode = paste("variables beta(p)",
"minimize(square_pos(norm(y - x * beta, 2)) + lambda * norm(beta, 1))",
sep = ";")
Lasso = CallCVX(cvxcode, const.var = list(p = p, x = x, y = y, lambda = 8), opt.var.names = "beta", setup.dir = setup.dir, matlab.call = "change/this/to/path/to/matlab")
beta_CVX = Lasso$beta
```
For convenience here, the results were saved in `CVXResult.RData`, and we simply load in the results.
```{r}
data(CVXResults)
```
In addition, we use `lars` to solve the same problem.
```{r,message=FALSE}
require(lars)
```
```{r}
fit_lars = lars(x, y, type = "lasso", intercept = F, normalize = F)
beta_lars = predict(fit_lars, s = 8/2, type = "coefficients", mode = "lambda")$coefficients
```
The results are listed below up to 6 decimal digits (due to convergence thresholds).
```{r}
cmp = round(cbind(beta_glmnet, beta_lars, beta_CVX), digits = 6)
colnames(cmp) = c("beta_glmnet", "beta_lars", "beta_CVX")
cmp
```
[Back to Top](#top)
## References
Jerome Friedman, Trevor Hastie and Rob Tibshirani. (2008).
Regularization Paths for Generalized Linear Models via Coordinate Descent
Journal of Statistical Software, Vol. 33(1), 1-22 Feb 2010.
Noah Simon, Jerome Friedman, Trevor Hastie and Rob Tibshirani. (2011).
Regularization Paths for Cox's Proportional Hazards Model via Coordinate Descent
Journal of Statistical Software, Vol. 39(5) 1-13.
Robert Tibshirani, Jacob Bien, Jerome Friedman, Trevor Hastie, Noah Simon, Jonathan Taylor, Ryan J. Tibshirani. (2010).
Strong Rules for Discarding Predictors in Lasso-type Problems
Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(2), 245-266.
Noah Simon, Jerome Friedman and Trevor Hastie (2013).
A Blockwise Descent Algorithm for Group-penalized Multiresponse and Multinomial Regression
(in arXiv, submitted)
glmnet/inst/mortran/ 0000755 0001762 0000144 00000000000 12340006760 014175 5 ustar ligges users glmnet/inst/mortran/glmnet5.m 0000644 0001762 0000144 00000441535 12340006661 015742 0 ustar ligges users "
c
c newGLMnet (5/12/14)
c
c
c Elastic net with squared-error loss
c
c dense predictor matrix:
c
c call elnet(ka,parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,
c intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr)
c
c x(no,ni) = predictor data matrix flat file (overwritten)
c
c
c sparse predictor matrix:
c
c call spelnet(ka,parm,no,ni,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
c isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr)
c
c x, ix, jx = predictor data matrix in compressed sparse row format
c
c
c other inputs:
c
c ka = algorithm flag
c ka=1 => covariance updating algorithm
c ka=2 => naive algorithm
c parm = penalty member index (0 <= parm <= 1)
c = 0.0 => ridge
c = 1.0 => lasso
c no = number of observations
c ni = number of predictor variables
c y(no) = response vector (overwritten)
c w(no)= observation weights (overwritten)
c jd(jd(1)+1) = predictor variable deletion flag
c jd(1) = 0 => use all variables
c jd(1) != 0 => do not use variables jd(2)...jd(jd(1)+1)
c vp(ni) = relative penalties for each predictor variable
c vp(j) = 0 => jth variable unpenalized
c cl(2,ni) = interval constraints on coefficient values (overwritten)
c cl(1,j) = lower bound for jth coefficient value (<= 0.0)
c cl(2,j) = upper bound for jth coefficient value (>= 0.0)
c ne = maximum number of variables allowed to enter largest model
c (stopping criterion)
c nx = maximum number of variables allowed to enter all models
c along path (memory allocation, nx > ne).
c nlam = (maximum) number of lamda values
c flmin = user control of lamda values (>=0)
c flmin < 1.0 => minimum lamda = flmin*(largest lamda value)
c flmin >= 1.0 => use supplied lamda values (see below)
c ulam(nlam) = user supplied lamda values (ignored if flmin < 1.0)
c thr = convergence threshold for each lamda solution.
c iterations stop when the maximum reduction in the criterion value
c as a result of each parameter update over a single pass
c is less than thr times the null criterion value.
c (suggested value, thr=1.0e-5)
c isd = predictor variable standarization flag:
c isd = 0 => regression on original predictor variables
c isd = 1 => regression on standardized predictor variables
c Note: output solutions always reference original
c variables locations and scales.
c intr = intercept flag
c intr = 0/1 => don't/do include intercept in model
c maxit = maximum allowed number of passes over the data for all lambda
c values (suggested values, maxit = 100000)
c
c output:
c
c lmu = actual number of lamda values (solutions)
c a0(lmu) = intercept values for each solution
c ca(nx,lmu) = compressed coefficient values for each solution
c ia(nx) = pointers to compressed coefficients
c nin(lmu) = number of compressed coefficients for each solution
c rsq(lmu) = R**2 values for each solution
c alm(lmu) = lamda values corresponding to each solution
c nlp = actual number of passes over the data for all lamda values
c jerr = error flag:
c jerr = 0 => no error
c jerr > 0 => fatal error - no output returned
c jerr < 7777 => memory allocation error
c jerr = 7777 => all used predictors have zero variance
c jerr = 10000 => maxval(vp) <= 0.0
C jerr < 0 => non fatal error - partial output:
c Solutions for larger lamdas (1:(k-1)) returned.
c jerr = -k => convergence for kth lamda value not reached
c after maxit (see above) iterations.
c jerr = -10000-k => number of non zero coefficients along path
c exceeds nx (see above) at kth lamda value.
c
c
c
c least-squares utility routines:
c
c
c uncompress coefficient vectors for all solutions:
c
c call solns(ni,nx,lmu,ca,ia,nin,b)
c
c input:
c
c ni,nx = input to elnet
c lmu,ca,ia,nin = output from elnet
c
c output:
c
c b(ni,lmu) = all elnet returned solutions in uncompressed format
c
c
c uncompress coefficient vector for particular solution:
c
c call uncomp(ni,ca,ia,nin,a)
c
c input:
c
c ni = total number of predictor variables
c ca(nx) = compressed coefficient values for the solution
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients for the solution
c
c output:
c
c a(ni) = uncompressed coefficient vector
c referencing original variables
c
c
c evaluate linear model from compressed coefficients and
c uncompressed predictor matrix:
c
c call modval(a0,ca,ia,nin,n,x,f);
c
c input:
c
c a0 = intercept
c ca(nx) = compressed coefficient values for a solution
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients for solution
c n = number of predictor vectors (observations)
c x(n,ni) = full (uncompressed) predictor matrix
c
c output:
c
c f(n) = model predictions
c
c
c evaluate linear model from compressed coefficients and
c compressed predictor matrix:
c
c call cmodval(a0,ca,ia,nin,x,ix,jx,n,f);
c
c input:
c
c a0 = intercept
c ca(nx) = compressed coefficient values for a solution
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients for solution
c x, ix, jx = predictor matrix in compressed sparse row format
c n = number of predictor vectors (observations)
c
c output:
c
c f(n) = model predictions
c
c
c
c
c Multiple response
c elastic net with squared-error loss
c
c dense predictor matrix:
c
c call multelnet(parm,no,ni,nr,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,
c jsd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr)
c
c x(no,ni) = predictor data matrix flat file (overwritten)
c
c
c sparse predictor matrix:
c
c call multspelnet(parm,no,ni,nr,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
c isd,jsd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr)
c
c x, ix, jx = predictor data matrix in compressed sparse row format
c
c other inputs:
c
c nr = number of response variables
c y(no,nr) = response data matrix (overwritten)
c jsd = response variable standardization flag
c jsd = 0 => regression using original response variables
c jsd = 1 => regression using standardized response variables
c Note: output solutions always reference original
c variables locations and scales.
c all other inputs same as elnet/spelnet above
c
c output:
c
c a0(nr,lmu) = intercept values for each solution
c ca(nx,nr,lmu) = compressed coefficient values for each solution
c all other outputs same as elnet/spelnet above
c (jerr = 90000 => bounds adjustment non convergence)
c
c
c
c multiple response least-squares utility routines:
c
c
c uncompress coefficient matrix for all solutions:
c
c call multsolns(ni,nx,nr,lmu,ca,ia,nin,b)
c
c input:
c
c ni,nx,nr = input to multelnet
c lmu,ca,ia,nin = output from multelnet
c
c output:
c
c b(ni,nr,lmu) = all multelnet returned solutions in uncompressed format
c
c
c uncompress coefficient matrix for particular solution:
c
c call multuncomp(ni,nr,nx,ca,ia,nin,a)
c
c input:
c
c ni,nr,nx = input to multelnet
c ca(nx,nr) = compressed coefficient values for the solution
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients for the solution
c
c output:
c
c a(ni,nr) = uncompressed coefficient matrix
c referencing original variables
c
c
c evaluate linear model from compressed coefficients and
c uncompressed predictor matrix:
c
c call multmodval(nx,nr,a0,ca,ia,nin,n,x,f);
c
c input:
c
c nx,nr = input to multelnet
c a0(nr) = intercepts
c ca(nx,nr) = compressed coefficient values for a solution
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients for solution
c n = number of predictor vectors (observations)
c x(n,ni) = full (uncompressed) predictor matrix
c
c output:
c
c f(nr,n) = model predictions
c
c
c evaluate linear model from compressed coefficients and
c compressed predictor matrix:
c
c call multcmodval(nx,nr,a0,ca,ia,nin,x,ix,jx,n,f);
c
c input:
c
c nx,nr = input to multelnet
c a0(nr) = intercepts
c ca(nx,nr) = compressed coefficient values for a solution
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients for solution
c x, ix, jx = predictor matrix in compressed sparse row format
c n = number of predictor vectors (observations)
c
c output:
c
c f(nr,n) = model predictions
c
c
c
c
c Symmetric binomial/multinomial logistic elastic net
c
c
c dense predictor matrix:
c
c call lognet (parm,no,ni,nc,x,y,o,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,
c intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,fdev,alm,nlp,jerr)
c
c x(no,ni) = predictor data matrix flat file (overwritten)
c
c
c sparse predictor matrix:
c
c call splognet (parm,no,ni,nc,x,ix,jx,y,o,jd,vp,cl,ne,nx,nlam,flmin,
c ulam,thr,isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,fdev,alm,nlp,jerr)
c
c x, ix, jx = predictor data matrix in compressed sparse row format
c
c
c other inputs:
c
c parm,no,ni,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit
c = same as elnet above.
c
c nc = number of classes (distinct outcome values)
c nc=1 => binomial two-class logistic regression
c (all output references class 1)
c y(no,max(2,nc)) = number of each class at each design point
c entries may have fractional values or all be zero (overwritten)
c o(no,nc) = observation off-sets for each class
c kopt = optimization flag
c kopt = 0 => Newton-Raphson (recommended)
c kpot = 1 => modified Newton-Raphson (sometimes faster)
c kpot = 2 => nonzero coefficients same for each class (nc > 1)
c
c
c output:
c
c lmu,ia,nin,alm,nlp = same as elent above
c
c a0(nc,lmu) = intercept values for each class at each solution
c ca(nx,nc,lmu) = compressed coefficient values for each class at
c each solution
c dev0 = null deviance (intercept only model)
c fdev(lmu) = fraction of devience explained by each solution
c jerr = error flag
c jerr = 0 => no error
c jerr > 0 => fatal error - no output returned
c jerr < 7777 => memory allocation error
c jerr = 7777 => all used predictors have zero variance
c jerr = 8000 + k => null probability < 1.0e-5 for class k
c jerr = 9000 + k => null probability for class k
c > 1.0 - 1.0e-5
c jerr = 10000 => maxval(vp) <= 0.0
c jerr = 90000 => bounds adjustment non convergence
C jerr < 0 => non fatal error - partial output:
c Solutions for larger lamdas (1:(k-1)) returned.
c jerr = -k => convergence for kth lamda value not reached
c after maxit (see above) iterations.
c jerr = -10000-k => number of non zero coefficients along path
c exceeds nx (see above) at kth lamda value.
c jerr = -20000-k => max(p*(1-p)) < 1.0e-6 at kth lamda value.
c o(no,nc) = training data values for last (lmu_th) solution linear
c combination.
c
c
c
c logistic/multinomial utilitity routines:
c
c
c uncompress coefficient vectors for all solutions:
c
c call lsolns(ni,nx,nc,lmu,ca,ia,nin,b)
c
c input:
c
c ni,nx,nc = input to lognet
c lmu,ca,ia,nin = output from lognet
c
c output:
c
c b(ni,nc,lmu) = all lognet returned solutions in uncompressed format
c
c
c uncompress coefficient vector for particular solution:
c
c call luncomp(ni,nx,nc,ca,ia,nin,a)
c
c input:
c
c ni, nx, nc = same as above
c ca(nx,nc) = compressed coefficient values (for each class)
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients
c
c output:
c
c a(ni,nc) = uncompressed coefficient vectors
c referencing original variables
c
c
c evaluate linear model from compressed coefficients and
c uncompressed predictor vectors:
c
c call lmodval(nt,x,nc,nx,a0,ca,ia,nin,ans);
c
c input:
c
c nt = number of observations
c x(nt,ni) = full (uncompressed) predictor vectors
c nc, nx = same as above
c a0(nc) = intercepts
c ca(nx,nc) = compressed coefficient values (for each class)
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients
c
c output:
c
c ans(nc,nt) = model predictions
c
c
c evaluate linear model from compressed coefficients and
c compressed predictor matrix:
c
c call lcmodval(nc,nx,a0,ca,ia,nin,x,ix,jx,n,f);
c
c input:
c
c nc, nx = same as above
c a0(nc) = intercept
c ca(nx,nc) = compressed coefficient values for a solution
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients for solution
c x, ix, jx = predictor matrix in compressed sparse row format
c n = number of predictor vectors (observations)
c
c output:
c
c f(nc,n) = model predictions
c
c
c
c
c Poisson elastic net
c
c
c dense predictor matrix:
c
c call fishnet (parm,no,ni,x,y,o,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
c isd,intr,maxit,lmu,a0,ca,ia,nin,dev0,fdev,alm,nlp,jerr)
c
c x(no,ni) = predictor data matrix flat file (overwritten)
c
c sparse predictor matrix:
c
c call spfishnet (parm,no,ni,x,ix,jx,y,o,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
c isd,intr,maxit,lmu,a0,ca,ia,nin,dev0,fdev,alm,nlp,jerr)
c
c x, ix, jx = predictor data matrix in compressed sparse row format
c
c other inputs:
c
c y(no) = observation response counts
c o(no) = observation off-sets
c parm,no,ni,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit
c = same as elnet above
c
c output:
c
c lmu,a0,ca,ia,nin,alm = same as elnet above
c dev0,fdev = same as lognet above
c nlp = total number of passes over predictor variables
c jerr = error flag
c jerr = 0 => no error
c jerr > 0 => fatal error - no output returned
c jerr < 7777 => memory allocation error
c jerr = 7777 => all used predictors have zero variance
c jerr = 8888 => negative response count y values
c jerr = 9999 => no positive observations weights
c jerr = 10000 => maxval(vp) <= 0.0
C jerr < 0 => non fatal error - partial output:
c Solutions for larger lamdas (1:(k-1)) returned.
c jerr = -k => convergence for kth lamda value not reached
c after maxit (see above) iterations.
c jerr = -10000-k => number of non zero coefficients along path
c exceeds nx (see above) at kth lamda value.
c o(no) = training data values for last (lmu_th) solution linear
c combination.
c
c
c Poisson utility routines:
c
c
c same as elnet above:
c
c call solns(ni,nx,lmu,ca,ia,nin,b)
c call uncomp(ni,ca,ia,nin,a)
c call modval(a0,ca,ia,nin,n,x,f);
c call cmodval(a0,ca,ia,nin,x,ix,jx,n,f);
c
c compute deviance for given uncompressed data and set of uncompressed
c solutions
c
c call deviance(no,ni,x,y,o,w,nsol,a0,a,flog,jerr)
c
c input:
c
c no = number of observations
c ni = number of predictor variables
c x(no,ni) = predictor data matrix flat file
c y(no) = observation response counts
c o(no) = observation off-sets
c w(no)= observation weights
c nsol = number of solutions
c a0(nsol) = intercept for each solution
c a(ni,nsol) = solution coefficient vectors (uncompressed)
c
c output:
c
c flog(nsol) = respective deviance values minus null deviance
c jerr = error flag - see above
c
c
c compute deviance for given compressed data and set of uncompressed solutions
c
c call spdeviance(no,ni,x,ix,jx,y,o,w,nsol,a0,a,flog,jerr)
c
c input:
c
c no = number of observations
c ni = number of predictor variables
c x, ix, jx = predictor data matrix in compressed sparse row format
c y(no) = observation response counts
c o(no) = observation off-sets
c w(no)= observation weights
c nsol = number of solutions
c a0(nsol) = intercept for each solution
c a(ni,nsol) = solution coefficient vectors (uncompressed)
c
c output
c
c flog(nsol) = respective deviance values minus null deviance
c jerr = error flag - see above
c
c
c compute deviance for given compressed data and compressed solutions
c
c call cspdeviance(no,x,ix,jx,y,o,w,nx,lmu,a0,ca,ia,nin,flog,jerr)
c
c input:
c
c no = number of observations
c x, ix, jx = predictor data matrix in compressed sparse row format
c y(no) = observation response counts
c o(no) = observation off-sets
c w(no)= observation weights
c nx = input to spfishnet
c lmu,a0(lmu),ca(nx,lmu),ia(nx),nin(lmu) = output from spfishnet
c
c output
c
c flog(lmu) = respective deviance values minus null deviance
c jerr = error flag - see above
c
c
c
c Elastic net with Cox proportional hazards model
c
c
c dense predictor matrix:
c
c call coxnet (parm,no,ni,x,y,d,o,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
c maxit,isd,lmu,ca,ia,nin,dev0,fdev,alm,nlp,jerr)
c
c input:
c
c x(no,ni) = predictor data matrix flat file (overwritten)
c y(no) = observation times
c d(no) = died/censored indicator
c d(i)=0.0 => y(i) = censoring time
c d(i)=1.0 => y(i) = death time
c o(no) = observation off-sets
c parm,no,ni,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,maxit
c = same as fishnet above
c
c output:
c
c lmu,ca,ia,nin,dev0,fdev,alm,nlp = same as fishnet above
c jerr = error flag
c jerr = 0 => no error - output returned
c jerr > 0 => fatal error - no output returned
c jerr < 7777 => memory allocation error
c jerr = 7777 => all used predictors have zero variance
c jerr = 8888 => all observations censored (d(i)=0.0)
c jerr = 9999 => no positive observations weights
c jerr = 10000 => maxval(vp) <= 0.0
c jerr = 20000, 30000 => initialization numerical error
C jerr < 0 => non fatal error - partial output:
c Solutions for larger lamdas (1:(k-1)) returned.
c jerr = -k => convergence for kth lamda value not reached
c after maxit (see above) iterations.
c jerr = -10000-k => number of non zero coefficients along path
c exceeds nx (see above) at kth lamda value.
c jerr = -30000-k => numerical error at kth lambda value
c o(no) = training data values for last (lmu_th) solution linear
c combination.
c
c
c
c coxnet utility routines:
c
c
c same as elnet above:
c
c call solns(ni,nx,lmu,ca,ia,nin,b)
c call uncomp(ni,ca,ia,nin,a)
c
c
c evaluate linear model from compressed coefficients and
c uncompressed predictor matrix:
c
c call cxmodval(ca,ia,nin,n,x,f);
c
c input:
c
c ca(nx) = compressed coefficient values for a solution
c ia(nx) = pointers to compressed coefficients
c nin = number of compressed coefficients for solution
c n = number of predictor vectors (observations)
c x(n,ni) = full (uncompressed) predictor matrix
c
c output:
c
c f(n) = model predictions
c
c
c compute log-likelihood for given data set and vectors of coefficients
c
c call loglike(no,ni,x,y,d,o,w,nvec,a,flog,jerr)
c
c input:
c
c no = number of observations
c ni = number of predictor variables
c x(no,ni) = predictor data matrix flat file
c y(no) = observation times
c d(no) = died/censored indicator
c d(i)=0.0 => y(i) = censoring time
c d(i)=1.0 => y(i) = death time
c o(no) = observation off-sets
c w(no)= observation weights
c nvec = number of coefficient vectors
c a(ni,nvec) = coefficient vectors (uncompressed)
c
c output
c
c flog(nvec) = respective log-likelihood values
c jerr = error flag - see coxnet above
c
c
c
c
c Changing internal parameter values
c
c
c call chg_fract_dev(fdev)
c fdev = minimum fractional change in deviance for stopping path
c default = 1.0e-5
c
c call chg_dev_max(devmax)
c devmax = maximum fraction of explained deviance for stopping path
c default = 0.999
c
c call chg_min_flmin(eps)
c eps = minimum value of flmin (see above). default= 1.0e-6
c
c call chg_big(big)
c big = large floating point number. default = 9.9e35
c
c call chg_min_lambdas(mnlam)
c mnlam = minimum number of path points (lambda values) allowed
c default = 5
c
c call chg_min_null_prob(pmin)
c pmin = minimum null probability for any class. default = 1.0e-9
c
c call chg _max_exp(exmx)
c exmx = maximum allowed exponent. default = 250.0
c
c call chg_bnorm(prec,mxit)
c prec = convergence threshold for multi response bounds adjustment
c solution. default = 1.0e-10.
c mxit = maximum iterations for multiresponse bounds adjustment solution
c default = 100.
c
c
c Obtain current internal parameter values
c
c call get_int_parms(fdev,eps,big,mnlam,devmax,pmin,exmx)
c call get_bnorm(prec,mxit);
c
c
c
"
subroutine get_int_parms(sml,eps,big,mnlam,rsqmax,pmin,exmx);
data sml0,eps0,big0,mnlam0,rsqmax0,pmin0,exmx0
/1.0e-5,1.0e-6,9.9e35,5,0.999,1.0e-9,250.0/;
sml=sml0; eps=eps0; big=big0; mnlam=mnlam0; rsqmax=rsqmax0;
pmin=pmin0; exmx=exmx0;
return;
entry chg_fract_dev(arg); sml0=arg; return;
entry chg_dev_max(arg); rsqmax0=arg; return;
entry chg_min_flmin(arg); eps0=arg; return;
entry chg_big(arg); big0=arg; return;
entry chg_min_lambdas(irg); mnlam0=irg; return;
entry chg_min_null_prob(arg); pmin0=arg; return;
entry chg_max_exp(arg); exmx0=arg; return;
end;
subroutine elnet
(ka,parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,
lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
real x(no,ni),y(no),w(no),vp(ni),ca(nx,nlam),cl(2,ni);
real ulam(nlam),a0(nlam),rsq(nlam),alm(nlam);
integer jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: vq;
%mortran
if maxval(vp).le.0.0 < jerr=10000; return;>
allocate(vq(1:ni),stat=jerr); if(jerr.ne.0) return;
vq=max(0.0,vp); vq=vq*ni/sum(vq);
if ka.eq.1 <
call elnetu
(parm,no,ni,x,y,w,jd,vq,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,
lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
>
else <
call elnetn
(parm,no,ni,x,y,w,jd,vq,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,
lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
>
deallocate(vq);
return;
end;
subroutine elnetu
(parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,maxit,
lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
real x(no,ni),y(no),w(no),vp(ni),ulam(nlam),cl(2,ni);
real ca(nx,nlam),a0(nlam),rsq(nlam),alm(nlam);
integer jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,g,xv,vlam
integer, dimension (:), allocatable :: ju
%mortran
allocate(g(1:ni),stat=jerr);
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vlam(1:nlam),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
call chkvars(no,ni,x,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; return;>
call standard(no,ni,x,y,w,isd,intr,ju,g,xm,xs,ym,ys,xv,jerr);
if(jerr.ne.0) return;
cl=cl/ys; if isd.gt.0 < >
if(flmin.ge.1.0) vlam=ulam/ys;
call elnet1(parm,ni,ju,vp,cl,g,no,ne,nx,x,nlam,flmin,vlam,thr,maxit,xv,
lmu,ca,ia,nin,rsq,alm,nlp,jerr);
if(jerr.gt.0) return;
a0(k)=0.0;
if(intr.ne.0) a0(k)=ym-dot_product(ca(1:nk,k),xm(ia(1:nk)));
>
deallocate(xm,xs,g,ju,xv,vlam);
return;
end;
subroutine standard (no,ni,x,y,w,isd,intr,ju,g,xm,xs,ym,ys,xv,jerr);
real x(no,ni),y(no),w(no),g(ni),xm(ni),xs(ni),xv(ni); integer ju(ni);
%fortran
real, dimension (:), allocatable :: v
%mortran
allocate(v(1:no),stat=jerr); if(jerr.ne.0) return;
w=w/sum(w); v=sqrt(w);
if intr.eq.0 < ym=0.0; y=v*y;
ys=sqrt(dot_product(y,y)-dot_product(v,y)**2); y=y/ys;
else < xs(j)=1.0;>
>
>
else <
if isd.eq.0 < xs=1.0;>
else <
xv=1.0;
>
ym=dot_product(w,y); y=v*(y-ym); ys=sqrt(dot_product(y,y)); y=y/ys;
>
g=0.0;
deallocate(v);
return;
end;
subroutine elnet1 (beta,ni,ju,vp,cl,g,no,ne,nx,x,nlam,flmin,ulam,thr,maxit,xv,
lmu,ao,ia,kin,rsqo,almo,nlp,jerr);
real vp(ni),g(ni),x(no,ni),ulam(nlam),ao(nx,nlam),rsqo(nlam),almo(nlam),xv(ni);
real cl(2,ni);
integer ju(ni),ia(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: a,da
integer, dimension (:), allocatable :: mm
real, dimension (:,:), allocatable :: c
allocate(c(1:ni,1:nx),stat=jerr)
%mortran
call get_int_parms(sml,eps,big,mnlam,rsqmax,pmin,exmx);
allocate(a(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(da(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
bta=beta; omb=1.0-bta;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
rsq=0.0; a=0.0; mm=0; /nlp,nin/=0; iz=0; mnl=min(mnlam,nlam);
elseif m.gt.2 < alm=alm*alf;>
elseif m.eq.1 < alm=big;>
else < alm=0.0;
alm=alf*alm/max(bta,1.0e-3);
>
dem=alm*omb; ab=alm*bta; rsq0=rsq; jz=1;
loop < if(iz*jz.ne.0) go to :b:; nlp=nlp+1; dlx=0.0;
if j.eq.k < c(j,nin)=xv(j); next;>
c(j,nin)=dot_product(x(:,j),x(:,k));
>
mm(k)=nin; ia(nin)=k;
>
del=a(k)-ak; rsq=rsq+del*(2.0*g(k)-del*xv(k));
dlx=max(xv(k)*del**2,dlx);
>
if(dlx.lt.thr) exit; if(nin.gt.nx) exit;
if nlp.gt.maxit < jerr=-m; return;>
:b: iz=1; da(1:nin)=a(ia(1:nin));
loop < nlp=nlp+1; dlx=0.0;
>
if(dlx.lt.thr) exit; if nlp.gt.maxit < jerr=-m; return;>
>
da(1:nin)=a(ia(1:nin))-da(1:nin);
jz=0;
>
if nin.gt.nx < jerr=-10000-m; exit;>
if(nin.gt.0) ao(1:nin,m)=a(ia(1:nin)); kin(m)=nin;
rsqo(m)=rsq; almo(m)=alm; lmu=m;
if(m.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if(rsq-rsq0.lt.sml*rsq) exit; if(rsq.gt.rsqmax) exit;
>
deallocate(a,mm,c,da);
return;
end;
subroutine elnetn (parm,no,ni,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,
intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
real vp(ni),x(no,ni),y(no),w(no),ulam(nlam),cl(2,ni);
real ca(nx,nlam),a0(nlam),rsq(nlam),alm(nlam);
integer jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,xv,vlam
integer, dimension (:), allocatable :: ju
%mortran
allocate(xm(1:ni),stat=jerr);
allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vlam(1:nlam),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
call chkvars(no,ni,x,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; return;>
call standard1(no,ni,x,y,w,isd,intr,ju,xm,xs,ym,ys,xv,jerr);
if(jerr.ne.0) return;
cl=cl/ys; if isd.gt.0 < >
if(flmin.ge.1.0) vlam=ulam/ys;
call elnet2(parm,ni,ju,vp,cl,y,no,ne,nx,x,nlam,flmin,vlam,thr,maxit,xv,
lmu,ca,ia,nin,rsq,alm,nlp,jerr);
if(jerr.gt.0) return;
a0(k)=0.0;
if(intr.ne.0) a0(k)=ym-dot_product(ca(1:nk,k),xm(ia(1:nk)));
>
deallocate(xm,xs,ju,xv,vlam);
return;
end;
subroutine standard1 (no,ni,x,y,w,isd,intr,ju,xm,xs,ym,ys,xv,jerr);
real x(no,ni),y(no),w(no),xm(ni),xs(ni),xv(ni); integer ju(ni);
%fortran
real, dimension (:), allocatable :: v
%mortran
allocate(v(1:no),stat=jerr); if(jerr.ne.0) return;
w=w/sum(w); v=sqrt(w);
if intr.eq.0 < ym=0.0; y=v*y;
ys=sqrt(dot_product(y,y)-dot_product(v,y)**2); y=y/ys;
else < xs(j)=1.0;>
>
go to :out:;
>
if isd.eq.0 < xs=1.0;>
else <
xv=1.0;
>
ym=dot_product(w,y); y=v*(y-ym); ys=sqrt(dot_product(y,y)); y=y/ys;
:out:deallocate(v);
return;
end;
subroutine elnet2(beta,ni,ju,vp,cl,y,no,ne,nx,x,nlam,flmin,ulam,thr,maxit,xv,
lmu,ao,ia,kin,rsqo,almo,nlp,jerr);
real vp(ni),y(no),x(no,ni),ulam(nlam),ao(nx,nlam),rsqo(nlam),almo(nlam),xv(ni);
real cl(2,ni);
integer ju(ni),ia(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: a,g
integer, dimension (:), allocatable :: mm,ix
%mortran
call get_int_parms(sml,eps,big,mnlam,rsqmax,pmin,exmx);
allocate(a(1:ni),stat=jerr);
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(g(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ix(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
bta=beta; omb=1.0-bta; ix=0;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
rsq=0.0; a=0.0; mm=0; /nlp,nin/=0; iz=0; mnl=min(mnlam,nlam); alm=0.0;
elseif m.gt.2 < alm=alm*alf;>
elseif m.eq.1 < alm=big;>
else < alm0=0.0;
alm0=alm0/max(bta,1.0e-3); alm=alf*alm0;
>
dem=alm*omb; ab=alm*bta; rsq0=rsq; jz=1;
tlam=bta*(2.0*alm-alm0);
loop < if(iz*jz.ne.0) go to :b:;
:again:nlp=nlp+1; dlx=0.0;
del=a(k)-ak; rsq=rsq+del*(2.0*gk-del*xv(k));
y=y-del*x(:,k); dlx=max(xv(k)*del**2,dlx);
>
if(nin.gt.nx) exit;
if dlx.lt.thr < ixx=0;
>
if(ixx.eq.1) go to :again:;
exit;
>
if nlp.gt.maxit < jerr=-m; return;>
:b: iz=1;
loop < nlp=nlp+1; dlx=0.0;
if(dlx.lt.thr) exit; if nlp.gt.maxit < jerr=-m; return;>
>
jz=0;
>
if nin.gt.nx < jerr=-10000-m; exit;>
if(nin.gt.0) ao(1:nin,m)=a(ia(1:nin)); kin(m)=nin;
rsqo(m)=rsq; almo(m)=alm; lmu=m;
if(m.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if(rsq-rsq0.lt.sml*rsq) exit; if(rsq.gt.rsqmax) exit;
>
deallocate(a,mm,g,ix);
return;
end;
subroutine chkvars(no,ni,x,ju);
real x(no,ni); integer ju(ni);
>
return;
end;
subroutine uncomp(ni,ca,ia,nin,a);
real ca(*),a(ni); integer ia(*);
a=0.0; if(nin.gt.0) a(ia(1:nin))=ca(1:nin);
return;
end;
subroutine modval(a0,ca,ia,nin,n,x,f);
real ca(nin),x(n,*),f(n); integer ia(nin);
f=a0; if(nin.le.0) return;
return;
end;
subroutine spelnet
(ka,parm,no,ni,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,
maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
real x(*),y(no),w(no),vp(ni),ulam(nlam),cl(2,ni);
real ca(nx,nlam),a0(nlam),rsq(nlam),alm(nlam);
integer ix(*),jx(*),jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: vq;
%mortran
if maxval(vp).le.0.0 < jerr=10000; return;>
allocate(vq(1:ni),stat=jerr); if(jerr.ne.0) return;
vq=max(0.0,vp); vq=vq*ni/sum(vq);
if ka.eq.1 <
call spelnetu
(parm,no,ni,x,ix,jx,y,w,jd,vq,cl,ne,nx,nlam,flmin,ulam,thr,isd,
intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
>
else <
call spelnetn
(parm,no,ni,x,ix,jx,y,w,jd,vq,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,
maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
>
deallocate(vq);
return;
end;
subroutine spelnetu
(parm,no,ni,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,intr,
maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
real x(*),y(no),w(no),vp(ni),ulam(nlam),cl(2,ni);
real ca(nx,nlam),a0(nlam),rsq(nlam),alm(nlam);
integer ix(*),jx(*),jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,g,xv,vlam
integer, dimension (:), allocatable :: ju
%mortran
allocate(g(1:ni),stat=jerr);
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vlam(1:nlam),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
call spchkvars(no,ni,x,ix,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; return;>
call spstandard(no,ni,x,ix,jx,y,w,ju,isd,intr,g,xm,xs,ym,ys,xv,jerr);
if(jerr.ne.0) return;
cl=cl/ys; if isd.gt.0 < >
if(flmin.ge.1.0) vlam=ulam/ys;
call spelnet1(parm,ni,g,no,w,ne,nx,x,ix,jx,ju,vp,cl,nlam,flmin,vlam,thr,maxit,
xm,xs,xv,lmu,ca,ia,nin,rsq,alm,nlp,jerr);
if(jerr.gt.0) return;
a0(k)=0.0;
if(intr.ne.0) a0(k)=ym-dot_product(ca(1:nk,k),xm(ia(1:nk)));
>
deallocate(xm,xs,g,ju,xv,vlam);
return;
end;
subroutine spstandard (no,ni,x,ix,jx,y,w,ju,isd,intr,g,xm,xs,ym,ys,xv,jerr);
real x(*),y(no),w(no),g(ni),xm(ni),xs(ni),xv(ni); integer ix(*),jx(*),ju(ni);
w=w/sum(w);
if intr.eq.0 < ym=0.0;
ys=sqrt(dot_product(w,y**2)-dot_product(w,y)**2); y=y/ys;
else < xs(j)=1.0;>
>
>
else <
if isd.eq.0 < xs=1.0;> else < xv=1.0;>
ym=dot_product(w,y); y=y-ym; ys=sqrt(dot_product(w,y**2)); y=y/ys;
>
g=0.0;
return;
end;
subroutine spelnet1(beta,ni,g,no,w,ne,nx,x,ix,jx,ju,vp,cl,nlam,flmin,ulam,
thr,maxit,xm,xs,xv,lmu,ao,ia,kin,rsqo,almo,nlp,jerr);
real g(ni),vp(ni),x(*),ulam(nlam),w(no);
real ao(nx,nlam),rsqo(nlam),almo(nlam),xm(ni),xs(ni),xv(ni),cl(2,ni);
integer ix(*),jx(*),ju(ni),ia(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: a,da
integer, dimension (:), allocatable :: mm
real, dimension (:,:), allocatable :: c
allocate(c(1:ni,1:nx),stat=jerr)
%mortran
call get_int_parms(sml,eps,big,mnlam,rsqmax,pmin,exmx);
allocate(a(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(da(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
bta=beta; omb=1.0-bta;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
rsq=0.0; a=0.0; mm=0; /nlp,nin/=0; iz=0; mnl=min(mnlam,nlam);
elseif m.gt.2 < alm=alm*alf;>
elseif m.eq.1 < alm=big;>
else < alm=0.0;
alm=alf*alm/max(bta,1.0e-3);
>
dem=alm*omb; ab=alm*bta; rsq0=rsq; jz=1;
loop < if(iz*jz.ne.0) go to :b:; nlp=nlp+1; dlx=0.0;
if j.eq.k < c(j,nin)=xv(j); next;>
c(j,nin)=
(row_prod(j,k,ix,jx,x,w)-xm(j)*xm(k))/(xs(j)*xs(k));
>
mm(k)=nin; ia(nin)=k;
>
del=a(k)-ak; rsq=rsq+del*(2.0*g(k)-del*xv(k));
dlx=max(xv(k)*del**2,dlx);
>
if(dlx.lt.thr) exit; if(nin.gt.nx) exit;
if nlp.gt.maxit < jerr=-m; return;>
:b: iz=1; da(1:nin)=a(ia(1:nin));
loop < nlp=nlp+1; dlx=0.0;
>
if(dlx.lt.thr) exit; if nlp.gt.maxit < jerr=-m; return;>
>
da(1:nin)=a(ia(1:nin))-da(1:nin);
jz=0;
>
if nin.gt.nx < jerr=-10000-m; exit;>
if(nin.gt.0) ao(1:nin,m)=a(ia(1:nin)); kin(m)=nin;
rsqo(m)=rsq; almo(m)=alm; lmu=m;
if(m.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if(rsq-rsq0.lt.sml*rsq) exit; if(rsq.gt.rsqmax) exit;
>
deallocate(a,mm,c,da);
return;
end;
subroutine spelnetn(parm,no,ni,x,ix,jx,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,
thr,isd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
real x(*),vp(ni),y(no),w(no),ulam(nlam),cl(2,ni);
real ca(nx,nlam),a0(nlam),rsq(nlam),alm(nlam);
integer ix(*),jx(*),jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,xv,vlam
integer, dimension (:), allocatable :: ju
%mortran
allocate(xm(1:ni),stat=jerr);
allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vlam(1:nlam),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
call spchkvars(no,ni,x,ix,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; return;>
call spstandard1(no,ni,x,ix,jx,y,w,ju,isd,intr,xm,xs,ym,ys,xv,jerr);
if(jerr.ne.0) return;
cl=cl/ys; if isd.gt.0 < >
if(flmin.ge.1.0) vlam=ulam/ys;
call spelnet2(parm,ni,y,w,no,ne,nx,x,ix,jx,ju,vp,cl,nlam,flmin,vlam,thr,maxit,
xm,xs,xv,lmu,ca,ia,nin,rsq,alm,nlp,jerr);
if(jerr.gt.0) return;
a0(k)=0.0;
if(intr.ne.0) a0(k)=ym-dot_product(ca(1:nk,k),xm(ia(1:nk)));
>
deallocate(xm,xs,ju,xv,vlam);
return;
end;
subroutine spstandard1 (no,ni,x,ix,jx,y,w,ju,isd,intr,xm,xs,ym,ys,xv,jerr);
real x(*),y(no),w(no),xm(ni),xs(ni),xv(ni); integer ix(*),jx(*),ju(ni);
w=w/sum(w);
if intr.eq.0 < ym=0.0;
ys=sqrt(dot_product(w,y**2)-dot_product(w,y)**2); y=y/ys;
else < xs(j)=1.0;>
>
return;
>
if isd.eq.0 < xs=1.0;> else < xv=1.0;>
ym=dot_product(w,y); y=y-ym; ys=sqrt(dot_product(w,y**2)); y=y/ys;
return;
end;
subroutine spelnet2(beta,ni,y,w,no,ne,nx,x,ix,jx,ju,vp,cl,nlam,flmin,ulam,
thr,maxit,xm,xs,xv,lmu,ao,ia,kin,rsqo,almo,nlp,jerr);
real y(no),w(no),x(*),vp(ni),ulam(nlam),cl(2,ni);
real ao(nx,nlam),rsqo(nlam),almo(nlam),xm(ni),xs(ni),xv(ni);
integer ix(*),jx(*),ju(ni),ia(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: a,g
integer, dimension (:), allocatable :: mm,iy
%mortran
call get_int_parms(sml,eps,big,mnlam,rsqmax,pmin,exmx);
allocate(a(1:ni),stat=jerr);
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(g(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(iy(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
bta=beta; omb=1.0-bta; alm=0.0; iy=0;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
rsq=0.0; a=0.0; mm=0; o=0.0; /nlp,nin/=0; iz=0; mnl=min(mnlam,nlam);
elseif m.gt.2 < alm=alm*alf;>
elseif m.eq.1 < alm=big;>
else < alm0=0.0;
alm0=alm0/max(bta,1.0e-3); alm=alf*alm0;
>
dem=alm*omb; ab=alm*bta; rsq0=rsq; jz=1;
tlam=bta*(2.0*alm-alm0);
loop < if(iz*jz.ne.0) go to :b:;
:again:nlp=nlp+1; dlx=0.0;
del=a(k)-ak; rsq=rsq+del*(2.0*gk-del*xv(k));
y(jx(jb:je))=y(jx(jb:je))-del*x(jb:je)/xs(k);
o=o+del*xm(k)/xs(k); dlx=max(xv(k)*del**2,dlx);
>
if(nin.gt.nx) exit;
if dlx.lt.thr < ixx=0;
>
if(ixx.eq.1) go to :again:;
exit;
>
if nlp.gt.maxit < jerr=-m; return;>
:b: iz=1;
loop < nlp=nlp+1; dlx=0.0;
if(dlx.lt.thr) exit; if nlp.gt.maxit < jerr=-m; return;>
>
jz=0;
>
if nin.gt.nx < jerr=-10000-m; exit;>
if(nin.gt.0) ao(1:nin,m)=a(ia(1:nin)); kin(m)=nin;
rsqo(m)=rsq; almo(m)=alm; lmu=m;
if(m.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if(rsq-rsq0.lt.sml*rsq) exit; if(rsq.gt.rsqmax) exit;
>
deallocate(a,mm,g,iy);
return;
end;
subroutine spchkvars(no,ni,x,ix,ju);
real x(*); integer ix(*),ju(ni);
>
else < t=x(jb); >
>
return;
end;
subroutine cmodval(a0,ca,ia,nin,x,ix,jx,n,f);
real ca(*),x(*),f(n); integer ia(*),ix(*),jx(*);
f=a0;
return;
end;
function row_prod(i,j,ia,ja,ra,w);
integer ia(*),ja(*); real ra(*),w(*);
row_prod=dot(ra(ia(i)),ra(ia(j)),ja(ia(i)),ja(ia(j)),
ia(i+1)-ia(i),ia(j+1)-ia(j),w);
return;
end;
function dot(x,y,mx,my,nx,ny,w);
real x(*),y(*),w(*); integer mx(*),my(*);
/i,j/=1; s=0.0;
loop <
until mx(i).ge.my(j) < i=i+1; if(i.gt.nx) go to :done:;>
if(mx(i).eq.my(j)) go to :equal:;
until my(j).ge.mx(i) < j=j+1; if(j.gt.ny) go to :done:;>
if(mx(i).eq.my(j)) go to :equal:; next;
:equal:s=s+w(mx(i))*x(i)*y(j);
i=i+1; if(i.gt.nx) exit; j=j+1; if(j.gt.ny) exit;
>
:done: dot=s;
return;
end;
subroutine lognet (parm,no,ni,nc,x,y,g,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
real x(no,ni),y(no,max(2,nc)),g(no,nc),vp(ni),ulam(nlam);
real ca(nx,nc,nlam),a0(nc,nlam),dev(nlam),alm(nlam),cl(2,ni);
integer jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,ww,vq,xv
integer, dimension (:), allocatable :: ju
%mortran
if maxval(vp).le.0.0 < jerr=10000; return;>
allocate(ww(1:no),stat=jerr);
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vq(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
if kopt.eq.2 < allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;>
if isd.gt.0 < allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;>
if(jerr.ne.0) return;
call chkvars(no,ni,x,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; return;>
vq=max(0.0,vp); vq=vq*ni/sum(vq);
sw=sum(ww); ww=ww/sw;
if nc.eq.1 < call lstandard1(no,ni,x,ww,ju,isd,intr,xm,xs);
if isd.gt.0 < >
call lognet2n(parm,no,ni,x,y(:,1),g(:,1),ww,ju,vq,cl,ne,nx,nlam,flmin,ulam,
thr,isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
>
elseif kopt.eq.2 < call multlstandard1(no,ni,x,ww,ju,isd,intr,xm,xs,xv);
if isd.gt.0 < >
call multlognetn(parm,no,ni,nc,x,y,g,ww,ju,vq,cl,ne,nx,nlam,flmin,ulam,thr,
intr,maxit,xv,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
>
else < call lstandard1(no,ni,x,ww,ju,isd,intr,xm,xs);
if isd.gt.0 < >
call lognetn(parm,no,ni,nc,x,y,g,ww,ju,vq,cl,ne,nx,nlam,flmin,ulam,thr,
isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
>
if(jerr.gt.0) return; dev0=2.0*sw*dev0;
>
if intr.eq.0 < a0(ic,k)=0.0;>
else < a0(ic,k)=a0(ic,k)-dot_product(ca(1:nk,ic,k),xm(ia(1:nk)));>
>
>
deallocate(ww,ju,vq,xm); if(isd.gt.0) deallocate(xs);
if(kopt.eq.2) deallocate(xv);
return;
end;
subroutine lstandard1 (no,ni,x,w,ju,isd,intr,xm,xs);
real x(no,ni),w(no),xm(ni),xs(ni); integer ju(ni);
if intr.eq.0 <
>
return;
>
>
return;
end;
subroutine multlstandard1 (no,ni,x,w,ju,isd,intr,xm,xs,xv);
real x(no,ni),w(no),xm(ni),xs(ni),xv(ni); integer ju(ni);
if intr.eq.0 <
>
return;
>
>
return;
end;
subroutine lognet2n(parm,no,ni,x,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,
isd,intr,maxit,kopt,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr);
real x(no,ni),y(no),g(no),w(no),vp(ni),ulam(nlam),cl(2,ni);
real a(nx,nlam),a0(nlam),dev(nlam),alm(nlam);
integer ju(ni),m(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: b,bs,v,r,xv,q,ga
integer, dimension (:), allocatable :: mm,ixx
%mortran
call get_int_parms(sml,eps,big,mnlam,devmax,pmin,exmx);
allocate(b(0:ni),stat=jerr);
allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ga(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(bs(0:ni),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ixx(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(r(1:no),stat=ierr); jerr=jerr+ierr;
allocate(v(1:no),stat=ierr); jerr=jerr+ierr;
allocate(q(1:no),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
fmax=log(1.0/pmin-1.0); fmin=-fmax; vmin=(1.0+pmin)*pmin*(1.0-pmin);
bta=parm; omb=1.0-bta;
q0=dot_product(w,y); if q0.le.pmin < jerr=8001; return;>
if q0.ge.1.0-pmin < jerr=9001; return;>
if(intr.eq.0.0) q0=0.5;
ixx=0; al=0.0; bz=0.0; if(intr.ne.0) bz=log(q0/(1.0-q0));
if nonzero(no,g).eq.0 < vi=q0*(1.0-q0); b(0)=bz; v=vi*w;
r=w*(y-q0); q=q0; xmz=vi; dev1=-(bz*q0+log(1.0-q0));
>
else < b(0)=0.0;
if intr.ne.0 < b(0)=azero(no,y,g,w,jerr); if(jerr.ne.0) return;>
q=1.0/(1.0+exp(-b(0)-g)); v=w*q*(1.0-q); r=w*(y-q); xmz=sum(v);
dev1=-(b(0)*q0+dot_product(w,y*g+log(1.0-q)));
>
if kopt.gt.0 <
if isd.gt.0.and.intr.ne.0 < xv=0.25;>
else < >
>
dev0=dev1;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
m=0; mm=0; /nlp,nin/=0; mnl=min(mnlam,nlam); bs=0.0; b(1:ni)=0.0;
shr=shri*dev0;
elseif ilm.gt.2 < al=al*alf;>
elseif ilm.eq.1 < al=big;>
else < al0=0.0;
al0=al0/max(bta,1.0e-3); al=alf*al0;
>
al2=al*omb; al1=al*bta; tlam=bta*(2.0*al-al0);
:again:continue;
loop < bs(0)=b(0); if(nin.gt.0) bs(m(1:nin))=b(m(1:nin));
if kopt.eq.0 <
>
loop < nlp=nlp+1; dlx=0.0;
else <
b(k)=max(cl(1,k),min(cl(2,k),sign(au,u)/(xv(k)+vp(k)*al2)));
>
d=b(k)-bk; if(abs(d).le.0.0) next; dlx=max(dlx,xv(k)*d**2);
r=r-d*v*x(:,k);
if mm(k).eq.0 < nin=nin+1; if(nin.gt.nx) exit;
mm(k)=nin; m(nin)=k;
>
>
if(nin.gt.nx) exit;
d=0.0; if(intr.ne.0) d=sum(r)/xmz;
if d.ne.0.0 < b(0)=b(0)+d; dlx=max(dlx,xmz*d**2); r=r-d*v;>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
loop < nlp=nlp+1; dlx=0.0;
else <
b(k)=max(cl(1,k),min(cl(2,k),sign(au,u)/(xv(k)+vp(k)*al2)));
>
d=b(k)-bk; if(abs(d).le.0.0) next; dlx=max(dlx,xv(k)*d**2);
r=r-d*v*x(:,k);
>
d=0.0; if(intr.ne.0) d=sum(r)/xmz;
if d.ne.0.0 < b(0)=b(0)+d; dlx=max(dlx,xmz*d**2); r=r-d*v;>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
>
>
if(nin.gt.nx) exit;
elseif fi.gt.fmax < q(i)=1.0;>
else < q(i)=1.0/(1.0+exp(-fi));>
>
v=w*q*(1.0-q); xmz=sum(v); if(xmz.le.vmin) exit; r=w*(y-q);
if xmz*(b(0)-bs(0))**2.lt.shr < ix=0;
if ix.eq.0 <
>
if(ix.eq.1) go to :again:;
exit;
>
>
>
if nin.gt.nx < jerr=-10000-ilm; exit;>
if(nin.gt.0) a(1:nin,ilm)=b(m(1:nin)); kin(ilm)=nin;
a0(ilm)=b(0); alm(ilm)=al; lmu=ilm;
devi=dev2(no,w,y,q,pmin);
dev(ilm)=(dev1-devi)/dev0; if(xmz.le.vmin) exit;
if(ilm.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if(dev(ilm).gt.devmax) exit; if(dev(ilm)-dev(ilm-1).lt.sml) exit;
>
g=log(q/(1.0-q));
deallocate(b,bs,v,r,xv,q,mm,ga,ixx);
return;
end;
function dev2(n,w,y,p,pmin);
real w(n),y(n),p(n);
pmax=1.0-pmin; s=0.0;
dev2=s;
return;
end;
function azero(n,y,g,q,jerr);
parameter(eps=1.0e-7);
real y(n),g(n),q(n);
%fortran
real, dimension (:), allocatable :: e,p,w
%mortran
allocate(e(1:n),stat=jerr);
allocate(p(1:n),stat=ierr); jerr=jerr+ierr;
allocate(w(1:n),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
az=0.0; e=exp(-g); qy=dot_product(q,y); p=1.0/(1.0+e);
loop < w=q*p*(1.0-p);
d=(qy-dot_product(q,p))/sum(w); az=az+d; if(abs(d).lt.eps) exit;
ea0=exp(-az); p=1.0/(1.0+ea0*e);
>
azero=az;
deallocate(e,p,w);
return;
end;
subroutine lognetn(parm,no,ni,nc,x,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,
isd,intr,maxit,kopt,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr);
real x(no,ni),y(no,nc),g(no,nc),w(no),vp(ni),ulam(nlam);
real a(nx,nc,nlam),a0(nc,nlam),dev(nlam),alm(nlam),cl(2,ni);
integer ju(ni),m(nx),kin(nlam);
%fortran
real, dimension (:,:), allocatable :: q
real, dimension (:), allocatable :: sxp,sxpl
real, dimension (:), allocatable :: di,v,r,ga
real, dimension (:,:), allocatable :: b,bs,xv
integer, dimension (:), allocatable :: mm,is,ixx
allocate(b(0:ni,1:nc),stat=jerr)
allocate(xv(1:ni,1:nc),stat=ierr); jerr=jerr+ierr
allocate(bs(0:ni,1:nc),stat=ierr); jerr=jerr+ierr
allocate(q(1:no,1:nc),stat=ierr); jerr=jerr+ierr
%mortran
call get_int_parms(sml,eps,big,mnlam,devmax,pmin,exmx); exmn=-exmx;
allocate(r(1:no),stat=ierr); jerr=jerr+ierr;
allocate(v(1:no),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(is(1:max(nc,ni)),stat=ierr); jerr=jerr+ierr;
allocate(sxp(1:no),stat=ierr); jerr=jerr+ierr;
allocate(sxpl(1:no),stat=ierr); jerr=jerr+ierr;
allocate(di(1:no),stat=ierr); jerr=jerr+ierr;
allocate(ga(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ixx(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
pmax=1.0-pmin; emin=pmin/pmax; emax=1.0/emin;
pfm=(1.0+pmin)*pmin; pfx=(1.0-pmin)*pmax; vmin=pfm*pmax;
bta=parm; omb=1.0-bta; dev1=0.0; dev0=0.0;
if q0.ge.1.0-pmin < jerr =9000+ic; return;>
if intr.eq.0 < q0=1.0/nc; b(0,ic)=0.0;>
else < b(0,ic)=log(q0); dev1=dev1-q0*b(0,ic);>
b(1:ni,ic)=0.0;
>
if(intr.eq.0) dev1=log(float(nc)); ixx=0; al=0.0;
if nonzero(no*nc,g).eq.0 <
b(0,:)=b(0,:)-sum(b(0,:))/nc; sxp=0.0;
>
else < sxp=0.0;
if intr.eq.0 < b(0,:)=0.0;>
else < call kazero(nc,no,y,g,w,b(0,:),jerr); if(jerr.ne.0) return;>
dev1=0.0;
sxpl=w*log(sxp);
>
>
dev0=dev0+dev1;
if kopt.gt.0 <
if isd.gt.0.and.intr.ne.0 < xv=0.25;>
else < >
>
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
m=0; mm=0; nin=0; nlp=0; mnl=min(mnlam,nlam); bs=0.0; shr=shri*dev0;
ga=0.0;
>
elseif ilm.gt.2 < al=al*alf;>
elseif ilm.eq.1 < al=big;>
else < al0=0.0;
al0=al0/max(bta,1.0e-3); al=alf*al0;
>
al2=al*omb; al1=al*bta; tlam=bta*(2.0*al-al0);
:again:continue;
loop < /ix,jx/=0; ig=0;
elseif pic.gt.pfx < pic=1.0; v(i)=0.0;>
else < v(i)=w(i)*pic*(1.0-pic); xmz=xmz+v(i);>
r(i)=w(i)*(y(i,ic)-pic);
>
if(xmz.le.vmin) next; ig=1;
if kopt.eq.0 <
>
loop < nlp=nlp+1; dlx=0.0;
else <
b(k,ic)=max(cl(1,k),min(cl(2,k),sign(au,u)/
(xv(k,ic)+vp(k)*al2)));
>
d=b(k,ic)-bk; if(abs(d).le.0.0) next;
dlx=max(dlx,xv(k,ic)*d**2); r=r-d*v*x(:,k);
if mm(k).eq.0 < nin=nin+1;
if nin.gt.nx < jx=1; exit;>
mm(k)=nin; m(nin)=k;
>
>
if(jx.gt.0) exit;
d=0.0; if(intr.ne.0) d=sum(r)/xmz;
if d.ne.0.0 < b(0,ic)=b(0,ic)+d; dlx=max(dlx,xmz*d**2); r=r-d*v;>
if(dlx.lt.shr) exit;
if nlp.gt.maxit < jerr=-ilm; return;>
loop < nlp=nlp+1; dlx=0.0;
else <
b(k,ic)=max(cl(1,k),min(cl(2,k),sign(au,u)/
(xv(k,ic)+vp(k)*al2)));
>
d=b(k,ic)-bk; if(abs(d).le.0.0) next;
dlx=max(dlx,xv(k,ic)*d**2); r=r-d*v*x(:,k);
>
d=0.0; if(intr.ne.0) d=sum(r)/xmz;
if d.ne.0.0 < b(0,ic)=b(0,ic)+d;
dlx=max(dlx,xmz*d**2); r=r-d*v;
>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
>
>
if(jx.gt.0) exit;
if(xmz*(b(0,ic)-bs(0,ic))**2.gt.shr) ix=1;
if ix.eq.0 <
>
>
>
s=-sum(b(0,:))/nc; b(0,:)=b(0,:)+s; di=s;
else < s=elc(parm,nc,cl(:,l),b(l,:),is);>
b(l,:)=b(l,:)-s; di=di-s*x(:,l);
>
di=exp(di); sxp=sxp*di;
if(jx.gt.0) exit; if(ig.eq.0) exit;
if ix.eq.0 <
>
>
if(ix.eq.1) go to :again:;
exit;
>
>
if jx.gt.0 < jerr=-10000-ilm; exit;> devi=0.0;
>
kin(ilm)=nin; alm(ilm)=al; lmu=ilm;
dev(ilm)=(dev1-devi)/dev0; if(ig.eq.0) exit;
if(ilm.lt.mnl) next; if(flmin.ge.1.0) next;
if(nintot(ni,nx,nc,a(1,1,ilm),m,nin,is).gt.ne) exit;
if(dev(ilm).gt.devmax) exit; if(dev(ilm)-dev(ilm-1).lt.sml) exit;
>
g=log(q);
deallocate(sxp,b,bs,v,r,xv,q,mm,is,ga,ixx);
return;
end;
subroutine kazero(kk,n,y,g,q,az,jerr);
parameter(eps=1.0e-7);
real y(n,kk),g(n,kk),q(n),az(kk);
%fortran
real, dimension (:), allocatable :: s
real, dimension (:,:), allocatable :: e
allocate(e(1:n,1:kk),stat=jerr)
%mortran
allocate(s(1:n),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
az=0.0; e=exp(g);
loop < dm=0.0;
d=t/u; az(k)=az(k)+d; ed=exp(d); dm=max(dm,abs(d));
>
> until dm.lt.eps;
az=az-sum(az)/kk;
deallocate(e,s);
return;
end;
function elc(parm,n,cl,a,m);
real a(n),cl(2); integer m(n);
fn=n; am=sum(a)/fn;
if parm.eq.0.0.or.n.eq.2 < elc=am; go to :chk:;>
call psort7(a,m,1,n);
if a(m(1)).eq.a(m(n)) < elc=a(1); go to :chk:;>
if mod(n,2).eq.1 < ad=a(m(n/2+1));>
else < ad=0.5*(a(m(n/2+1))+a(m(n/2)));>
if parm.eq.1.0 < elc=ad; go to :chk:;>
b1=min(am,ad); b2=max(am,ad); k2=1;
until a(m(k2)).gt.b1 < k2=k2+1;> k1=k2-1;
until a(m(k2)).ge.b2 < k2=k2+1;>
r=parm/((1.0-parm)*fn); is=0; sm=n-2*(k1-1);
>
if is.ne.0 < elc=s; go to :chk:;> r2=2.0*r; s1=a(m(k1)); am2=2.0*am;
cri=r2*sum(abs(a-s1))+s1*(s1-am2); elc=s1;
s1=s;
>
:chk:elc=max(maxval(a-cl(2)),min(minval(a-cl(1)),elc));
return;
end;
function nintot(ni,nx,nc,a,m,nin,is);
real a(nx,nc); integer m(nx),is(ni);
is=0; nintot=0;
>
return;
end;
subroutine luncomp(ni,nx,nc,ca,ia,nin,a);
real ca(nx,nc),a(ni,nc); integer ia(nx);
a=0.0;
return;
end;
subroutine lmodval(nt,x,nc,nx,a0,ca,ia,nin,ans);
real a0(nc),ca(nx,nc),x(nt,*),ans(nc,nt); integer ia(nx);
>
return;
end;
subroutine splognet (parm,no,ni,nc,x,ix,jx,y,g,jd,vp,cl,ne,nx,nlam,flmin,
ulam,thr,isd,intr,maxit,kopt,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
real x(*),y(no,max(2,nc)),g(no,nc),vp(ni),ulam(nlam);
real ca(nx,nc,nlam),a0(nc,nlam),dev(nlam),alm(nlam),cl(2,ni);
integer ix(*),jx(*),jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,ww,vq,xv
integer, dimension (:), allocatable :: ju
%mortran
if maxval(vp).le.0.0 < jerr=10000; return;>
allocate(ww(1:no),stat=jerr);
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vq(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;
if kopt.eq.2 < allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;>
if(jerr.ne.0) return;
call spchkvars(no,ni,x,ix,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; return;>
vq=max(0.0,vp); vq=vq*ni/sum(vq);
sw=sum(ww); ww=ww/sw;
if nc.eq.1 < call splstandard2(no,ni,x,ix,jx,ww,ju,isd,intr,xm,xs);
if isd.gt.0 < >
call sprlognet2n(parm,no,ni,x,ix,jx,y(:,1),g(:,1),ww,ju,vq,cl,ne,nx,nlam,
flmin,ulam,thr,isd,intr,maxit,kopt,xm,xs,lmu,a0,ca,ia,nin,dev0,dev,
alm,nlp,jerr);
>
elseif kopt.eq.2 <
call multsplstandard2(no,ni,x,ix,jx,ww,ju,isd,intr,xm,xs,xv);
if isd.gt.0 < >
call multsprlognetn(parm,no,ni,nc,x,ix,jx,y,g,ww,ju,vq,cl,ne,nx,nlam,flmin,
ulam,thr,intr,maxit,xv,xm,xs,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
>
else < call splstandard2(no,ni,x,ix,jx,ww,ju,isd,intr,xm,xs);
if isd.gt.0 < >
call sprlognetn(parm,no,ni,nc,x,ix,jx,y,g,ww,ju,vq,cl,ne,nx,nlam,flmin,
ulam,thr,isd,intr,maxit,kopt,xm,xs,lmu,a0,ca,
ia,nin,dev0,dev,alm,nlp,jerr);
>
if(jerr.gt.0) return; dev0=2.0*sw*dev0;
>
if intr.eq.0 < a0(ic,k)=0.0;>
else < a0(ic,k)=a0(ic,k)-dot_product(ca(1:nk,ic,k),xm(ia(1:nk)));>
>
>
deallocate(ww,ju,vq,xm,xs); if(kopt.eq.2) deallocate(xv);
return;
end;
subroutine multsplstandard2(no,ni,x,ix,jx,w,ju,isd,intr,xm,xs,xv);
real x(*),w(no),xm(ni),xs(ni),xv(ni); integer ix(*),jx(*),ju(ni);
if intr.eq.0 <
else < xs(j)=1.0;>
>
return;
>
>
if(isd.eq.0) xs=1.0;
return;
end;
subroutine splstandard2(no,ni,x,ix,jx,w,ju,isd,intr,xm,xs);
real x(*),w(no),xm(ni),xs(ni); integer ix(*),jx(*),ju(ni);
if intr.eq.0 <
else < xs(j)=1.0;>
>
return;
>
if(isd.eq.0) xs=1.0;
return;
end;
subroutine sprlognet2n (parm,no,ni,x,ix,jx,y,g,w,ju,vp,cl,ne,nx,nlam,
flmin,ulam,shri,isd,intr,maxit,kopt,xb,xs,
lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr);
real x(*),y(no),g(no),w(no),vp(ni),ulam(nlam),cl(2,ni);
real a(nx,nlam),a0(nlam),dev(nlam),alm(nlam);
real xb(ni),xs(ni); integer ix(*),jx(*),ju(ni),m(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: xm,b,bs,v,r,sc,xv,q,ga
integer, dimension (:), allocatable :: mm,ixx
%mortran
call get_int_parms(sml,eps,big,mnlam,devmax,pmin,exmx);
allocate(b(0:ni),stat=jerr);
allocate(xm(0:ni),stat=ierr); jerr=jerr+ierr;
allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(bs(0:ni),stat=ierr); jerr=jerr+ierr;
allocate(ga(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ixx(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(q(1:no),stat=ierr); jerr=jerr+ierr;
allocate(r(1:no),stat=ierr); jerr=jerr+ierr;
allocate(v(1:no),stat=ierr); jerr=jerr+ierr;
allocate(sc(1:no),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
fmax=log(1.0/pmin-1.0); fmin=-fmax; vmin=(1.0+pmin)*pmin*(1.0-pmin);
bta=parm; omb=1.0-bta;
q0=dot_product(w,y); if q0.le.pmin < jerr=8001; return;>
if q0.ge.1.0-pmin < jerr=9001; return;>
if(intr.eq.0) q0=0.5; bz=0.0; if(intr.ne.0) bz=log(q0/(1.0-q0));
if nonzero(no,g).eq.0 < vi=q0*(1.0-q0); b(0)=bz; v=vi*w;
r=w*(y-q0); q=q0; xm(0)=vi; dev1=-(bz*q0+log(1.0-q0));
>
else < b(0)=0.0;
if intr.ne.0 < b(0)=azero(no,y,g,w,jerr); if(jerr.ne.0) return;>
q=1.0/(1.0+exp(-b(0)-g)); v=w*q*(1.0-q); r=w*(y-q); xm(0)=sum(v);
dev1=-(b(0)*q0+dot_product(w,y*g+log(1.0-q)));
>
if kopt.gt.0 <
if isd.gt.0.and.intr.ne.0 < xv=0.25;>
else <
>
>
b(1:ni)=0.0; dev0=dev1;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
m=0; mm=0; nin=0; /o,svr/=0.0; mnl=min(mnlam,nlam); bs=0.0; /nlp,nin/=0;
shr=shri*dev0; al=0.0; ixx=0;
elseif ilm.gt.2 < al=al*alf;>
elseif ilm.eq.1 < al=big;>
else < al0=0.0;
al0=al0/max(bta,1.0e-3); al=alf*al0;
>
al2=al*omb; al1=al*bta; tlam=bta*(2.0*al-al0);
:again:continue;
loop < bs(0)=b(0); if(nin.gt.0) bs(m(1:nin))=b(m(1:nin));
>
loop < nlp=nlp+1; dlx=0.0;
else <
b(k)=max(cl(1,k),min(cl(2,k),sign(au,u)/(xv(k)+vp(k)*al2)));
>
d=b(k)-bk; if(abs(d).le.0.0) next; dlx=max(dlx,xv(k)*d**2);
if mm(k).eq.0 < nin=nin+1; if(nin.gt.nx) exit;
mm(k)=nin; m(nin)=k; sc(1:jn)=v(jx(jb:je));
xm(k)=dot_product(sc(1:jn),x(jb:je));
>
r(jx(jb:je))=r(jx(jb:je))-d*v(jx(jb:je))*x(jb:je)/xs(k);
o=o+d*(xb(k)/xs(k));
svr=svr-d*(xm(k)-xb(k)*xm(0))/xs(k);
>
if(nin.gt.nx) exit;
d=0.0; if(intr.ne.0) d=svr/xm(0);
if d.ne.0.0 < b(0)=b(0)+d; dlx=max(dlx,xm(0)*d**2); r=r-d*v;
svr=svr-d*xm(0);
>
if(dlx.lt.shr) exit;
if nlp.gt.maxit < jerr=-ilm; return;>
loop < nlp=nlp+1; dlx=0.0;
else <
b(k)=max(cl(1,k),min(cl(2,k),sign(au,u)/(xv(k)+vp(k)*al2)));
>
d=b(k)-bk; if(abs(d).le.0.0) next; dlx=max(dlx,xv(k)*d**2);
r(jx(jb:je))=r(jx(jb:je))-d*v(jx(jb:je))*x(jb:je)/xs(k);
o=o+d*(xb(k)/xs(k));
svr=svr-d*(xm(k)-xb(k)*xm(0))/xs(k);
>
d=0.0; if(intr.ne.0) d=svr/xm(0);
if d.ne.0.0 < b(0)=b(0)+d; dlx=max(dlx,xm(0)*d**2); r=r-d*v;
svr=svr-d*xm(0);
>
if(dlx.lt.shr) exit;
if nlp.gt.maxit < jerr=-ilm; return;>
>
>
if(nin.gt.nx) exit;
sc=b(0); b0=0.0;
sc=sc+b0;
elseif fi.gt.fmax < q(i)=1.0;>
else < q(i)=1.0/(1.0+exp(-fi));>
>
v=w*q*(1.0-q); xm(0)=sum(v); if(xm(0).lt.vmin) exit;
r=w*(y-q); svr=sum(r); o=0.0;
if xm(0)*(b(0)-bs(0))**2.lt.shr < kx=0;
if kx.eq.0 <
>
if(kx.eq.1) go to :again:;
exit;
>
>
>
if nin.gt.nx < jerr=-10000-ilm; exit;>
if(nin.gt.0) a(1:nin,ilm)=b(m(1:nin)); kin(ilm)=nin;
a0(ilm)=b(0); alm(ilm)=al; lmu=ilm;
devi=dev2(no,w,y,q,pmin);
dev(ilm)=(dev1-devi)/dev0;
if(ilm.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if(dev(ilm).gt.devmax) exit; if(dev(ilm)-dev(ilm-1).lt.sml) exit;
if(xm(0).lt.vmin) exit;
>
g=log(q/(1.0-q));
deallocate(xm,b,bs,v,r,sc,xv,q,mm,ga,ixx);
return;
end;
subroutine sprlognetn(parm,no,ni,nc,x,ix,jx,y,g,w,ju,vp,cl,ne,nx,nlam,flmin,
ulam,shri,isd,intr,maxit,kopt,xb,xs,lmu,a0,a,m,kin,dev0,dev,alm,nlp,jerr);
real x(*),y(no,nc),g(no,nc),w(no),vp(ni),ulam(nlam),xb(ni),xs(ni);
real a(nx,nc,nlam),a0(nc,nlam),dev(nlam),alm(nlam),cl(2,ni);
integer ix(*),jx(*),ju(ni),m(nx),kin(nlam);
%fortran
real, dimension (:,:), allocatable :: q
real, dimension (:), allocatable :: sxp,sxpl
real, dimension (:), allocatable :: sc,xm,v,r,ga
real, dimension (:,:), allocatable :: b,bs,xv
integer, dimension (:), allocatable :: mm,is,iy
allocate(b(0:ni,1:nc),stat=jerr)
allocate(xv(1:ni,1:nc),stat=ierr); jerr=jerr+ierr
allocate(bs(0:ni,1:nc),stat=ierr); jerr=jerr+ierr
allocate(q(1:no,1:nc),stat=ierr); jerr=jerr+ierr
%mortran
call get_int_parms(sml,eps,big,mnlam,devmax,pmin,exmx); exmn=-exmx;
allocate(xm(0:ni),stat=ierr); jerr=jerr+ierr;
allocate(r(1:no),stat=ierr); jerr=jerr+ierr;
allocate(v(1:no),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ga(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(iy(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(is(1:max(nc,ni)),stat=ierr); jerr=jerr+ierr;
allocate(sxp(1:no),stat=ierr); jerr=jerr+ierr;
allocate(sxpl(1:no),stat=ierr); jerr=jerr+ierr;
allocate(sc(1:no),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
pmax=1.0-pmin; emin=pmin/pmax; emax=1.0/emin;
pfm=(1.0+pmin)*pmin; pfx=(1.0-pmin)*pmax; vmin=pfm*pmax;
bta=parm; omb=1.0-bta; dev1=0.0; dev0=0.0;
if q0.ge.1.0-pmin < jerr =9000+ic; return;>
if(intr.eq.0) q0=1.0/nc;
b(1:ni,ic)=0.0; b(0,ic)=0.0;
if intr.ne.0 < b(0,ic)=log(q0); dev1=dev1-q0*b(0,ic);>
>
if(intr.eq.0) dev1=log(float(nc)); iy=0; al=0.0;
if nonzero(no*nc,g).eq.0 <
b(0,:)=b(0,:)-sum(b(0,:))/nc; sxp=0.0;
>
else < sxp=0.0;
if intr.eq.0 < b(0,:)=0.0;>
else < call kazero(nc,no,y,g,w,b(0,:),jerr); if(jerr.ne.0) return;>
dev1=0.0;
sxpl=w*log(sxp);
>
>
dev0=dev0+dev1;
if kopt.gt.0 <
if isd.gt.0.and.intr.ne.0 < xv=0.25;>
else <
>
>
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
m=0; mm=0; nin=0; nlp=0; mnl=min(mnlam,nlam); bs=0.0; svr=0.0; o=0.0;
shr=shri*dev0; ga=0.0;
>
elseif ilm.gt.2 < al=al*alf;>
elseif ilm.eq.1 < al=big;>
else < al0=0.0;
al0=al0/max(bta,1.0e-3); al=alf*al0;
>
al2=al*omb; al1=al*bta; tlam=bta*(2.0*al-al0);
:again:continue;
loop < /ixx,jxx/=0; ig=0;
elseif pic.gt.pfx < pic=1.0; v(i)=0.0;>
else < v(i)=w(i)*pic*(1.0-pic); xm(0)=xm(0)+v(i);>
r(i)=w(i)*(y(i,ic)-pic); svr=svr+r(i);
>
if(xm(0).le.vmin) next; ig=1;
>
loop < nlp=nlp+1; dlx=0.0;
else <
b(k,ic)=max(cl(1,k),min(cl(2,k),sign(au,u)/
(xv(k,ic)+vp(k)*al2)));
>
d=b(k,ic)-bk; if(abs(d).le.0.0) next;
dlx=max(dlx,xv(k,ic)*d**2);
if mm(k).eq.0 < nin=nin+1;
if nin.gt.nx < jxx=1; exit;>
mm(k)=nin; m(nin)=k;
xm(k)=dot_product(v(jx(jb:je)),x(jb:je));
>
r(jx(jb:je))=r(jx(jb:je))-d*v(jx(jb:je))*x(jb:je)/xs(k);
o=o+d*(xb(k)/xs(k));
svr=svr-d*(xm(k)-xb(k)*xm(0))/xs(k);
>
if(jxx.gt.0) exit;
d=0.0; if(intr.ne.0) d=svr/xm(0);
if d.ne.0.0 < b(0,ic)=b(0,ic)+d; dlx=max(dlx,xm(0)*d**2);
r=r-d*v; svr=svr-d*xm(0);
>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
loop < nlp=nlp+1; dlx=0.0;
else <
b(k,ic)=max(cl(1,k),min(cl(2,k),sign(au,u)/
(xv(k,ic)+vp(k)*al2)));
>
d=b(k,ic)-bk; if(abs(d).le.0.0) next;
dlx=max(dlx,xv(k,ic)*d**2);
r(jx(jb:je))=r(jx(jb:je))-d*v(jx(jb:je))*x(jb:je)/xs(k);
o=o+d*(xb(k)/xs(k));
svr=svr-d*(xm(k)-xb(k)*xm(0))/xs(k);
>
d=0.0; if(intr.ne.0) d=svr/xm(0);
if d.ne.0.0 < b(0,ic)=b(0,ic)+d; dlx=max(dlx,xm(0)*d**2);
r=r-d*v; svr=svr-d*xm(0);
>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
>
>
if(jxx.gt.0) exit;
if(xm(0)*(b(0,ic)-bs(0,ic))**2.gt.shr) ixx=1;
if ixx.eq.0 <
>
>
sc=b(0,ic)+g(:,ic); b0=0.0;
sc=min(max(exmn,sc+b0),exmx);
sxp=sxp-q(:,ic);
q(:,ic)=min(max(emin*sxp,exp(sc)),emax*sxp);
sxp=sxp+q(:,ic);
>
s=-sum(b(0,:))/nc; b(0,:)=b(0,:)+s; sc=s; b0=0.0;
else < s=elc(parm,nc,cl(:,l),b(l,:),is);>
b(l,:)=b(l,:)-s;
jb=ix(l); je=ix(l+1)-1;
sc(jx(jb:je))=sc(jx(jb:je))-s*x(jb:je)/xs(l);
b0=b0+s*xb(l)/xs(l);
>
sc=sc+b0; sc=exp(sc); sxp=sxp*sc;
if(jxx.gt.0) exit; if(ig.eq.0) exit;
if ixx.eq.0 <
>
>
if(ixx.eq.1) go to :again:;
exit;
>
>
if jxx.gt.0 < jerr=-10000-ilm; exit;> devi=0.0;
>
kin(ilm)=nin; alm(ilm)=al; lmu=ilm;
dev(ilm)=(dev1-devi)/dev0; if(ig.eq.0) exit;
if(ilm.lt.mnl) next; if(flmin.ge.1.0) next;
if(nintot(ni,nx,nc,a(1,1,ilm),m,nin,is).gt.ne) exit;
if(dev(ilm).gt.devmax) exit; if(dev(ilm)-dev(ilm-1).lt.sml) exit;
>
g=log(q);
deallocate(sxp,b,bs,v,r,xv,q,mm,is,xm,sc,ga,iy);
return;
end;
subroutine lcmodval(nc,nx,a0,ca,ia,nin,x,ix,jx,n,f);
real a0(nc),ca(nx,nc),x(*),f(nc,n); integer ia(*),ix(*),jx(*);
>
return;
end;
subroutine coxnet (parm,no,ni,x,y,d,g,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
maxit,isd,lmu,ca,ia,nin,dev0,dev,alm,nlp,jerr);
real x(no,ni),y(no),d(no),g(no),w(no),vp(ni),ulam(nlam);
real ca(nx,nlam),dev(nlam),alm(nlam),cl(2,ni);
integer jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xs,ww,vq
integer, dimension (:), allocatable :: ju
%mortran
if maxval(vp).le.0.0 < jerr=10000; return;>
allocate(ww(1:no),stat=jerr);
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vq(1:ni),stat=ierr); jerr=jerr+ierr;
if isd.gt.0 < allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;>
if(jerr.ne.0) return;
call chkvars(no,ni,x,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; return;>
vq=max(0.0,vp); vq=vq*ni/sum(vq);
ww=max(0.0,w); sw=sum(ww);
if sw.le.0.0 < jerr=9999; return;> ww=ww/sw;
call cstandard(no,ni,x,ww,ju,isd,xs);
if isd.gt.0 < >
call coxnet1(parm,no,ni,x,y,d,g,ww,ju,vq,cl,ne,nx,nlam,flmin,ulam,thr,
isd,maxit,lmu,ca,ia,nin,dev0,dev,alm,nlp,jerr);
if(jerr.gt.0) return; dev0=2.0*sw*dev0;
if isd.gt.0 < >
deallocate(ww,ju,vq); if(isd.gt.0) deallocate(xs);
return;
end;
subroutine cstandard (no,ni,x,w,ju,isd,xs);
real x(no,ni),w(no),xs(ni); integer ju(ni);
>
return;
end;
subroutine coxnet1(parm,no,ni,x,y,d,g,q,ju,vp,cl,ne,nx,nlam,flmin,ulam,cthri,
isd,maxit,lmu,ao,m,kin,dev0,dev,alm,nlp,jerr);
real x(no,ni),y(no),q(no),d(no),g(no),vp(ni),ulam(nlam);
real ao(nx,nlam),dev(nlam),alm(nlam),cl(2,ni);
integer ju(ni),m(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: w,dk,v,xs,wr,a,as,f,dq
real, dimension (:), allocatable :: e,uu,ga
integer, dimension (:), allocatable :: jp,kp,mm,ixx
%mortran
call get_int_parms(sml,eps,big,mnlam,devmax,pmin,exmx);
sml=sml*100.0; devmax=devmax*0.99/0.999;
allocate(e(1:no),stat=jerr);
allocate(uu(1:no),stat=ierr); jerr=jerr+ierr;
allocate(f(1:no),stat=ierr); jerr=jerr+ierr;
allocate(w(1:no),stat=ierr); jerr=jerr+ierr;
allocate(v(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(a(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(as(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ga(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ixx(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(jp(1:no),stat=ierr); jerr=jerr+ierr;
allocate(kp(1:no),stat=ierr); jerr=jerr+ierr;
allocate(dk(1:no),stat=ierr); jerr=jerr+ierr;
allocate(wr(1:no),stat=ierr); jerr=jerr+ierr;
allocate(dq(1:no),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0)go to :done:;
call groups(no,y,d,q,nk,kp,jp,t0,jerr);
if(jerr.ne.0) go to :done:; alpha=parm;
oma=1.0-alpha; nlm=0; ixx=0; al=0.0;
dq=d*q; call died(no,nk,dq,kp,jp,dk);
a=0.0; f(1)=0.0; fmax=log(huge(f(1))*0.1);
if nonzero(no,g).ne.0 < f=g-dot_product(q,g);
e=q*exp(sign(min(abs(f),fmax),f));
>
else < f=0.0; e=q;>
r0=risk(no,ni,nk,dq,dk,f,e,kp,jp,uu);
rr=-(dot_product(dk(1:nk),log(dk(1:nk)))+r0); dev0=rr;
>
call outer(no,nk,dq,dk,kp,jp,e,wr,w,jerr,uu);
if(jerr.ne.0) go to :done:;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
m=0; mm=0; /nlp,nin/=0; mnl=min(mnlam,nlam); as=0.0; cthr=cthri*dev0;
elseif ilm.gt.2 < al=al*alf;>
elseif ilm.eq.1 < al=big;>
else < al0=0.0;
al0=al0/max(parm,1.0e-3); al=alf*al0;
>
sa=alpha*al; omal=oma*al; tlam=alpha*(2.0*al-al0);
:again:continue;
loop < if(nin.gt.0) as(m(1:nin))=a(m(1:nin));
call vars(no,ni,x,w,ixx,v);
loop < nlp=nlp+1; dli=0.0;
else < at=max(cl(1,j),min(cl(2,j),sign(abs(u)-vp(j)*sa,u)/
(v(j)+vp(j)*omal)));
>
if at.ne.a(j) < del=at-a(j); a(j)=at; dli=max(dli,v(j)*del**2);
wr=wr-del*w*x(:,j); f=f+del*x(:,j);
if mm(j).eq.0 < nin=nin+1; if(nin.gt.nx) exit;
mm(j)=nin; m(nin)=j;
>
>
>
if(nin.gt.nx) exit; if(dli.lt.cthr) exit;
if nlp.gt.maxit < jerr=-ilm; return;>
loop < nlp=nlp+1; dli=0.0;
else < at=max(cl(1,j),min(cl(2,j),sign(abs(u)-vp(j)*sa,u)/
(v(j)+vp(j)*omal)));
>
if at.ne.a(j) < del=at-a(j); a(j)=at; dli=max(dli,v(j)*del**2);
wr=wr-del*w*x(:,j); f=f+del*x(:,j);
>
>
if(dli.lt.cthr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
>
>
if(nin.gt.nx) exit;
e=q*exp(sign(min(abs(f),fmax),f));
call outer(no,nk,dq,dk,kp,jp,e,wr,w,jerr,uu);
if jerr.ne.0 < jerr=jerr-ilm; go to :done:;>
ix=0;
if ix.eq.0 <
>
if(ix.eq.1) go to :again:;
exit;
>
>
if nin.gt.nx < jerr=-10000-ilm; exit;>
if(nin.gt.0) ao(1:nin,ilm)=a(m(1:nin)); kin(ilm)=nin;
alm(ilm)=al; lmu=ilm;
dev(ilm)=(risk(no,ni,nk,dq,dk,f,e,kp,jp,uu)-r0)/rr;
if(ilm.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if((dev(ilm)-dev(ilm-mnl+1))/dev(ilm).lt.sml) exit;
if(dev(ilm).gt.devmax) exit;
>
g=f;
:done: deallocate(e,uu,w,dk,v,xs,f,wr,a,as,jp,kp,dq,mm,ga,ixx);
return;
end;
subroutine cxmodval(ca,ia,nin,n,x,f);
real ca(nin),x(n,*),f(n); integer ia(nin);
f=0.0; if(nin.le.0) return;
return;
end;
subroutine groups(no,y,d,q,nk,kp,jp,t0,jerr);
real y(no),d(no),q(no); integer jp(no),kp(*"nk");
call psort7(y,jp,1,no);
nj=0;
if nj.eq.0 < jerr=20000; return;>
j=1; until d(jp(j)).gt.0.0 < j=j+1;> until j.gt.nj;
if j.ge.nj-1 < jerr=30000; return;>
t0=y(jp(j)); j0=j-1;
if j0.gt.0 <
until y(jp(j0)).lt.t0 < j0=j0-1;> until j0.eq.0;
if j0.gt.0 < nj=nj-j0; >
>
jerr=0; nk=0; yk=t0; j=2;
loop <
until d(jp(j)).gt.0.0.and.y(jp(j)).gt.yk < j=j+1;> until j.gt.nj;
nk=nk+1; kp(nk)=j-1; if(j.gt.nj) exit;
if j.eq.nj < nk=nk+1; kp(nk)=nj; exit;>
yk=y(jp(j)); j=j+1;
>
return;
end;
subroutine outer(no,nk,d,dk,kp,jp,e,wr,w,jerr,u);
real d(no),dk(nk),wr(no),w(no);
real e(no),u(no),b,c; integer kp(nk),jp(no);
call usk(no,nk,kp,jp,e,u);
b=dk(1)/u(1); c=dk(1)/u(1)**2; jerr=0;
wr(i)=d(i)-e(i)*b;
>
wr(i)=d(i)-e(i)*b;
>
>
return;
end;
subroutine vars(no,ni,x,w,ixx,v);
real x(no,ni),w(no),v(ni); integer ixx(ni);
return;
end;
subroutine died(no,nk,d,kp,jp,dk);
real d(no),dk(nk); integer kp(nk),jp(no);
dk(1)=sum(d(jp(1:kp(1))));
return;
end;
subroutine usk(no,nk,kp,jp,e,u);
real e(no),u(nk),h; integer kp(nk),jp(no);
h=0.0;
u(k)=h;
>
return;
end;
function risk(no,ni,nk,d,dk,f,e,kp,jp,u);
real d(no),dk(nk),f(no);
integer kp(nk),jp(no); real e(no),u(nk),s;
call usk(no,nk,kp,jp,e,u); u=log(u);
risk=dot_product(d,f)-dot_product(dk,u);
return;
end;
subroutine loglike(no,ni,x,y,d,g,w,nlam,a,flog,jerr);
real x(no,ni),y(no),d(no),g(no),w(no),a(ni,nlam),flog(nlam);
%fortran
real, dimension (:), allocatable :: dk,f,xm,dq,q
real, dimension (:), allocatable :: e,uu
integer, dimension (:), allocatable :: jp,kp
%mortran
allocate(e(1:no),stat=jerr);
allocate(q(1:no),stat=ierr); jerr=jerr+ierr;
allocate(uu(1:no),stat=ierr); jerr=jerr+ierr;
allocate(f(1:no),stat=ierr); jerr=jerr+ierr;
allocate(dk(1:no),stat=ierr); jerr=jerr+ierr;
allocate(jp(1:no),stat=ierr); jerr=jerr+ierr;
allocate(kp(1:no),stat=ierr); jerr=jerr+ierr;
allocate(dq(1:no),stat=ierr); jerr=jerr+ierr;
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) go to :done:;
q=max(0.0,w); sw=sum(q);
if sw.le.0.0 < jerr=9999; go to :done:;>
call groups(no,y,d,q,nk,kp,jp,t0,jerr);
if(jerr.ne.0) go to :done:; fmax=log(huge(e(1))*0.1);
dq=d*q; call died(no,nk,dq,kp,jp,dk); gm=dot_product(q,g)/sw;
flog(lam)=risk(no,ni,nk,dq,dk,f,e,kp,jp,uu);
>
:done: deallocate(e,uu,dk,f,jp,kp,dq);
return;
end;
subroutine fishnet (parm,no,ni,x,y,g,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
isd,intr,maxit,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
real x(no,ni),y(no),g(no),w(no),vp(ni),ulam(nlam);
real ca(nx,nlam),a0(nlam),dev(nlam),alm(nlam),cl(2,ni);
integer jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,ww,vq
integer, dimension (:), allocatable :: ju
%mortran
if maxval(vp).le.0.0 < jerr=10000; return;>
if minval(y).lt.0.0 < jerr=8888; return;>
allocate(ww(1:no),stat=jerr);
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vq(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
if isd.gt.0 < allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;>
if(jerr.ne.0) return;
call chkvars(no,ni,x,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; go to :done:;>
vq=max(0.0,vp); vq=vq*ni/sum(vq);
ww=max(0.0,w); sw=sum(ww); if sw.le.0.0 < jerr=9999; go to :done:;>
ww=ww/sw;
call lstandard1(no,ni,x,ww,ju,isd,intr,xm,xs);
if isd.gt.0 < >
call fishnet1(parm,no,ni,x,y,g,ww,ju,vq,cl,ne,nx,nlam,flmin,ulam,thr,
isd,intr,maxit,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
if(jerr.gt.0) go to :done:; dev0=2.0*sw*dev0;
else < a0(k)=a0(k)-dot_product(ca(1:nk,k),xm(ia(1:nk)));>
>
:done:deallocate(ww,ju,vq,xm); if(isd.gt.0) deallocate(xs);
return;
end;
subroutine fishnet1(parm,no,ni,x,y,g,q,ju,vp,cl,ne,nx,nlam,flmin,ulam,shri,
isd,intr,maxit,lmu,a0,ca,m,kin,dev0,dev,alm,nlp,jerr);
real x(no,ni),y(no),g(no),q(no),vp(ni),ulam(nlam);
real ca(nx,nlam),a0(nlam),dev(nlam),alm(nlam),cl(2,ni);
integer ju(ni),m(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: t,w,wr,v,a,f,as,ga
integer, dimension (:), allocatable :: mm,ixx
%mortran
call get_int_parms(sml,eps,big,mnlam,devmax,pmin,exmx); sml=sml*10.0;
allocate(a(1:ni),stat=jerr);
allocate(as(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(t(1:no),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ga(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ixx(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(wr(1:no),stat=ierr); jerr=jerr+ierr;
allocate(v(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(w(1:no),stat=ierr); jerr=jerr+ierr;
allocate(f(1:no),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
bta=parm; omb=1.0-bta;
t=q*y; yb=sum(t); fmax=log(huge(bta)*0.1);
if nonzero(no,g).eq.0 <
if intr.ne.0 < w=q*yb; az=log(yb); f=az; dv0=yb*(az-1.0);>
else < w=q; az=0.0; f=az; dv0=-1.0;>
>
else < w=q*exp(sign(min(abs(g),fmax),g)); v0=sum(w);
if intr.ne.0 < eaz=yb/v0; w=eaz*w; az=log(eaz); f=az+g;
dv0=dot_product(t,g)-yb*(1.0-az);
>
else < az=0.0; f=g; dv0=dot_product(t,g)-v0;>
>
a=0.0; as=0.0; wr=t-w; v0=1.0; if(intr.ne.0) v0=yb; dvr=-yb;
dvr=dvr-dv0; dev0=dvr;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
m=0; mm=0; /nlp,nin/=0; mnl=min(mnlam,nlam); shr=shri*dev0; ixx=0; al=0.0;
elseif ilm.gt.2 < al=al*alf;>
elseif ilm.eq.1 < al=big;>
else < al0=0.0;
al0=al0/max(bta,1.0e-3); al=alf*al0;
>
al2=al*omb; al1=al*bta; tlam=bta*(2.0*al-al0);
:again:continue;
loop <
az0=az; if(nin.gt.0) as(m(1:nin))=a(m(1:nin));
loop < nlp=nlp+1; dlx=0.0;
else <
a(k)=max(cl(1,k),min(cl(2,k),sign(au,u)/(v(k)+vp(k)*al2)));
>
if(a(k).eq.ak) next; d=a(k)-ak; dlx=max(dlx,v(k)*d**2);
wr=wr-d*w*x(:,k); f=f+d*x(:,k);
if mm(k).eq.0 < nin=nin+1; if(nin.gt.nx) exit;
mm(k)=nin; m(nin)=k;
>
>
if(nin.gt.nx) exit;
if intr.ne.0 < d=sum(wr)/v0;
az=az+d; dlx=max(dlx,v0*d**2); wr=wr-d*w; f=f+d;
>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
loop < nlp=nlp+1; dlx=0.0;
else <
a(k)=max(cl(1,k),min(cl(2,k),sign(au,u)/(v(k)+vp(k)*al2)));
>
if(a(k).eq.ak) next; d=a(k)-ak; dlx=max(dlx,v(k)*d**2);
wr=wr-d*w*x(:,k); f=f+d*x(:,k)
>
if intr.ne.0 < d=sum(wr)/v0; az=az+d;
dlx=max(dlx,v0*d**2); wr=wr-d*w; f=f+d;
>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
>
>
if(nin.gt.nx) exit;
w=q*exp(sign(min(abs(f),fmax),f)); v0=sum(w); wr=t-w;
if v0*(az-az0)**2.lt.shr < ix=0;
if ix.eq.0 <
>
if(ix.eq.1) go to :again:;
exit;
>
>
>
if nin.gt.nx < jerr=-10000-ilm; exit;>
if(nin.gt.0) ca(1:nin,ilm)=a(m(1:nin)); kin(ilm)=nin;
a0(ilm)=az; alm(ilm)=al; lmu=ilm;
dev(ilm)=(dot_product(t,f)-v0-dv0)/dvr;
if(ilm.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if((dev(ilm)-dev(ilm-mnl+1))/dev(ilm).lt.sml) exit;
if(dev(ilm).gt.devmax) exit;
>
g=f;
:done:deallocate(t,w,wr,v,a,f,as,mm,ga,ixx);
return;
end;
function nonzero(n,v);
real v(n);
nonzero=0; >
return;
end;
subroutine solns(ni,nx,lmu,a,ia,nin,b);
real a(nx,lmu),b(ni,lmu); integer ia(nx),nin(lmu);
return;
end;
subroutine lsolns(ni,nx,nc,lmu,a,ia,nin,b);
real a(nx,nc,lmu),b(ni,nc,lmu); integer ia(nx),nin(lmu);
return;
end;
subroutine deviance(no,ni,x,y,g,q,nlam,a0,a,flog,jerr);
real x(no,ni),y(no),g(no),q(no),a(ni,nlam),a0(nlam),flog(nlam);
%fortran
real, dimension (:), allocatable :: w
%mortran
if minval(y).lt.0.0 < jerr=8888; return;>
allocate(w(1:no),stat=jerr); if(jerr.ne.0) return;
w=max(0.0,q); sw=sum(w); if sw.le.0.0 < jerr=9999; go to :done:;>
yb=dot_product(w,y)/sw; fmax=log(huge(y(1))*0.1);
flog(lam)=2.0*(sw*yb*(log(yb)-1.0)-s);
>
:done: deallocate(w);
return;
end;
subroutine spfishnet (parm,no,ni,x,ix,jx,y,g,w,jd,vp,cl,ne,nx,nlam,flmin,
ulam,thr,isd,intr,maxit,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
real x(*),y(no),g(no),w(no),vp(ni),ulam(nlam),cl(2,ni);
real ca(nx,nlam),a0(nlam),dev(nlam),alm(nlam);
integer ix(*),jx(*),jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,ww,vq
integer, dimension (:), allocatable :: ju
%mortran
if maxval(vp).le.0.0 < jerr=10000; return;>
if minval(y).lt.0.0 < jerr=8888; return;>
allocate(ww(1:no),stat=jerr);
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(vq(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
call spchkvars(no,ni,x,ix,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; go to :done:;>
vq=max(0.0,vp); vq=vq*ni/sum(vq);
ww=max(0.0,w); sw=sum(ww); if sw.le.0.0 < jerr=9999; go to :done:;>
ww=ww/sw;
call splstandard2(no,ni,x,ix,jx,ww,ju,isd,intr,xm,xs);
if isd.gt.0 < >
call spfishnet1(parm,no,ni,x,ix,jx,y,g,ww,ju,vq,cl,ne,nx,nlam,flmin,ulam,thr,
isd,intr,maxit,xm,xs,lmu,a0,ca,ia,nin,dev0,dev,alm,nlp,jerr);
if(jerr.gt.0) go to :done:; dev0=2.0*sw*dev0;
else < a0(k)=a0(k)-dot_product(ca(1:nk,k),xm(ia(1:nk)));>
>
:done:deallocate(ww,ju,vq,xm,xs);
return;
end;
subroutine spfishnet1(parm,no,ni,x,ix,jx,y,g,q,ju,vp,cl,ne,nx,nlam,flmin,ulam,
shri,isd,intr,maxit,xb,xs,lmu,a0,ca,m,kin,dev0,dev,alm,nlp,jerr);
real x(*),y(no),g(no),q(no),vp(ni),ulam(nlam),xb(ni),xs(ni);
real ca(nx,nlam),a0(nlam),dev(nlam),alm(nlam),cl(2,ni);
integer ix(*),jx(*),ju(ni),m(nx),kin(nlam);
%fortran
real, dimension (:), allocatable :: qy,t,w,wr,v,a,as,xm,ga
integer, dimension (:), allocatable :: mm,ixx
%mortran
call get_int_parms(sml,eps,big,mnlam,devmax,pmin,exmx); sml=sml*10.0;
allocate(a(1:ni),stat=jerr);
allocate(as(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(t(1:no),stat=ierr); jerr=jerr+ierr;
allocate(mm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ga(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ixx(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(wr(1:no),stat=ierr); jerr=jerr+ierr;
allocate(v(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(w(1:no),stat=ierr); jerr=jerr+ierr;
allocate(qy(1:no),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
bta=parm; omb=1.0-bta; fmax=log(huge(bta)*0.1);
qy=q*y; yb=sum(qy);
if nonzero(no,g).eq.0 < t=0.0;
if intr.ne.0 < w=q*yb; az=log(yb); uu=az;
xm=yb*xb; dv0=yb*(az-1.0);
>
else < w=q; xm=0.0; /uu,az/=0.0; dv0=-1.0;>
>
else < w=q*exp(sign(min(abs(g),fmax),g)); ww=sum(w); t=g;
if intr.ne.0 < eaz=yb/ww;
w=eaz*w; /az,uu/=log(eaz); dv0=dot_product(qy,g)-yb*(1.0-az);
>
else < /uu,az/=0.0; dv0=dot_product(qy,g)-ww;>
>
tt=yb*uu; ww=1.0; if(intr.ne.0) ww=yb; wr=qy-q*(yb*(1.0-uu)); a=0.0; as=0.0;
dvr=-yb;
dvr=dvr-dv0; dev0=dvr;
if flmin.lt.1.0 < eqs=max(eps,flmin); alf=eqs**(1.0/(nlam-1));>
m=0; mm=0; /nlp,nin/=0; mnl=min(mnlam,nlam); shr=shri*dev0; al=0.0; ixx=0;
elseif ilm.gt.2 < al=al*alf;>
elseif ilm.eq.1 < al=big;>
else < al0=0.0;
al0=al0/max(bta,1.0e-3); al=alf*al0;
>
al2=al*omb; al1=al*bta; tlam=bta*(2.0*al-al0);
:again:continue;
loop <
az0=az; if(nin.gt.0) as(m(1:nin))=a(m(1:nin));
loop <
nlp=nlp+1; dlx=0.0;
else <
a(k)=max(cl(1,k),min(cl(2,k),sign(au,u)/(v(k)+vp(k)*al2)));
>
if(a(k).eq.ak) next;
if mm(k).eq.0 < nin=nin+1; if(nin.gt.nx) exit;
mm(k)=nin; m(nin)=k;
>
d=a(k)-ak; dlx=max(dlx,v(k)*d**2); dv=d/xs(k);
wr(jx(jb:je))=wr(jx(jb:je))-dv*w(jx(jb:je))*x(jb:je);
t(jx(jb:je))=t(jx(jb:je))+dv*x(jb:je);
uu=uu-dv*xb(k); tt=tt-dv*xm(k);
>
if(nin.gt.nx) exit;
if intr.ne.0 < d=tt/ww-uu;
az=az+d; dlx=max(dlx,ww*d**2); uu=uu+d;
>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
loop < nlp=nlp+1; dlx=0.0;
else <
a(k)=max(cl(1,k),min(cl(2,k),sign(au,u)/(v(k)+vp(k)*al2)));
>
if(a(k).eq.ak) next; d=a(k)-ak; dlx=max(dlx,v(k)*d**2);
dv=d/xs(k); wr(jx(jb:je))=wr(jx(jb:je))-dv*w(jx(jb:je))*x(jb:je);
t(jx(jb:je))=t(jx(jb:je))+dv*x(jb:je);
uu=uu-dv*xb(k); tt=tt-dv*xm(k);
>
if intr.ne.0 < d=tt/ww-uu; az=az+d;
dlx=max(dlx,ww*d**2); uu=uu+d;
>
if(dlx.lt.shr) exit; if nlp.gt.maxit < jerr=-ilm; return;>
>
>
if(nin.gt.nx) exit;
euu=exp(sign(min(abs(uu),fmax),uu));
w=euu*q*exp(sign(min(abs(t),fmax),t)); ww=sum(w);
wr=qy-w*(1.0-uu); tt=sum(wr);
if ww*(az-az0)**2.lt.shr < kx=0;
if kx.eq.0 <
>
if(kx.eq.1) go to :again:;
exit;
>
>
>
if nin.gt.nx < jerr=-10000-ilm; exit;>
if(nin.gt.0) ca(1:nin,ilm)=a(m(1:nin)); kin(ilm)=nin;
a0(ilm)=az; alm(ilm)=al; lmu=ilm;
dev(ilm)=(dot_product(qy,t)+yb*uu-ww-dv0)/dvr;
if(ilm.lt.mnl) next; if(flmin.ge.1.0) next;
me=0; if(me.gt.ne) exit;
if((dev(ilm)-dev(ilm-mnl+1))/dev(ilm).lt.sml) exit;
if(dev(ilm).gt.devmax) exit;
>
g=t+uu;
:done:deallocate(t,w,wr,v,a,qy,xm,as,mm,ga,ixx);
return;
end;
subroutine spdeviance(no,ni,x,ix,jx,y,g,q,nlam,a0,a,flog,jerr);
real x(*),y(no),g(no),q(no),a(ni,nlam),a0(nlam),flog(nlam);
integer ix(*),jx(*);
%fortran
real, dimension (:), allocatable :: w,f
%mortran
if minval(y).lt.0.0 < jerr=8888; return;>
allocate(w(1:no),stat=jerr);
allocate(f(1:no),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
w=max(0.0,q); sw=sum(w); if sw.le.0.0 < jerr=9999; go to :done:;>
yb=dot_product(w,y)/sw; fmax=log(huge(y(1))*0.1);
f=f+g;
s=dot_product(w,y*f-exp(sign(min(abs(f),fmax),f)));
flog(lam)=2.0*(sw*yb*(log(yb)-1.0)-s);
>
:done: deallocate(w,f);
return;
end;
subroutine cspdeviance(no,x,ix,jx,y,g,q,nx,nlam,a0,ca,ia,nin,flog,jerr);
real x(*),y(no),g(no),q(no),ca(nx,nlam),a0(nlam),flog(nlam);
integer ix(*),jx(*),nin(nlam),ia(nx);
%fortran
real, dimension (:), allocatable :: w,f
%mortran
if minval(y).lt.0.0 < jerr=8888; return;>
allocate(w(1:no),stat=jerr);
allocate(f(1:no),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
w=max(0.0,q); sw=sum(w); if sw.le.0.0 < jerr=9999; go to :done:;>
yb=dot_product(w,y)/sw; fmax=log(huge(y(1))*0.1);
f=f+g;
s=dot_product(w,y*f-exp(sign(min(abs(f),fmax),f)));
flog(lam)=2.0*(sw*yb*(log(yb)-1.0)-s);
>
:done: deallocate(w,f);
return;
end;
subroutine multelnet
(parm,no,ni,nr,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,isd,jsd,intr,maxit,
lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
real x(no,ni),y(no,nr),w(no),vp(ni),ca(nx,nr,nlam);
real ulam(nlam),a0(nr,nlam),rsq(nlam),alm(nlam),cl(2,ni);
integer jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: vq;
%mortran
if maxval(vp).le.0.0 < jerr=10000; return;>
allocate(vq(1:ni),stat=jerr); if(jerr.ne.0) return;
vq=max(0.0,vp); vq=vq*ni/sum(vq);
call multelnetn(parm,no,ni,nr,x,y,w,jd,vq,cl,ne,nx,nlam,flmin,ulam,thr,isd,
jsd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
deallocate(vq);
return;
end;
subroutine multelnetn (parm,no,ni,nr,x,y,w,jd,vp,cl,ne,nx,nlam,flmin,ulam,thr,
isd,jsd,intr,maxit,lmu,a0,ca,ia,nin,rsq,alm,nlp,jerr);
real vp(ni),x(no,ni),y(no,nr),w(no),ulam(nlam),cl(2,ni);
real ca(nx,nr,nlam),a0(nr,nlam),rsq(nlam),alm(nlam);
integer jd(*),ia(nx),nin(nlam);
%fortran
real, dimension (:), allocatable :: xm,xs,xv,ym,ys
integer, dimension (:), allocatable :: ju
real, dimension (:,:,:), allocatable :: clt
allocate(clt(1:2,1:nr,1:ni),stat=jerr);
%mortran
allocate(xm(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xs(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(ym(1:nr),stat=ierr); jerr=jerr+ierr;
allocate(ys(1:nr),stat=ierr); jerr=jerr+ierr;
allocate(ju(1:ni),stat=ierr); jerr=jerr+ierr;
allocate(xv(1:ni),stat=ierr); jerr=jerr+ierr;
if(jerr.ne.0) return;
call chkvars(no,ni,x,ju);
if(jd(1).gt.0) ju(jd(2:(jd(1)+1)))=0;
if maxval(ju).le.0 < jerr=7777; return;>
call multstandard1(no,ni,nr,x,y,w,isd,jsd,intr,ju,xm,xs,ym,ys,xv,ys0,jerr);
if(jerr.ne.0) return;
>>
if isd.gt.0 < >>>
if jsd.gt.0 < >>>
call multelnet2(parm,ni,nr,ju,vp,clt,y,no,ne,nx,x,nlam,flmin,ulam,thr,maxit,xv,
ys0,lmu,ca,ia,nin,rsq,alm,nlp,jerr);
if(jerr.gt.0) return;