Package 'CondCopulas'

Estimation of the conditional parameters of a parametric conditional copula with discrete conditioning events.

Description

By Sklar's theorem, any conditional distribution function can be written as

$F_{1,2|A}(x_1, x_2) = c_{1,2|A}(F_{1|A}(x_1), F_{2,A}(x_2)),$

where $A$ is an event and $c_{1,2|A}$ is a copula depending on the event $A$. In this function, we assume that we have a partition $A_1,... A_p$ of the probability space, and that for each $k=1,...,p$, the conditional copula is parametric according to the following model

$c_{1,2|Ak} = c_{\theta(Ak)},$

for some parameter $\theta(Ak)$ depending on the realized event $Ak$. This function uses canonical maximum likelihood to estimate $\theta(Ak)$ and the corresponding copulas $c_{1,2|Ak}$.

Usage

bCond.estParamCopula(U1, U2, family, partition)

Arguments

U1	vector of n conditional pseudo-observations of the first conditioned variable.
U2	vector of n conditional pseudo-observations of the second conditioned variable.
family	the family of conditional copulas used for each conditioning event $A_k$. If not of length $p$, it is recycled to match the number of events $p$.
partition	matrix of size n * p, where p is the number of conditioning events that are considered. partition[i,j] should be the indicator of whether the i-th observation belongs or not to the j-th conditioning event

Value

a list of size p containing the p conditional copulas

References

Derumigny, A., & Fermanian, J. D. (2017). About tests of the “simplifying” assumption for conditional copulas. Dependence Modeling, 5(1), 154-197. doi:10.1515/demo-2017-0011

Derumigny, A., & Fermanian, J. D. (2022) Conditional empirical copula processes and generalized dependence measures Electronic Journal of Statistics, 16(2), 5692-5719. doi:10.1214/22-EJS2075

See Also

bCond.pobs for the computation of (conditional) pseudo-observations in this framework.

bCond.simpA.param for a test of the simplifying assumption that all these conditional copulas are equal (assuming they all belong to the same parametric family). bCond.simpA.CKT for a test of the simplifying assumption that all these conditional copulas are equal, based on the equality of conditional Kendall's tau.

Examples

n = 800
Z = stats::runif(n = n)
CKT = 0.2 * as.numeric(Z <= 0.3) +
  0.5 * as.numeric(Z > 0.3 & Z <= 0.5) +
  - 0.8 * as.numeric(Z > 0.5)
simCopula = VineCopula::BiCopSim(N = n,
  par = VineCopula::BiCopTau2Par(CKT, family = 1), family = 1)
X1 = simCopula[,1]
X2 = simCopula[,2]
partition = cbind(Z <= 0.3, Z > 0.3 & Z <= 0.5, Z > 0.5)
condPseudoObs = bCond.pobs(X = cbind(X1, X2), partition = partition)

estimatedCondCopulas = bCond.estParamCopula(
  U1 = condPseudoObs[,1], U2 = condPseudoObs[,2],
  family = 1, partition = partition)
print(estimatedCondCopulas)
# Comparison with the true conditional parameters: 0.2, 0.5, -0.8.

---

Computing the pseudo-observations in case of discrete conditioning events

Description

Let $A_1, ..., A_p$ be $p$ events forming a partition of a probability space and $X_1, ..., X_d$ be $d$ random variables. Assume that we observe $n$ i.i.d. replications of $(X_1, ..., X_d)$, and that for each $i=1, ..., d$,

$V_{i,j|A} = F_{X_j | A_k}(X_{i,j} | A_k),$

we also know which of the $A_k$ was realized. This function computes the pseudo-observations where $k$ is such that the event $A_k$ is realized for the $i$-th observation.

Usage

bCond.pobs(X, partition)

Arguments

X	matrix of size n * d observations of conditioned variables.
partition	matrix of size n * p, where p is the number of conditioning events that are considered. partition[i,k] should be the indicator of whether the i-th observation belongs or not to the k-th conditioning event.

Value

a matrix of size n * d containing the conditional pseudo-observations $V_{i,j|A}$.

References

Derumigny, A., & Fermanian, J. D. (2017). About tests of the “simplifying” assumption for conditional copulas. Dependence Modeling, 5(1), 154-197. doi:10.1515/demo-2017-0011

Derumigny, A., & Fermanian, J. D. (2022) Conditional empirical copula processes and generalized dependence measures Electronic Journal of Statistics, 16(2), 5692-5719. doi:10.1214/22-EJS2075

See Also

bCond.estParamCopula for the estimation of a (conditional) parametric copula model in this framework.

bCond.treeCKT that provides a binary tree based on conditional Kendall's tau and that can be used to derive relevant conditioning events.

Examples

n = 800
Z = stats::runif(n = n)
CKT = 0.2 * as.numeric(Z <= 0.3) +
  0.5 * as.numeric(Z > 0.3 & Z <= 0.5) +
  - 0.8 * as.numeric(Z > 0.5)
simCopula = VineCopula::BiCopSim(N = n,
  par = VineCopula::BiCopTau2Par(CKT, family = 1), family = 1)
X1 = simCopula[,1]
X2 = simCopula[,2]
partition = cbind(Z <= 0.3, Z > 0.3 & Z <= 0.5, Z > 0.5)
condPseudoObs = bCond.pobs(X = cbind(X1, X2),
                           partition = partition)

---

Function for testing the simplifying assumption with data-driven box-type conditioning events

Description

This function takes in parameter the matrix of (observations) of the conditioned variables and either matrixInd, a matrix of indicator variables describing which events occur for which observations

Usage

bCond.simpA.CKT(
  XI,
  XJ = NULL,
  matrixInd = NULL,
  minCut = 0,
  minProb = 0.01,
  minSize = minProb * nrow(XI),
  nPoints_xJ = 10,
  type.quantile = 7,
  verbose = 2,
  methodTree = "doSplit",
  propTree = 0.5,
  methodPvalue = "bootNP",
  nBootstrap = 100
)

Arguments

XI	matrix of size n*p of observations of the conditioned variables.
XJ	matrix of size n*(d-p) containing observations of the conditioning vector.
matrixInd	a matrix of indexes of size (n, N.boxes) describing for each observation i to which box ( = event) it belongs. If it is NULL, then a tree will be estimated to provide relevant boxes (by using bCond.treeCKT()) and then converting to a matrixInd by treeCKT2matrixInd().
minCut	minimum difference in probabilities that is necessary to cut.
minProb	minimum probability of being in one of the node.
minSize	minimum number of observations in each node. This is an alternative to minProb and has priority over it.
nPoints_xJ	number of points in the grid that are considered when choosing the point for splitting the tree.
type.quantile	way of computing the quantiles, see stats::quantile().
verbose	control the text output of the procedure. If verbose = 0, suppress all output. If verbose = 2, the progress of the computation is printed during the computation.
methodTree	method for constructing the tree doSplit some part of the data is used for constructing the tree and the other part for constructing the test statistic using the boxes defined by the estimated tree. The share of the data used for construction the tree is controlled by the parameter propTree. noSplit all of the data is used for both the tree and the test statistic on it. Note that p-values obtained by this method have an upward bias due to the lack of independence between these two steps. Only used if matrixInd is not provided.
propTree	share of observations used to build the tree (the rest of the observations are used for the computation of the p-value). Only used if matrixInd is not provided.
methodPvalue	method for computing the p-value covMatrix by computation of the covariance matrix of the random vector $(\tau_{i,k|X_J \in A_j}, 1\leq,i,k\leq p, 1\leq j \leq m)$. bootNP by the usual non-parametric bootstrap bootInd by the independent bootstrap
nBootstrap	number of bootstrap replications (Only used if methodPvalue is not covMatrix).

Value

a list with the following components

• p.value the estimated p-value.

• stat the test statistic.

• treeCKT the estimated tree if matrixInd is not provided.

• vec_statB the vector of bootstrapped statistics if methodPvalue is not covMatrix.

Author(s)

Alexis Derumigny, Jean-David Fermanian and Aleksey Min

References

Derumigny, A., Fermanian, J. D., & Min, A. (2022). Testing for equality between conditional copulas given discretized conditioning events. Canadian Journal of Statistics. doi:10.1002/cjs.11742

Derumigny, A., & Fermanian, J. D. (2022) Conditional empirical copula processes and generalized dependence measures Electronic Journal of Statistics, 16(2), 5692-5719. doi:10.1214/22-EJS2075

See Also

bCond.simpA.param for a test of this simplifying assumption in a parametric framework.

bCond.treeCKT provides the binary tree that is used in this function (if matrixInd is not provided).

Tests of the simplifying assumption for conditional copulas with a continuous conditioning variable:

• simpA.NP in a nonparametric setting

• simpA.param in a (semi)parametric setting, where the conditional copula belongs to a parametric family, but the conditional margins are estimated arbitrarily through kernel smoothing

• simpA.kendallReg: test based on the constancy of conditional Kendall's tau

Examples

set.seed(1)
n = 200
XJ = MASS::mvrnorm(n = n, mu = c(3,3), Sigma = rbind(c(1, 0.2), c(0.2, 1)))
XI = matrix(nrow = n, ncol = 2)
high_XJ1 = which(XJ[,1] > 4)
XI[high_XJ1, ]  = MASS::mvrnorm(n = length(high_XJ1), mu = c(10,10),
                                Sigma = rbind(c(1, 0.8), c(0.8, 1)))
XI[-high_XJ1, ] = MASS::mvrnorm(n = n - length(high_XJ1), mu = c(8,8),
                                Sigma = rbind(c(1, -0.2), c(-0.2, 1)))

result = bCond.simpA.CKT(XI = XI, XJ = XJ, minSize = 10, verbose = 2,
                         methodTree = "doSplit", nBootstrap = 4)
print(result$p.value)
result2 = bCond.simpA.CKT(XI = XI, XJ = XJ, minSize = 10, verbose = 2,
                          methodTree = "noSplit", nBootstrap = 4)
print(result2$p.value)

---

Test of the assumption that a conditional copulas does not vary through a list of discrete conditioning events

Description

Test of the assumption that a conditional copulas does not vary through a list of discrete conditioning events

Usage

bCond.simpA.param(
  X1,
  X2,
  partition,
  family,
  testStat = "T2c_tau",
  typeBoot = "boot.NP",
  nBootstrap = 100
)

Arguments

X1	vector of n observations of the first conditioned variable.
X2	vector of n observations of the second conditioned variable.
partition	matrix of size n * p, where p is the number of conditioning events that are considered. partition[i,j] should be the indicator of whether the i-th observation belongs or not to the j-th conditioning event.
family	family of parametric copulas used
testStat	test statistic used. Possible choices are T2c_par $\sum_{box} (\theta_0 - \theta(box))^2$ T2c_tau Same as above, except that the copula family is now parametrized by its Kendall's tau instead of its natural parameter.
typeBoot	type of bootstrap used
nBootstrap	number of bootstrap replications

Value

a list containing

• true_stat: the value of the test statistic computed on the whole sample

• vect_statB: a vector of length nBootstrap containing the bootstrapped test statistics.

• p_val: the p-value of the test.

References

Derumigny, A., & Fermanian, J. D. (2017). About tests of the “simplifying” assumption for conditional copulas. Dependence Modeling, 5(1), 154-197. doi:10.1515/demo-2017-0011

Derumigny, A., & Fermanian, J. D. (2022) Conditional empirical copula processes and generalized dependence measures Electronic Journal of Statistics, 16(2), 5692-5719. doi:10.1214/22-EJS2075

See Also

bCond.estParamCopula for the estimation of a (conditional) parametric copula model in this framework.

bCond.simpA.CKT for a test of the simplifying assumption that all these conditional copulas are equal, based on the equality of conditional Kendall's tau (i.e. without any parametric assumption).

Tests of the simplifying assumption for conditional copulas with a continuous conditioning variable:

• simpA.NP in a nonparametric setting

• simpA.param in a (semi)parametric setting, where the conditional copula belongs to a parametric family, but the conditional margins are estimated arbitrarily through kernel smoothing

• simpA.kendallReg: test based on the constancy of conditional Kendall's tau

Examples

n = 800
Z = stats::runif(n = n)
CKT = 0.2 * as.numeric(Z <= 0.3) +
  0.5 * as.numeric(Z > 0.3 & Z <= 0.5) +
  + 0.3 * as.numeric(Z > 0.5)
family = 3
simCopula = VineCopula::BiCopSim(N = n,
  par = VineCopula::BiCopTau2Par(CKT, family = family), family = family)
X1 = simCopula[,1]
X2 = simCopula[,2]
partition = cbind(Z <= 0.3, Z > 0.3 & Z <= 0.5, Z > 0.5)

result = bCond.simpA.param(X1 = X1, X2 = X2, testStat = "T2c_tau",
  partition = partition, family = family, typeBoot = "boot.paramInd")
print(result$p_val)

n = 800
Z = stats::runif(n = n)
CKT = 0.1
family = 3
simCopula = VineCopula::BiCopSim(N = n,
  par = VineCopula::BiCopTau2Par(CKT, family = family), family = family)
X1 = simCopula[,1]
X2 = simCopula[,2]
partition = cbind(Z <= 0.3, Z > 0.3 & Z <= 0.5, Z > 0.5)

result = bCond.simpA.param(X1 = X1, X2 = X2,
  partition = partition, family = family, typeBoot = "boot.NP")
print(result$p_val)

---

Construct a binary tree for the modeling the conditional Kendall's tau

Description

This function takes in parameter two matrices of observations: the first one contains the observations of XI (the conditioned variables) and the second on contains the observations of XJ (the conditioning variables). The goal of this procedure is to find which of the variables in XJ have important influence on the dependence between the components of XI, (measured by the Kendall's tau).

Usage

bCond.treeCKT(
  XI,
  XJ,
  minCut = 0,
  minProb = 0.01,
  minSize = minProb * nrow(XI),
  nPoints_xJ = 10,
  type.quantile = 7,
  verbose = 2
)

Arguments

XI	matrix of size n*p of observations of the conditioned variables.
XJ	matrix of size n*(d-p) containing observations of the conditioning vector.
minCut	minimum difference in probabilities that is necessary to cut.
minProb	minimum probability of being in one of the node.
minSize	minimum number of observations in each node. This is an alternative to minProb and has priority over it.
nPoints_xJ	number of points in the grid that are considered when choosing the point for splitting the tree.
type.quantile	way of computing the quantiles, see stats::quantile().
verbose	control the text output of the procedure. If verbose = 0, suppress all output. If verbose = 2, the progress of the computation is printed during the computation.

Details

The object return by this function is a binary tree. Each leaf of this tree correspond to one event (or, equivalently, one subset of $R^{dim(XJ)}$), and the conditional Kendall's tau conditionally to it.

Value

the estimated tree using the data 'XI, XJ'.

References

Derumigny, A., Fermanian, J. D., & Min, A. (2022). Testing for equality between conditional copulas given discretized conditioning events. Canadian Journal of Statistics. doi:10.1002/cjs.11742

See Also

bCond.simpA.CKT for a test of the simplifying assumption that all these conditional Kendall's tau are equal.

treeCKT2matrixInd for converting this tree to a matrix of indicators of each event. matrixInd2matrixCKT for getting the matrix of estimated conditional Kendall's taus for each event.

CKT.estimate for the estimation of pointwise conditional Kendall's tau, i.e. assuming a continuous conditioning variable $Z$.

Examples

set.seed(1)
n = 400
XJ = MASS::mvrnorm(n = n, mu = c(3,3), Sigma = rbind(c(1, 0.2), c(0.2, 1)))
XI = matrix(nrow = n, ncol = 2)
high_XJ1 = which(XJ[,1] > 4)
XI[high_XJ1, ]  = MASS::mvrnorm(n = length(high_XJ1), mu = c(10,10),
                                Sigma = rbind(c(1, 0.8), c(0.8, 1)))
XI[-high_XJ1, ] = MASS::mvrnorm(n = n - length(high_XJ1), mu = c(8,8),
                                Sigma = rbind(c(1, -0.2), c(-0.2, 1)))

result = bCond.treeCKT(XI = XI, XJ = XJ, minSize = 50, verbose = 2)
# Plotting the corresponding tree using the "DiagrammeR" package
if (requireNamespace("DiagrammeR", quietly = TRUE)){
  plot(result)
}

# Number of observations in the first two children
print(length(data.tree::GetAttribute(result$children[[1]], "condObs")))
print(length(data.tree::GetAttribute(result$children[[2]], "condObs")))

---

Estimation of conditional Kendall's tau between two variables X1 and X2 given Z = z

Description

Let $X_1$ and $X_2$ be two random variables. The goal of this function is to estimate the conditional Kendall's tau (a dependence measure) between $X_1$ and $X_2$ given $Z=z$ for a conditioning variable $Z$. Conditional Kendall's tau between $X_1$ and $X_2$ given $Z=z$ is defined as:

$P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) > 0 | Z_1 = Z_2 = z)$

$- P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) < 0 | Z_1 = Z_2 = z),$

where $(X_{1,1}, X_{1,2}, Z_1)$ and $(X_{2,1}, X_{2,2}, Z_2)$ are two independent and identically distributed copies of $(X_1, X_2, Z)$. In other words, conditional Kendall's tau is the difference between the probabilities of observing concordant and discordant pairs from the conditional law of

$(X_1, X_2) | Z=z.$

This function can use different estimators for conditional Kendall's tau, see the description of the parameter methodEstimation for a complete list of possibilities.

Usage

CKT.estimate(
  X1 = NULL, X2 = NULL, Z = NULL,
  newZ = Z, methodEstimation, h,
  listPhi = if(methodEstimation == "kendallReg")
               {list( function(x){return(x)}   ,
                      function(x){return(x^2)} ,
                      function(x){return(x^3)} )
               } else {list(identity)} ,
  ... ,
  observedX1 = NULL, observedX2 = NULL, observedZ = NULL )

Arguments

X1	a vector of $n$ observations of the first variable
X2	a vector of $n$ observations of the second variable
Z	a vector of $n$ observations of the conditioning variable, or a matrix with $n$ rows of observations of the conditioning vector (if $Z$ is multivariate).
newZ	the new values for the conditioning variable $Z$ at which the conditional Kendall's tau should be estimated. If observedZ is a vector, then newZ must be a vector as well. If observedZ is a matrix, then newZ must be a matrix as well, with the same number of columns ( = the dimension of $Z$).
methodEstimation	method for estimating the conditional Kendall's tau. Possible estimation methods are: "kernel": kernel smoothing, as described in (Derumigny, & Fermanian (2019a)) "kendallReg": regression-type model, as described in (Derumigny, & Fermanian (2020)) "tree", "randomForest", "logit", and "neuralNetwork": use the relationship between conditional Kendall's tau and classification problems to use the respective classification algorithms for the estimation of conditional Kendall's tau, as described in (Derumigny, & Fermanian (2019b))
h	the bandwidth
listPhi	the list of transformations to be applied to the conditioning variable $Z$ (in case of regression-type models).
...	other parameters passed to the estimating functions CKT.fit.tree, CKT.fit.randomForest, CKT.fit.GLM, CKT.fit.nNets, CKT.predict.kNN, CKT.kernel and CKT.kendallReg.fit.
observedX1, observedX2, observedZ	old parameter names for X1, X2, Z. Support for this will be removed at a later version.

Value

the vector of estimated conditional Kendall's tau at each of the observations of newZ.

References

Derumigny, A., & Fermanian, J. D. (2019a). A classification point-of-view about conditional Kendall’s tau. Computational Statistics & Data Analysis, 135, 70-94. doi:10.1016/j.csda.2019.01.013

Derumigny, A., & Fermanian, J. D. (2019b). On kernel-based estimation of conditional Kendall’s tau: finite-distance bounds and asymptotic behavior. Dependence Modeling, 7(1), 292-321. doi:10.1515/demo-2019-0016

Derumigny, A., & Fermanian, J. D. (2020). On Kendall’s regression. Journal of Multivariate Analysis, 178, 104610. doi:10.1016/j.jmva.2020.104610

See Also

the specialized functions for estimating conditional Kendall's tau for each method: CKT.fit.tree, CKT.fit.randomForest, CKT.fit.GLM, CKT.fit.nNets, CKT.predict.kNN, CKT.fit.randomForest, CKT.kernel and CKT.kendallReg.fit.

See also the nonparametric estimator of conditional copula models estimateNPCondCopula, and the parametric estimators of conditional copula models estimateParCondCopula.

In the case where $Z$ is discrete or in the case of discrete conditioning events, see bCond.treeCKT.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 300
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

newZ = seq(2,10,by = 0.1)
h = 0.1
estimatedCKT_tree <- CKT.estimate(
  X1 = X1, X2 = X2, Z = Z,
  newZ = newZ,
  methodEstimation = "tree", h = h)

estimatedCKT_rf <- CKT.estimate(
  X1 = X1, X2 = X2, Z = Z,
  newZ = newZ,
  methodEstimation = "randomForest", h = h)

estimatedCKT_GLM <- CKT.estimate(
  X1 = X1, X2 = X2, Z = Z,
  newZ = newZ,
  methodEstimation = "logit", h = h,
  listPhi = list(function(x){return(x)}, function(x){return(x^2)},
                 function(x){return(x^3)}) )

estimatedCKT_kNN <- CKT.estimate(
  X1 = X1, X2 = X2, Z = Z,
  newZ = newZ,
  methodEstimation = "nearestNeighbors", h = h,
  number_nn = c(50,80, 100, 120,200),
  partition = 4
  )

estimatedCKT_nNet <- CKT.estimate(
  X1 = X1, X2 = X2, Z = Z,
  newZ = newZ,
  methodEstimation = "neuralNetwork", h = h,
  )

estimatedCKT_kernel <- CKT.estimate(
  X1 = X1, X2 = X2, Z = Z,
  newZ = newZ,
  methodEstimation = "kernel", h = h,
  )

estimatedCKT_kendallReg <- CKT.estimate(
   X1 = X1, X2 = X2, Z = Z,
   newZ = newZ,
   methodEstimation = "kendallReg", h = h)

# Comparison between true Kendall's tau (in black)
# and estimated Kendall's tau (in other colors)
trueConditionalTau = -0.9 + 1.8 * pnorm(newZ, mean = 5, sd = 2)
plot(newZ, trueConditionalTau , col="black",
   type = "l", ylim = c(-1, 1))
lines(newZ, estimatedCKT_tree, col = "red")
lines(newZ, estimatedCKT_rf, col = "blue")
lines(newZ, estimatedCKT_GLM, col = "green")
lines(newZ, estimatedCKT_kNN, col = "purple")
lines(newZ, estimatedCKT_nNet, col = "coral")
lines(newZ, estimatedCKT_kernel, col = "skyblue")
lines(newZ, estimatedCKT_kendallReg, col = "darkgreen")

---

Estimation of conditional Kendall's taus by penalized GLM

Description

The function CKT.fit.GLM fits a regression model for the conditional Kendall's tau $\tau_{1,2|Z}$ between two variables $X_1$ and $X_2$ conditionally to some predictors $Z$. More precisely, this function fits the model

$\tau_{1,2|Z} = 2 * \Lambda( \beta_0 + \beta_1 \phi_1(Z) + ... + \beta_p \phi_p(Z) )$

for a link function $\Lambda$, and $p$ real-valued functions $\phi_1, ..., \phi_p$. The function CKT.predict.GLM predicts the values of conditional Kendall's tau for some values of the conditioning variable $Z$.

Usage

CKT.fit.GLM(
  datasetPairs,
  designMatrix = datasetPairs[, 2:(ncol(datasetPairs) - 3), drop = FALSE],
  link = "logit",
  ...
)

CKT.predict.GLM(fit, newZ)

Arguments

datasetPairs	the matrix of pairs and corresponding values of the kernel as provided by datasetPairs.
designMatrix	the matrix of predictor to be used for the fitting of the model. It should have the same number of rows as the datasetPairs.
link	link function, can be one of logit, probit, cloglog, cauchit).
...	other parameters passed to ordinalNet::ordinalNet().
fit	result of a call to CKT.fit.GLM
newZ	new matrix of observations of the conditioning vector $Z$, with the same number of variables and same names as the designMatrix that was used to fit the GLM.

Value

CKT.fit.GLM returns the fitted GLM, an object with S3 class ordinalNet.

CKT.predict.GLM returns a vector of (predicted) conditional Kendall's taus of the same size as the number of rows of the matrix newZ.

References

Derumigny, A., & Fermanian, J. D. (2019). A classification point-of-view about conditional Kendall’s tau. Computational Statistics & Data Analysis, 135, 70-94. (Algorithm 2) doi:10.1016/j.csda.2019.01.013

See Also

See also other estimators of conditional Kendall's tau: CKT.fit.tree, CKT.fit.randomForest, CKT.fit.nNets, CKT.predict.kNN, CKT.kernel, CKT.kendallReg.fit, and the more general wrapper CKT.estimate.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 400
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = 2*plogis(-1 + 0.8*Z - 0.1*Z^2) - 1
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

datasetP = datasetPairs(X1 = X1, X2 = X2, Z = Z, h = 0.07, cut = 0.9)
designMatrix = cbind(datasetP[,2], datasetP[,2]^2)
fitCKT_GLM <- CKT.fit.GLM(
  datasetPairs = datasetP, designMatrix = designMatrix,
  maxiterOut = 10, maxiterIn = 5)
print(coef(fitCKT_GLM))
# These are rather close to the true coefficients -1, 0.8, -0.1
# used to generate the data above.

newZ = seq(2,10,by = 0.1)
estimatedCKT_GLM = CKT.predict.GLM(
  fit = fitCKT_GLM, newZ = cbind(newZ, newZ^2))

# Comparison between true Kendall's tau (in red)
# and estimated Kendall's tau (in black)
trueConditionalTau = 2*plogis(-1 + 0.8*newZ - 0.1*newZ^2) - 1
plot(newZ, trueConditionalTau , col="red",
   type = "l", ylim = c(-1, 1))
lines(newZ, estimatedCKT_GLM)

---

Estimation of conditional Kendall's taus by model averaging of neural networks

Description

Let $X_1$ and $X_2$ be two random variables. The goal of this function is to estimate the conditional Kendall's tau (a dependence measure) between $X_1$ and $X_2$ given $Z=z$ for a conditioning variable $Z$. Conditional Kendall's tau between $X_1$ and $X_2$ given $Z=z$ is defined as:

$P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) > 0 | Z_1 = Z_2 = z)$

$- P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) < 0 | Z_1 = Z_2 = z),$

where $(X_{1,1}, X_{1,2}, Z_1)$ and $(X_{2,1}, X_{2,2}, Z_2)$ are two independent and identically distributed copies of $(X_1, X_2, Z)$. In other words, conditional Kendall's tau is the difference between the probabilities of observing concordant and discordant pairs from the conditional law of

$(X_1, X_2) | Z=z.$

This function estimates conditional Kendall's tau using neural networks. This is possible by the relationship between estimation of conditional Kendall's tau and classification problems (see Derumigny and Fermanian (2019)): estimation of conditional Kendall's tau is equivalent to the prediction of concordance in the space of pairs of observations.

Usage

CKT.fit.nNets(
  datasetPairs,
  designMatrix = datasetPairs[, 2:(ncol(datasetPairs) - 3), drop = FALSE],
  vecSize = rep(3, times = 10),
  nObs_per_NN = 0.9 * nrow(designMatrix),
  verbose = 1
)

Arguments

datasetPairs	the matrix of pairs and corresponding values of the kernel as provided by datasetPairs.
designMatrix	the matrix of predictor to be used for the fitting of the tree
vecSize	vector with the number of neurons for each network
nObs_per_NN	number of observations used for each neural network.
verbose	a number indicated what to print 0: nothing printed at all. 1: a message is printed at the convergence of each neural network. 2: details are printed for each optimization of each network.

Value

CKT.fit.nNets returns a list of the fitted neural networks

References

Derumigny, A., & Fermanian, J. D. (2019). A classification point-of-view about conditional Kendall’s tau. Computational Statistics & Data Analysis, 135, 70-94. (Algorithm 7) doi:10.1016/j.csda.2019.01.013

See Also

See also other estimators of conditional Kendall's tau: CKT.fit.tree, CKT.fit.randomForest, CKT.fit.GLM, CKT.predict.kNN, CKT.kernel, CKT.kendallReg.fit, and the more general wrapper CKT.estimate.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 800
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

newZ = seq(2,10,by = 0.1)
datasetP = datasetPairs(X1 = X1, X2 = X2, Z = Z, h = 0.07, cut = 0.9)

fitCKT_nets <- CKT.fit.nNets(datasetPairs = datasetP)
estimatedCKT_nNets <- CKT.predict.nNets(
  fit = fitCKT_nets, newZ = matrix(newZ, ncol = 1))

# Comparison between true Kendall's tau (in black)
# and estimated Kendall's tau (in red)
trueConditionalTau = -0.9 + 1.8 * pnorm(newZ, mean = 5, sd = 2)
plot(newZ, trueConditionalTau , col="black",
   type = "l", ylim = c(-1, 1))
lines(newZ, estimatedCKT_nNets, col = "red")

---

Fit a Random Forest that can be used for the estimation of conditional Kendall's tau.

Description

Let $X_1$ and $X_2$ be two random variables. The goal of this function is to estimate the conditional Kendall's tau (a dependence measure) between $X_1$ and $X_2$ given $Z=z$ for a conditioning variable $Z$. Conditional Kendall's tau between $X_1$ and $X_2$ given $Z=z$ is defined as:

$P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) > 0 | Z_1 = Z_2 = z)$

$- P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) < 0 | Z_1 = Z_2 = z),$

where $(X_{1,1}, X_{1,2}, Z_1)$ and $(X_{2,1}, X_{2,2}, Z_2)$ are two independent and identically distributed copies of $(X_1, X_2, Z)$. In other words, conditional Kendall's tau is the difference between the probabilities of observing concordant and discordant pairs from the conditional law of

$(X_1, X_2) | Z=z.$

These functions estimate and predict conditional Kendall's tau using a random forest. This is possible by the relationship between estimation of conditional Kendall's tau and classification problems (see Derumigny and Fermanian (2019)): estimation of conditional Kendall's tau is equivalent to the prediction of concordance in the space of pairs of observations.

Usage

CKT.fit.randomForest(
  datasetPairs,
  designMatrix = data.frame(x = datasetPairs[, 2:(ncol(datasetPairs) - 3)]),
  n,
  nTree = 10,
  mindev = 0.008,
  mincut = 0,
  nObs_per_Tree = ceiling(0.8 * n),
  nVar_per_Tree = ceiling(0.8 * (ncol(datasetPairs) - 4)),
  verbose = FALSE,
  nMaxDepthAllowed = 10
)

CKT.predict.randomForest(fit, newZ)

Arguments

datasetPairs	the matrix of pairs and corresponding values of the kernel as provided by datasetPairs.
designMatrix	the matrix of predictor to be used for the fitting of the tree
n	the original sample size of the dataset
nTree	number of trees of the Random Forest.
mindev	a factor giving the minimum deviation for a node to be splitted. See tree::tree.control() for more details.
mincut	the minimum number of observations (of pairs) in a node See tree::tree.control() for more details.
nObs_per_Tree	number of observations kept in each tree.
nVar_per_Tree	number of variables kept in each tree.
verbose	if TRUE, a message is printed after fitting each tree.
nMaxDepthAllowed	the maximum number of errors of type "the tree cannot be fitted" or "is too deep" before stopping the procedure.
fit	result of a call to CKT.fit.randomForest.
newZ	new matrix of observations, with the same number of variables. and same names as the designMatrix that was used to fit the Random Forest.

Value

a list with two components

• list_tree a list of size nTree composed of all the fitted trees.

• list_variables a list of size nTree composed of the (predictor) variables for each tree.

CKT.predict.randomForest returns a vector of (predicted) conditional Kendall's taus of the same size as the number of rows of the newZ.

References

Derumigny, A., & Fermanian, J. D. (2019). A classification point-of-view about conditional Kendall’s tau. Computational Statistics & Data Analysis, 135, 70-94. (Algorithm 4) doi:10.1016/j.csda.2019.01.013

Examples

# We simulate from a conditional copula
set.seed(1)
N = 800
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

datasetP = datasetPairs(X1 = X1, X2 = X2, Z = Z, h = 0.07, cut = 0.9)
est_RF = CKT.fit.randomForest(datasetPairs = datasetP, n = N,
  mindev = 0.008)

newZ = seq(1,10,by = 0.1)
prediction = CKT.predict.randomForest(fit = est_RF,
   newZ = data.frame(x=newZ))
# Comparison between true Kendall's tau (in red)
# and estimated Kendall's tau (in black)
plot(newZ, prediction, type = "l", ylim = c(-1,1))
lines(newZ, -0.9 + 1.8 * pnorm(newZ, mean = 5, sd = 2), col="red")

---

Estimation of conditional Kendall's taus using a classification tree

Description

Let $X_1$ and $X_2$ be two random variables. The goal of this function is to estimate the conditional Kendall's tau (a dependence measure) between $X_1$ and $X_2$ given $Z=z$ for a conditioning variable $Z$. Conditional Kendall's tau between $X_1$ and $X_2$ given $Z=z$ is defined as:

$P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) > 0 | Z_1 = Z_2 = z)$

$- P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) < 0 | Z_1 = Z_2 = z),$

where $(X_{1,1}, X_{1,2}, Z_1)$ and $(X_{2,1}, X_{2,2}, Z_2)$ are two independent and identically distributed copies of $(X_1, X_2, Z)$. In other words, conditional Kendall's tau is the difference between the probabilities of observing concordant and discordant pairs from the conditional law of

$(X_1, X_2) | Z=z.$

These functions estimate and predict conditional Kendall's tau using a classification tree. This is possible by the relationship between estimation of conditional Kendall's tau and classification problems (see Derumigny and Fermanian (2019)): estimation of conditional Kendall's tau is equivalent to the prediction of concordance in the space of pairs of observations.

Usage

CKT.fit.tree(datasetPairs, mindev = 0.008, mincut = 0)

CKT.predict.tree(fit, newZ)

Arguments

datasetPairs	the matrix of pairs and corresponding values of the kernel as provided by datasetPairs.
mindev	a factor giving the minimum deviation for a node to be splitted. See tree::tree.control() for more details.
mincut	the minimum number of observations (of pairs) in a node See tree::tree.control() for more details.
fit	result of a call to CKT.fit.tree
newZ	new matrix of observations, with the same number of variables. and same names as the designMatrix that was used to fit the tree.

Value

CKT.fit.tree returns the fitted tree.

CKT.predict.tree returns a vector of (predicted) conditional Kendall's taus of the same size as the number of rows of newZ.

References

Derumigny, A., & Fermanian, J. D. (2019). A classification point-of-view about conditional Kendall’s tau. Computational Statistics & Data Analysis, 135, 70-94. (Section 3.2) doi:10.1016/j.csda.2019.01.013

See Also

See also other estimators of conditional Kendall's tau: CKT.fit.nNets, CKT.fit.randomForest, CKT.fit.GLM, CKT.predict.kNN, CKT.kernel, CKT.kendallReg.fit, and the more general wrapper CKT.estimate.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 800
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

datasetP = datasetPairs(X1 = X1, X2 = X2, Z = Z, h = 0.07, cut = 0.9)
est_Tree = CKT.fit.tree(datasetPairs = datasetP, mindev = 0.008)
print(est_Tree)

newZ = seq(1,10,by = 0.1)
prediction = CKT.predict.tree(fit = est_Tree, newZ = data.frame(x=newZ))
# Comparison between true Kendall's tau (in red)
# and estimated Kendall's tau (in black)
plot(newZ, prediction, type = "l", ylim = c(-1,1))
lines(newZ, -0.9 + 1.8 * pnorm(newZ, mean = 5, sd = 2), col="red")

---

Choose the bandwidth for kernel estimation of conditional Kendall's tau using cross-validation

Description

Let $X_1$ and $X_2$ be two random variables. The goal here is to estimate the conditional Kendall's tau (a dependence measure) between $X_1$ and $X_2$ given $Z=z$ for a conditioning variable $Z$. Conditional Kendall's tau between $X_1$ and $X_2$ given $Z=z$ is defined as:

$P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) > 0 | Z_1 = Z_2 = z)$

$- P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) < 0 | Z_1 = Z_2 = z),$

where $(X_{1,1}, X_{1,2}, Z_1)$ and $(X_{2,1}, X_{2,2}, Z_2)$ are two independent and identically distributed copies of $(X_1, X_2, Z)$. For this, a kernel-based estimator is used, as described in (Derumigny & Fermanian (2019)). These functions aims at finding the best bandwidth h among a given range_h by cross-validation. They use either:

• leave-one-out cross-validation: function CKT.hCV.l1out

• or K-folds cross-validation: function CKT.hCV.Kfolds

Usage

CKT.hCV.l1out(
  X1 = NULL,
  X2 = NULL,
  Z = NULL,
  range_h,
  matrixSignsPairs = NULL,
  nPairs = 10 * length(X1),
  typeEstCKT = "wdm",
  kernel.name = "Epa",
  progressBar = TRUE,
  verbose = FALSE,
  observedX1 = NULL,
  observedX2 = NULL,
  observedZ = NULL
)

CKT.hCV.Kfolds(
  X1,
  X2,
  Z,
  ZToEstimate,
  range_h,
  matrixSignsPairs = NULL,
  typeEstCKT = "wdm",
  kernel.name = "Epa",
  Kfolds = 5,
  progressBar = TRUE,
  verbose = FALSE,
  observedX1 = NULL,
  observedX2 = NULL,
  observedZ = NULL
)

Arguments

X1	a vector of n observations of the first variable
X2	a vector of n observations of the second variable
Z	vector of observed values of Z. If Z is multivariate, then this is a matrix whose rows correspond to the observations of Z
range_h	vector containing possible values for the bandwidth.
matrixSignsPairs	square matrix of signs of all pairs, produced by computeMatrixSignPairs(observedX1, observedX2). Only needed if typeEstCKT is not the default 'wdm'.
nPairs	number of pairs used in the cross-validation criteria.
typeEstCKT	type of estimation of the conditional Kendall's tau.
kernel.name	name of the kernel used for smoothing. Possible choices are "Gaussian" (Gaussian kernel) and "Epa" (Epanechnikov kernel).
progressBar	if TRUE, a progressbar for each h is displayed to show the progress of the computation.
verbose	if TRUE, print the score of each h during the procedure.
observedX1, observedX2, observedZ	old parameter names for X1, X2, Z. Support for this will be removed at a later version.
ZToEstimate	vector of fixed conditioning values at which the difference between the two conditional Kendall's tau should be computed. Can also be a matrix whose lines are the conditioning vectors at which the difference between the two conditional Kendall's tau should be computed.
Kfolds	number of subsamples used.

Value

Both functions return a list with two components:

• hCV: the chosen bandwidth

• scores: vector of the same length as range_h giving the value of the CV criteria for each of the h tested. Lower score indicates a better fit.

References

Derumigny, A., & Fermanian, J. D. (2019). On kernel-based estimation of conditional Kendall’s tau: finite-distance bounds and asymptotic behavior. Dependence Modeling, 7(1), 292-321. Page 296, Equation (4). doi:10.1515/demo-2019-0016

See Also

CKT.kernel for the corresponding estimator of conditional Kendall's tau by kernel smoothing.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 200
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

newZ = seq(2,10,by = 0.1)
range_h = 3:10

resultCV <- CKT.hCV.l1out(X1 = X1, X2 = X2, Z = Z,
                          range_h = range_h, nPairs = 100)

resultCV <- CKT.hCV.Kfolds(X1 = X1, X2 = X2, Z = Z,
                           range_h = range_h, ZToEstimate = newZ)

plot(range_h, resultCV$scores, type = "b")

---

Fit Kendall's regression, a GLM-type model for conditional Kendall's tau

Description

The function CKT.kendallReg.fit fits a regression-type model for the conditional Kendall's tau between two variables $X_1$ and $X_2$ conditionally to some predictors Z. More precisely, it fits the model

$\Lambda(\tau_{X_1, X_2 | Z = z}) = \sum_{j=1}^{p'} \beta_j \psi_j(z),$

where $\tau_{X_1, X_2 | Z = z}$ is the conditional Kendall's tau between $X_1$ and $X_2$ conditionally to $Z=z$, $\Lambda$ is a function from $]-1, 1]$ to $R$, $(\beta_1, \dots, \beta_p)$ are unknown coefficients to be estimated and $\psi_1, \dots, \psi_{p'})$ are a dictionary of functions. To estimate $beta$, we used the penalized estimator which is defined as the minimizer of the following criteria

$\frac{1}{2n'} \sum_{i=1}^{n'} [\Lambda(\hat\tau_{X_1, X_2 | Z = z_i}) - \sum_{j=1}^{p'} \beta_j \psi_j(z_i)]^2 + \lambda * |\beta|_1,$

where the $z_i$ are a second sample (here denoted by ZToEstimate).

The function CKT.kendallReg.predict predicts the conditional Kendall's tau between two variables $X_1$ and $X_2$ given $Z=z$ for some new values of $z$.

Usage

CKT.kendallReg.fit(
  X1 = NULL,
  X2 = NULL,
  Z = NULL,
  ZToEstimate,
  designMatrixZ = cbind(ZToEstimate, ZToEstimate^2, ZToEstimate^3),
  newZ = designMatrixZ,
  h_kernel,
  Lambda = identity,
  Lambda_inv = identity,
  lambda = NULL,
  Kfolds_lambda = 10,
  l_norm = 1,
  h_lambda = h_kernel,
  ...,
  observedX1 = NULL,
  observedX2 = NULL,
  observedZ = NULL
)

CKT.kendallReg.predict(fit, newZ, lambda = NULL, Lambda_inv = identity)

Arguments

X1	a vector of n observations of the first variable $X_1$.
X2	a vector of n observations of the second variable $X_2$.
Z	a vector of n observations of the conditioning variable, or a matrix with n rows of observations of the conditioning vector (if $Z$ is multivariate).
ZToEstimate	the intermediary dataset of observations of $Z$ at which the conditional Kendall's tau should be estimated.
designMatrixZ	the transformation of the ZToEstimate that will be used as predictors. By default, no transformation is applied.
newZ	the new observations of the conditioning variable.
h_kernel	bandwidth used for the first step of kernel smoothing.
Lambda	the function to be applied on conditional Kendall's tau. By default, the identity function is used.
Lambda_inv	the functional inverse of Lambda. By default, the identity function is used.
lambda	the regularization parameter. If NULL, then it is chosen by K-fold cross validation. Internally, cross-validation is performed by the function CKT.KendallReg.LambdaCV.
Kfolds_lambda	the number of folds used in the cross-validation procedure to choose lambda.
l_norm	type of norm used for selection of the optimal lambda by cross-validation. l_norm=1 corresponds to the sum of absolute values of differences between predicted and estimated conditional Kendall's tau while l_norm=2 corresponds to the sum of squares of differences.
h_lambda	the smoothing bandwidth used in the cross-validation procedure to choose lambda.
...	other arguments to be passed to CKT.kernel for the first step (kernel-based) estimator of conditional Kendall's tau.
observedX1, observedX2, observedZ	old parameter names for X1, X2, Z. Support for this will be removed at a later version.
fit	the fitted model, obtained by a call to CKT.kendallReg.fit.

Value

The function CKT.kendallReg.fit returns a list with the following components:

• estimatedCKT: the estimated CKT at the new data points newZ.

• fit: the fitted model, of S3 class glmnet (see glmnet::glmnet for more details).

• lambda: the value of the penalized parameter used. (i.e. either the one supplied by the user or the one determined by cross-validation)

CKT.kendallReg.predict returns the predicted values of conditional Kendall's tau.

References

Derumigny, A., & Fermanian, J. D. (2020). On Kendall’s regression. Journal of Multivariate Analysis, 178, 104610. doi:10.1016/j.jmva.2020.104610

See Also

See also other estimators of conditional Kendall's tau: CKT.fit.tree, CKT.fit.randomForest, CKT.fit.nNets, CKT.predict.kNN, CKT.kernel, CKT.fit.GLM, and the more general wrapper CKT.estimate.

See also the test of the simplifying assumption that a conditional copula does not depend on the value of the conditioning variable using the nullity of Kendall's regression coefficients: simpA.kendallReg.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 400
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

newZ = seq(2, 10, by = 0.1)
estimatedCKT_kendallReg <- CKT.kendallReg.fit(
   X1 = X1, X2 = X2, Z = Z,
   ZToEstimate = newZ, h_kernel = 0.07)

coef(estimatedCKT_kendallReg$fit,
     s = estimatedCKT_kendallReg$lambda)

# Comparison between true Kendall's tau (in black)
# and estimated Kendall's tau (in red)
trueConditionalTau = -0.9 + 1.8 * pnorm(newZ, mean = 5, sd = 2)
plot(newZ, trueConditionalTau , col="black",
   type = "l", ylim = c(-1, 1))
lines(newZ, estimatedCKT_kendallReg$estimatedCKT, col = "red")

---

Kendall's regression: choice of the penalization parameter by K-folds cross-validation

Description

In this model, three variables $X_1$, $X_2$ and $Z$ are observed. We try to model the conditional Kendall's tau between $X_1$ and $X_2$ conditionally to $Z=z$, as follows:

$\Lambda(\tau_{X_1, X_2 | Z = z}) = \sum_{i=1}^{p'} \beta_i \psi_i(z),$

where $\tau_{X_1, X_2 | Z = z}$ is the conditional Kendall's tau between $X_1$ and $X_2$ conditionally to $Z=z$, $\Lambda$ is a function from $]-1, 1[]$ to $R$, $(\beta_1, \dots, \beta_p)$ are unknown coefficients to be estimated and $\psi_1, \dots, \psi_{p'})$ are a dictionary of functions. To estimate $beta$, we used the penalized estimator which is defined as the minimizer of the following criteria

$\frac{1}{2n'} \sum_{i=1}^{n'} [\Lambda(\hat\tau_{X_1, X_2 | Z = z}) - \sum_{j=1}^{p'} \beta_j \psi_j(z)]^2 + \lambda * |\beta|_1.$

This function chooses the penalization parameter $lambda$ by cross-validation.

Usage

CKT.KendallReg.LambdaCV(
  X1 = NULL,
  X2 = NULL,
  Z = NULL,
  ZToEstimate,
  designMatrixZ = cbind(ZToEstimate, ZToEstimate^2, ZToEstimate^3),
  typeEstCKT = 4,
  h_lambda,
  Lambda = identity,
  kernel.name = "Epa",
  Kfolds_lambda = 10,
  l_norm = 1,
  matrixSignsPairs = NULL,
  progressBars = "global",
  observedX1 = NULL,
  observedX2 = NULL,
  observedZ = NULL
)

Arguments

X1	a vector of n observations of the first variable $X_1$.
X2	a vector of n observations of the second variable $X_2$.
Z	a vector of n observations of the conditioning variable, or a matrix with n rows of observations of the conditioning vector (if $Z$ is multivariate).
ZToEstimate	the new data of observations of Z at which the conditional Kendall's tau should be estimated.
designMatrixZ	the transformation of the ZToEstimate that will be used as predictors. By default, no transformation is applied.
typeEstCKT	type of estimation of the conditional Kendall's tau.
h_lambda	the smoothing bandwidth used in the cross-validation procedure to choose lambda.
Lambda	the function to be applied on conditional Kendall's tau. By default, the identity function is used.
kernel.name	name of the kernel. Possible choices are "Gaussian" (Gaussian kernel) and "Epa" (Epanechnikov kernel).
Kfolds_lambda	the number of folds used in the cross-validation procedure to choose lambda.
l_norm	type of norm used for selection of the optimal lambda. l_norm=1 corresponds to the sum of absolute values of differences between predicted and estimated conditional Kendall's tau while l_norm=2 corresponds to the sum of squares of differences.
matrixSignsPairs	the results of a call to computeMatrixSignPairs (if already computed). If NULL (the default value), the matrixSignsPairs will be computed again from the data.
progressBars	should progress bars be displayed? Possible values are "none": no progress bar at all. "global": only one global progress bar (default behavior) "eachStep": uses a global progress bar + one progress bar for each kernel smoothing step.
observedX1, observedX2, observedZ	old parameter names for X1, X2, Z. Support for this will be removed at a later version.

Value

A list with the following components

• lambdaCV: the chosen value of the penalization parameters lambda.

• vectorLambda: a vector containing the values of lambda that have been compared.

• vectorMSEMean: the estimated MSE for each value of lambda in vectorLambda

• vectorMSESD: the estimated standard deviation of the MSE for each lambda. It can be used to construct confidence intervals for estimates of the MSE given by vectorMSEMean.

References

Derumigny, A., & Fermanian, J. D. (2020). On Kendall’s regression. Journal of Multivariate Analysis, 178, 104610.

See Also

the main fitting function CKT.kendallReg.fit.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 400
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

newZ = seq(2, 10, by = 0.1)
result <- CKT.KendallReg.LambdaCV(X1 = X1, X2 = X2, Z = Z,
                                  ZToEstimate = newZ, h_lambda = 2)

plot(x = result$vectorLambda, y = result$vectorMSEMean,
     type = "l", log = "x")

---

Estimation of conditional Kendall's tau using kernel smoothing

Description

Let $X_1$ and $X_2$ be two random variables. The goal of this function is to estimate the conditional Kendall's tau (a dependence measure) between $X_1$ and $X_2$ given $Z=z$ for a conditioning variable $Z$. Conditional Kendall's tau between $X_1$ and $X_2$ given $Z=z$ is defined as:

$P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) > 0 | Z_1 = Z_2 = z)$

$- P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) < 0 | Z_1 = Z_2 = z),$

where $(X_{1,1}, X_{1,2}, Z_1)$ and $(X_{2,1}, X_{2,2}, Z_2)$ are two independent and identically distributed copies of $(X_1, X_2, Z)$. For this, a kernel-based estimator is used, as described in (Derumigny, & Fermanian (2019)).

Usage

CKT.kernel(
  X1 = NULL,
  X2 = NULL,
  Z = NULL,
  newZ,
  h,
  kernel.name = "Epa",
  methodCV = "Kfolds",
  Kfolds = 5,
  nPairs = 10 * length(observedX1),
  typeEstCKT = "wdm",
  progressBar = TRUE,
  observedX1 = NULL,
  observedX2 = NULL,
  observedZ = NULL
)

Arguments

X1	a vector of n observations of the first variable (or a 1-column matrix)
X2	a vector of n observations of the second variable (or a 1-column matrix)
Z	a vector of n observations of the conditioning variable, or a matrix with n rows of observations of the conditioning vector
newZ	the new data of observations of Z at which the conditional Kendall's tau should be estimated.
h	the bandwidth used for kernel smoothing. If this is a vector, then cross-validation is used following the method given by argument methodCV to choose the best bandwidth before doing the estimation.
kernel.name	name of the kernel used for smoothing. Possible choices are "Gaussian" (Gaussian kernel) and "Epa" (Epanechnikov kernel).
methodCV	method used for the cross-validation. Possible choices are "leave-one-out" and "Kfolds".
Kfolds	number of subsamples used, if methodCV = "Kfolds".
nPairs	number of pairs used in the cross-validation criteria, if methodCV = "leave-one-out".
typeEstCKT	type of estimation of the conditional Kendall's tau. Possible choices are 1 and 3 produced biased estimators. 2 does not attain the full range $[-1,1]$. Therefore these 3 choices are not recommended for applications on real data. 4 is an improved version of 1,2,3 that has less bias and attains the full range $[-1,1]$. "wdm" is the default version and produces the same results as 4 when they are no ties in the data.
progressBar	control the display of progress bars. Possible choices are: 0 no progress bar is displayed 1 a general progress bar is displayed 2 and larger values: a general progress bar is displayed, and additionally, a progressbar for each value of h is displayed to show the progress of the computation. This only applies when the bandwidth is chosen by cross-validation (i.e. when h is a vector).
observedX1, observedX2, observedZ	old parameter names for X1, X2, Z. Support for this will be removed at a later version.

Details

Choice of the bandwidth h. The choice of the bandwidth must be done carefully. In the univariate case, the default kernel (Epanechnikov kernel) has a support on $[-1,1]$, so for a bandwidth h, estimation of conditional Kendall's tau at $Z=z$ will only use points for which $Z_i \in [z \pm h]$. As usual in nonparametric estimation, h should not be too small (to avoid having a too large variance) and should not be large (to avoid having a too large bias).

We recommend that for each $z$ for which the conditional Kendall's tau $\tau_{X_1, X_2 | Z=z}$ is estimated, the set $\{i: Z_i \in [z \pm h] \}$ should contain at least 20 points and not more than 30% of the points of the whole dataset. Note that for a consistent estimation, as the sample size $n$ tends to the infinity, h should tend to $0$ while the size of the set $\{i: Z_i \in [z \pm h]\}$ should also tend to the infinity. Indeed the conditioning points should be closer and closer to the point of interest $z$ (small h) and more and more numerous (h tending to 0 slowly enough).

In the multivariate case, similar recommendations can be made. Because of the curse of dimensionality, a larger sample will be necessary to reach the same level of precision as in the univariate case.

Value

a list with two components

• estimatedCKT the vector of size NROW(newZ) containing the values of the estimated conditional Kendall's tau.

• finalh the bandwidth h that was finally used for kernel smoothing (either the one specified by the user or the one chosen by cross-validation if multiple bandwidths were given.)

References

Derumigny, A., & Fermanian, J. D. (2019). On kernel-based estimation of conditional Kendall’s tau: finite-distance bounds and asymptotic behavior. Dependence Modeling, 7(1), 292-321. doi:10.1515/demo-2019-0016

See Also

CKT.estimate for other estimators of conditional Kendall's tau. CKTmatrix.kernel for a generalization of this function when the conditioned vector is of dimension d instead of dimension 2 here.

See CKT.hCV.l1out for manual selection of the bandwidth h by leave-one-out or K-folds cross-validation.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 800
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

newZ = seq(2,10,by = 0.1)
estimatedCKT_kernel <- CKT.kernel(
   X1 = X1, X2 = X2, Z = Z,
   newZ = newZ, h = 0.1, kernel.name = "Epa")$estimatedCKT

# Comparison between true Kendall's tau (in black)
# and estimated Kendall's tau (in red)
trueConditionalTau = -0.9 + 1.8 * pnorm(newZ, mean = 5, sd = 2)
plot(newZ, trueConditionalTau , col = "black",
     type = "l", ylim = c(-1, 1))
lines(newZ, estimatedCKT_kernel, col = "red")

---

Prediction of conditional Kendall's tau using nearest neighbors

Description

Let $X_1$ and $X_2$ be two random variables. The goal of this function is to estimate the conditional Kendall's tau (a dependence measure) between $X_1$ and $X_2$ given $Z=z$ for a conditioning variable $Z$. Conditional Kendall's tau between $X_1$ and $X_2$ given $Z=z$ is defined as:

$P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) > 0 | Z_1 = Z_2 = z)$

$- P( (X_{1,1} - X_{2,1})(X_{1,2} - X_{2,2}) < 0 | Z_1 = Z_2 = z),$

where $(X_{1,1}, X_{1,2}, Z_1)$ and $(X_{2,1}, X_{2,2}, Z_2)$ are two independent and identically distributed copies of $(X_1, X_2, Z)$. In other words, conditional Kendall's tau is the difference between the probabilities of observing concordant and discordant pairs from the conditional law of

$(X_1, X_2) | Z=z.$

This function estimates conditional Kendall's tau using a nearest neighbors. This is possible by the relationship between estimation of conditional Kendall's tau and classification problems (see Derumigny and Fermanian (2019)): estimation of conditional Kendall's tau is equivalent to the prediction of concordance in the space of pairs of observations.

Usage

CKT.predict.kNN(
  datasetPairs,
  designMatrix = datasetPairs[, 2:(ncol(datasetPairs) - 3), drop = FALSE],
  newZ,
  number_nn,
  weightsVariables = 1,
  normLp = 2,
  constantA = 1,
  partition = NULL,
  verbose = 1,
  lengthVerbose = 100,
  methodSort = "partial.sort"
)

Arguments

datasetPairs	the matrix of pairs and corresponding values of the kernel as provided by datasetPairs.
designMatrix	the matrix of predictors. They must have the same number of variables as newZ and the same number of observations as inputMatrix, i.e. there should be one "multivariate observation" of the predictor for each pair.
newZ	the matrix of predictors for which we want to estimate the conditional Kendall's taus at these values.
number_nn	vector of numbers of nearest neighbors to use. If several number of neighbors are given (local) aggregation is performed using Lepski's method on the subset determined by the partition.
weightsVariables	optional argument to give different weights $w_j$ to each variable.
normLp	the p in the weighted p-norm $|| x ||_p = \sum_j w_j * x_j^p$ used to determine the distance in the computation of the nearest neighbors.
constantA	a tuning parameter that controls the adaptation. The higher, the smoother it is; while the smaller, the least smooth it is.
partition	used only if length(number_nn) > 1. It is the number of subsets to consider for the local choice of the number of nearest neighbors ; or a vector giving the id of each observations among the subsets. If NULL, only one set is used.
verbose	if TRUE, this print information each lengthVerbose iterations
lengthVerbose	number of iterations at each time for which progress is printed.
methodSort	is the sorting method used to find the nearest neighbors. Possible choices are ecdf (uses the ecdf to order the points to find the neighbors) and partial.sort uses a partial sorting algorithm. This parameter should not matter except for the computation time.

Value

a list with two components

• estimatedCKT the estimated conditional Kendall's tau, a vector of the same size as the number of rows in newZ;

• vect_k_chosen the locally selected number of nearest neighbors, a vector of the same size as the number of rows in newZ.

References

Derumigny, A., & Fermanian, J. D. (2019). A classification point-of-view about conditional Kendall’s tau. Computational Statistics & Data Analysis, 135, 70-94. (Algorithm 5) doi:10.1016/j.csda.2019.01.013

See Also

See also other estimators of conditional Kendall's tau: CKT.fit.tree, CKT.fit.randomForest, CKT.fit.nNets, CKT.fit.randomForest, CKT.fit.GLM, CKT.kernel, CKT.kendallReg.fit, and the more general wrapper CKT.estimate.

Examples

# We simulate from a conditional copula
set.seed(1)
N = 800
Z = rnorm(n = N, mean = 5, sd = 2)
conditionalTau = -0.9 + 1.8 * pnorm(Z, mean = 5, sd = 2)
simCopula = VineCopula::BiCopSim(N=N , family = 1,
    par = VineCopula::BiCopTau2Par(1 , conditionalTau ))
X1 = qnorm(simCopula[,1])
X2 = qnorm(simCopula[,2])

newZ = seq(2,10,by = 0.1)
datasetP = datasetPairs(X1 = X1, X2 = X2, Z = Z, h = 0.07, cut = 0.9)
estimatedCKT_knn <- CKT.predict.kNN(
  datasetPairs = datasetP,
  newZ = matrix(newZ,ncol = 1),
  number_nn = c(50,80, 100, 120,200),
  partition = 8)

# Comparison between true Kendall's tau (in black)
# and estimated Kendall's tau (in red)
trueConditionalTau = -0.9 + 1.8 * pnorm(newZ, mean = 5, sd = 2)
plot(newZ, trueConditionalTau , col="black",
   type = "l", ylim = c(-1, 1))
lines(newZ, estimatedCKT_knn$estimatedCKT, col = "red")

---