Crossvalidation

To evaluate the skill of the model a crossvalidated hindcast is produced by applying the leave one year out methodology. In this way, the data corresponding to each year is omitted from the \(Y\) and \(Z\) matrices and MCA is applied with the remaining years. In each iteration, \(\Psi\) is calculated, as well as the predicted \(\hat{Z}\) for the omitted year, using the value of the predictor \(Y\) for that particular year. With the crossvalidated hindcast, the skill is evaluated by calculating the anomaly correlation coefficient (ACC) between \(Z\) and the crossvalidated \(\hat{Z}\). For each MCA iteration, the scf, \(U_s\) and \(V_s\), \(S_{UZ}\) and \(S_{UY}\) are stored in order to test the stability of the modes and its sensitivity for each particular year.

Crossvalidation is performed with class spy4cast.spy4cast.Crossvalidation

To perform Crossvalidation on predictor, \(Y\) and a predictand \(Z\) with Spy4Cast you need to load the datasets, slice them and preprocess them first:

from spy4cast import Dataset, Region, Month, spy4cast

y = Dataset("predictor.nc", folder="datasets").open().slice(
    Region(lat0=-20, latf=20, lon0=-10, lonf=40,
           month0=Month.JAN, monthf=Month.MAR, year0=1964, yearf=1994))
z = Dataset("predictand.nc", folder="datasets").open().slice(
    Region(lat0=-10, latf=-40, lon0=-180, lonf=-100,
           month0=Month.FEB, monthf=Month.APR, year0=1964, yearf=1994))
yp = spy4cast.Preprocess(y)
zp = spy4cast.Preprocess(z)
cross = spy4cast.Crossvalidation(yp, zp, nm=3, alpha=0.1)
cross.plot(show_plot=True, halt_program=True)

The following figure is an example of the figures that can be created with Crossvalidation. It is done with the code from the manual Spy4Cast Manual of example EquatorialAtlantic_Impact_Nino.ipynb