John Mount has an interesting post looking at replicating linear models without training data:
Let’s work an example in R. Suppose we are working with a linear regression model and from our donor system we have extracted the following representation of the model as “intercept” and “betas”.
intercept <- 3 betas <- c(weight = 2, height = 4)Our goal is to build a linear regression model that has the above coefficients. The way we are going to do this is by building our own synthetic data set such that the regression fit through this data set yields these coefficients.
It’s fairly straightforward to do this for linear models; as things get more complicated, however, the difficulty level spikes.