Generates reference distribution

Generates the reference distribution of norms for a stimulus set.

Usage

generateReferenceDistribution2IFC(
  rdata,
  iter = 10000,
  ncores = default_ncores(),
  response_seed = NULL,
  save_rdata = TRUE,
  baseimage = NULL,
  reference_stimuli = NULL,
  reference_method = c("gram", "images"),
  mask = NA
)

Arguments

rdata: Path to the .Rdata file written when the stimuli were generated. It holds the contrast parameters of every stimulus.
iter: Number of simulated classification images, each built from random responses; the distribution holds one norm per image.
ncores: Number of CPU cores used to render the saved noise with reference_method = "images" (default: detectCores() - 1; 2 under R CMD check, per CRAN policy). "gram" does not use it. Each core runs a worker holding its own copy of the noise basis and a render's working memory: 0.8 to 1 GB per worker at 512 pixels in https://github.com/rdotsch/rcicr/blob/main/analyses/worker-memory.md. No more workers start than there are trials.
response_seed: Optional seed for the simulated random responses. The default, NULL, continues from the state the stimulus generator left behind, as described under Reproducibility; it needs the stimulus seed saved in the file. A number gives an independent draw of the null from the same stimuli.
save_rdata: Boolean: write the reference distribution into the rdata file (default TRUE). With FALSE, later calls to computeInfoVal2IFC keep using what the file already holds. Set it to FALSE whenever you set response_seed, so a one-off null does not become the file's permanent reference. The exception is a file without a stimulus seed: there the seeded reference is the one to keep, as the file's reproducible reference.
baseimage: Base-image label, the same key passed to generateCI(). Required when the base images have different noise parameters. With a single base image, or base images sharing one parameter set, leave it at NULL.
reference_stimuli: Optional stimulus numbers, each once, to build the reference over instead of every saved stimulus: the stimuli a classification image was built from, when that is not all of them. See "Matching the reference to the classification image" in computeInfoVal2IFC. The simulated responses continue the same stream as the default, so the reference is reproducible from the file. Passing every saved stimulus is the same as the default, NULL.
reference_method: "gram" (the default) or "images": how the reference norms are computed. "images" reproduces rcicr 1.5.0 and earlier bit for bit. See "Reference method" below.
mask: Optional mask, in any form generateCI accepts: a 0/1 matrix or the path to a PNG, black (0) where masked. The reference is then built over the unmasked pixels only, as computeInfoVal2IFC needs for a classification image computed with that mask, and stored apart from the default one. The default, NA, uses every pixel. See "Masked classification images" in computeInfoVal2IFC.

Value

The reference distribution, invisibly, as a numeric vector of iter norms. Unless save_rdata = FALSE, it is also added to the rdata file as reference_norms, with reference_norms_seed recording the response_seed it was drawn with. A later computeInfoVal2IFC call on the same file then reuses it instead of simulating again. For independent base images it goes in reference_norms_by_base instead, as described above. A reference over reference_stimuli goes in reference_norms_by_stimuli, a list with one entry per stimulus set (and base image, where the bases have different noise), leaving the default reference untouched.

Details

computeInfoVal2IFC scores a classification image against this distribution. By default the result is saved in the rdata file for later reuse.

Reproducibility

With the default response_seed = NULL, the reference distribution depends only on the stimulus .Rdata file: not on the current random state, not on ncores, and, for files that record the RNG kind, not on the session's RNGkind. Two researchers computing InfoVal from the same stimulus file get the same reference distribution, and the same number, on any machine and in any session.

This needs the stimulus seed saved in the file. A file without one (made with generateStimuli2IFC(seed = NULL), or with the field removed) has no stream to continue, so the default stops with an error; pass a response_seed instead.

Stimulus files record the RNGkind the stimuli were drawn under, as rng_kind, and the simulated responses are drawn under it, with or without a response_seed. When it differs from the session's kind, the caller's random stream is restored afterwards, kind and position, so the session is not left on the file's generator. In the session's own kind the stream is left where the draws end, as it always was. Files written before the kind was recorded draw under the session's kind: a session with a different RNGkind() draws different responses, and so a different null. To reproduce an earlier reference from such a file, use the RNG kind that built it; see https://github.com/rdotsch/rcicr/issues/315.

The noise is rebuilt from the basis and parameters saved in the stimulus file, so the base images are never reopened and a moved or archived experiment can still be scored. The simulated responses continue the random stream from the saved stimulus seed, after the draws for one parameter matrix, which reproduces the historical default reference.

Pass a response_seed to draw a different null from the same stimuli, for instance to check how much Monte Carlo error a given iter leaves in your InfoVal. This changes only the simulated responses, not the stimuli or the noise basis the null is built on.

Reference method

reference_method = "gram", the default, computes the norms from the stimulus Gram matrix, without calling generateNoiseImage() or multiplying the noise for every draw. For a small stimulus set the Gram matrix is built from the noise rendered through the sparse basis; otherwise from the basis's cross-product, kept sparse. Either way the basis is built a block of image columns at a time, so no full-size copy of it, or of the noise, is held. "images" is the calculation of rcicr 1.5.0 and earlier: every noise image rendered, in parallel over ncores, and multiplied for every draw. It reproduces references from those versions bit for bit.

The two agree to rounding. No configuration measured was bit-identical, and none differed by more than a relative 5e-14 in a single norm (reference BLAS; see https://github.com/rdotsch/rcicr/blob/main/analyses/gram-reference-accuracy.md).

Computing a reference with "gram" prints a message saying so, and naming "images" as the way to reproduce earlier references. Silence it with suppressMessages(). The method is never switched for you.

A reference already stored in the file is reused as stored, whichever method is asked for. To rebuild one with a particular method, add force_gen_ref_dist = TRUE in computeInfoVal2IFC, or call this function with save_rdata = TRUE. The method a reference was computed with is stored beside it, as reference_norms_method or as method in each reference_norms_by_base and reference_norms_by_stimuli entry.

Independent base images

When the base images have different parameter matrices, baseimage says whose noise to use. The distributions are then stored in reference_norms_by_base, one entry per base label, each holding norms and response_seed. A shared reference_norms in such a file is neither used nor overwritten. Files whose base images share one parameter matrix keep using reference_norms.

Examples

# a synthetic square grayscale image stands in for a real base face photo
base_face <- tempfile(fileext = ".png")
png::writePNG(matrix(runif(32 * 32), 32, 32), base_face)

stimulus_path <- tempfile("stimuli")
generateStimuli2IFC(
  base_face_files = list(face = base_face),
  n_trials = 6,
  img_size = 32,
  stimulus_path = stimulus_path,
  seed = 1,
  ncores = 1,
  nscales = 1,
  save_as_png = FALSE
)
#> 
  |                                                                            
  |                                                                      |   0%
  |                                                                            
  |============                                                          |  17%
  |                                                                            
  |=======================                                               |  33%
  |                                                                            
  |===================================                                   |  50%
  |                                                                            
  |===============================================                       |  67%
  |                                                                            
  |==========================================================            |  83%
  |                                                                            
  |======================================================================| 100%
rdata_file <- list.files(stimulus_path, pattern = "\\.Rdata$", full.names = TRUE)[1]

# iter is kept tiny here for a fast example; in practice use iter >= 10000.
suppressWarnings(generateReferenceDistribution2IFC(rdata_file, iter = 3, ncores = 1))
#> Building the reference from the saved noise, please wait...
#> Computing reference distribution, please wait...
#> InfoVal reference computed with reference_method = "gram". References from rcicr 1.5.0 and earlier used "images"; pass reference_method = "images" to reproduce them bit for bit.
#> 
  |                                                                            
  |                                                                      |   0%
  |                                                                            
  |======================================================================| 100%
#> 
#> Saving simulated reference distribution to rdata file...