[CaCL] Impossible?

[Marten van Schijndel]

Ok. Finally following up on our TFCE discussion:
I'm not sure the MaxT approach is impossibly conservative, but it is
quite conservative. Nathan was correct, however, in his statement that
the distribution converges to a fixed point since you draw from the Max
distribution every time you resample, so it won't Zeno to
near-Bonferroni levels of pain in the limit. My mistaken statement was
based on a quick scan of background work and a broken inference engine.
(The first half-paper of) the Kutas paper mentions that increasing the
number of iterations makes the test more conservative. Given that we're
talking about bootstrapping, and given that such a statement is
trivially true for a small number of permutations, I inferred that their
simulations must show that it's true for large numbers of
permutations... but looking over them, they don't. That said, too large
a number of permutations will dramatically increase runtime without any
benefit once the null distribution's stable. TFCE is not a speedy
algorithm due to the summed sliding threshold scoring algorithm which
must be done for each new permutation over the entire p-value space.

The best practices TFCE paper I mentioned is Pernet et al. (2015).
Attached for those who are interested in such things. And who wouldn't
be after our rousing discussion today?

Regarding sadness:
The problem is really that I'll need to rerun the tests, which makes me
sad because I'm lazy. Also, I'm looking at areas that are fairly close
together but that don't share a cluster, so partitioning the two into
separate tests is hairy and possibly overconstraining. I'm pretty sure
that if you apply TFCE to an area that's too constrained you'll still be
left with sadness. If Cz and all its neighbors are significant due to
our manipulation (which we know from pilot data), and (in order to
maximize the interpretability of our results) we restrict the domain of
current testing to Cz and its neighbors during the time we suspect the
effect to be and run TFCE, our permuted sample space will look very
similar to our observed samples, so TFCE will be too conservative.

Marty
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pernet_etal-2015-tfce_methods_review.pdf
Type: application/pdf
Size: 4330889 bytes
Desc: not available
URL: <http://lists.osu.edu/pipermail/cacl/attachments/20150911/bdec5042/attachment-0001.pdf>