[CaCL] Impossible?

[Marten van Schijndel]

Nope. The permutation's across a different dimension. Nathan, you're
right. I should be happy. I'm on a roll today.

Marty

On 09/11/2015 09:46 PM, Marten van Schijndel wrote:
> Ok. Finally following up on our TFCE discussion:
> I'm not sure the MaxT approach is impossibly conservative, but it is
> quite conservative. Nathan was correct, however, in his statement that
> the distribution converges to a fixed point since you draw from the Max
> distribution every time you resample, so it won't Zeno to
> near-Bonferroni levels of pain in the limit. My mistaken statement was
> based on a quick scan of background work and a broken inference engine.
> (The first half-paper of) the Kutas paper mentions that increasing the
> number of iterations makes the test more conservative. Given that we're
> talking about bootstrapping, and given that such a statement is
> trivially true for a small number of permutations, I inferred that their
> simulations must show that it's true for large numbers of
> permutations... but looking over them, they don't. That said, too large
> a number of permutations will dramatically increase runtime without any
> benefit once the null distribution's stable. TFCE is not a speedy
> algorithm due to the summed sliding threshold scoring algorithm which
> must be done for each new permutation over the entire p-value space.
> 
> The best practices TFCE paper I mentioned is Pernet et al. (2015).
> Attached for those who are interested in such things. And who wouldn't
> be after our rousing discussion today?
> 
> Regarding sadness:
> The problem is really that I'll need to rerun the tests, which makes me
> sad because I'm lazy. Also, I'm looking at areas that are fairly close
> together but that don't share a cluster, so partitioning the two into
> separate tests is hairy and possibly overconstraining. I'm pretty sure
> that if you apply TFCE to an area that's too constrained you'll still be
> left with sadness. If Cz and all its neighbors are significant due to
> our manipulation (which we know from pilot data), and (in order to
> maximize the interpretability of our results) we restrict the domain of
> current testing to Cz and its neighbors during the time we suspect the
> effect to be and run TFCE, our permuted sample space will look very
> similar to our observed samples, so TFCE will be too conservative.
> 
> Marty
>