my searchlight, and some (final?) thoughts

Since I've been thinking about searchlight shapes, here's a diagram of the two-voxel radius searchlight I'm currently using in a few projects. As in the previous post, the center voxel is black, the one-voxel radius surrounds red, and the two-voxel radius surrounds purple.

This two-voxel radius searchlight has 13 + 21 + 24 + 21 + 13 = 92 voxels in the surround, which makes it larger than the three-voxel radius faces-must-touch searchlight (at 86 surrounding voxels).

So how should we grow searchlights, edge-face-corner, edge-face, faces-only? Beats me. I can imaging comparing the results of a few different shapes in a particular dataset, but I doubt there's a single shape that is always best. But the searchlight shape should be added to our list of how to describe a searchlight analysis.

PS - I use R code to generate lists of adjacent voxels for running searchlight analyses; contact me if you'd like a copy/details.

pyMVPA searchlight shapes

Michael Hanke listed the number of voxels for different searchlight radii in pyMVPA. He also wrote that pyMVPA considers neighboring voxels to be those that share two edges (i.e., a face).

Here's a diagram of following the faces-touching rule. The center voxel is black, the one-voxel radius surrounds red, the two-voxel radius surrounds purple, and the three-voxel radius surrounds blue.

This gives 1 + 9 + 21 + 24 + 21 + 9 + 1 = 86 surrounding voxels for the three-voxel radius.

But Michael wrote that a three-voxel radius searchlight in pyMVPA has 123 voxels (counting the center). If so, at left is how I could come by that count: 13 + 28 + 40 + 28 + 13 = 122.

This shape is not quite symmetrical: there are four voxels to the left, right, front, and back of the center voxel but only two above and below.

Anyone know if this is what pyMVPA does?

UPDATE: Neither of these sketches is correct, see pyMVPA searchlight: solved.

searchlight shapes

How exactly the searchlight was shaped is one of those details that is usually not mentioned in searchlight analysis papers but is not obvious, mostly since voxels are cubes.

Nikolaus Kriegeskorte (the father of searchlight analysis) uses the illustration on the left.

Expanded, I think this would look like the picture at left. There are 32 voxels surrounding the center; the center voxel in black and surrounding in blue.

But there are other ways of thinking of a searchlight. Here are three ways of defining a one-voxel radius searchlight.

Personally, I implemented the middle version (edges and faces, not corners, 18 voxels) as a one-voxel radius searchlight in my code. Larger radii are calculated iteratively (so for a two-voxel radius, combine the one-voxel radius searchlights defined by treating all 18 surrounding voxels present in the one-voxel radius searchlight as centers). I don't think one version of defining surrounds is necessarily better than others, but we do need to specify the voxels we are including.


update 11 September: I changed the caption of the searchlight shape images for clarity.

what MVPA detects

MVPA is not a universal solution for fMRI; it's always necessary to carefully think about what types of activity changes you want to detect.

For example, suppose these are timecourses for two people completing a task with two types of trials. It's clear that in both people there is a very strong effect of the trials: for person #1 the BOLD goes up during trial type A and down during trial type B; the reverse is true for person #2.

"Standard" MVPA (e.g. linear svm) will detect both of these patterns equally well: there is a consistent difference between trial types A and B in both people. In addition, the difference in direction is usually not reflected in the analysis: often only each subject's accuracy is taken to the second level.

This can be a feature, or a bug, depending on the hypotheses: If you want to identify regions with consistent differences in activation in each person, regardless of what those differences are, it's a feature. If you want to identify regions with a particular sort of difference in activation, it can be a bug.

My suggestion is usually that if what you want to detect is a difference in the amount of BOLD (e.g. "there will be more BOLD in this brain region during this condition compared to that condition") then it's probably best to look at some sort of mass-univariate/GLM/SPM analysis. But if you want to detect consistent differences regardless of the amount or direction of BOLD change (e.g. "the BOLD in this brain region will be different in my conditions"), then MVPA is more suitable.

Note also that a linear svm is perfectly happy to detect areas in which adjacent voxels have opposite changes in BOLD - the two timecourses above can be within the same ROI yet be detected quite well as an informative area. As before, this can be a feature or a bug. So, again, if you want to detect consistent regional differences in the overall amount of BOLD, you probably don't want to use "standard" MVPA.

Nipype: comments from Yaroslav

Yaroslav Halchenko was kind enough to provide some detailed feedback on my previous post, which he allowed me to excerpt here.

"a decent understanding of both nipype and the programs it's calling (from fsl or whatever)."

Yaroslav: yes and no. As with any software -- decent understanding is desired but not strictly necessary. nipype comes with "pre-cooked" workflows for some standard analyses so you would just need to tune the script/parameters up to input your data/design.

"as most useful to a skilled programmer wanting to run several versions of an analysis" 

Yaroslav: That is one of the cases. I could argue that the most useful is an efficient utilization of computing resources. i.e running many computations in parallel (e.g. per each functional run/ subject etc). 

also -- noone runs their analysis ones with all parameters decided a priory and not 'tuning them up' -- that is also where nipype comes useful since it would be smart to recompute only what needs to be recomputed. Thus possibly helping to avoid human errors.

 as for a "skilled programmer" -- well... somewhat -- scripting nipype may be could be easier but altogether it is not that bad actually. Just as with any scripting/hacking with practice comes (greater) efficiency. 

Jo: I saw mentions of using it in parallel. Our computer support people had a devil of a time (and basically failed) with python on our shared systems in the past. But we're also having problems with matlab/spm scripts tying up way too much of the systems and not getting properly managed. So that might be worth a closer look for us.

"It's often necessary to check initial output before moving to the next step in the pipeline; these errors would need to be caught somehow (go back and check intermediate files?)" 

Yaroslav: you could code your "pipeline" incrementally indeed and check all the intermediate results... or just run it once and then check them and rerun pipeline if you would need to change any of the parameters. I do not see it being much different from what you would get with other software -- once again boiling down to how particular/anal you want to be about checking your data/results. Also you could code/assemble pieces which would provide you with reports of the specific processing steps so you could later on glance over them rapidly.

"Nipype will need to be updated any time one of the underlying programs is updated." 

Yaroslav: again -- yes and no -- nipype (interfaces) would need to be updated only if corresponding programs' command line interface gets changed in some non-backward compatible fashion. Most of the software suites avoid such evil actions; but yes -- nipype would need to be (manually) updated to accommodate new cmdline options, and at times remove some if they were removed from the original software.

Jo: And I would think the defaults. It'd be troublesome if, say, a default Nipype motion-correction routine used a different motion-correction algorithm specification than the GUI version.

Yaroslav: well -- they are trying to match the defaults and as long as it doesn't change and you do not mix GUI/non-gui -- how do you know that GUI's is different/better.

Jo: I don't know that the GUI options are better. But practically speaking, many people (at least with spm) use the default options for most choices, so I was thinking we'd need to confirm that the options are the same running it through the GUI or Nipype, just for consistency.

"I've had terrible troubles before with left/right image flipping and shifting centerpoints when changing from one software package to another (and that's not even thinking about changing from one atlas to another). Trying to make packages that have different standard spaces play nicely together strikes me as very non-trivial..."

Yaroslav:  RIGHT ON -- that is indeed one of the problems I have heard about (mixing AFNI and FSL IIRC) and to say the truth I am not sure how far nipype could help assuring correct orientation... if that is possible at all -- it would be a good question to nipype folks I guess. BUT many tools do play nicely together without doing all kinds of crazy flips, so, although it is a valid concern, it is manageable (run pipeline and verify correct orientation by inspecting results through the steps) and might be limited to only a limited set of abusers.

"... Second, would there be a benefit to using Nipype if the pipeline only includes one program (e.g. why Nipype instead of spm8 batching)?"

Yaroslav:   well -- not everyone is using spm8 and those tools might lack back processing/dependencies tracking etc... and even with spm8 -- you might like eventually to add few additional steps provided by other toolkits and would be ready to do so with ease (given you also check for possible flip).

Nipype impressions

I was asked to take a look at Nipype (great logo, Arno!). Here are some of my impressions; I haven't tried coding or installing it. If you have, I'd be curious to hear how it's gone and it you think any of this is off the mark.

Nipype aims to connect different fMRI software packages (spm, fsl, etc) so that you can program a reproducible set of steps. Your preprocessing could mix programs (e.g. smooth in one program, realign in another), and it should be easier to switch a step from one program to another (e.g. smooth in fsl instead of spm) with relatively few changes to the program.

I like that Nipype makes it possible to reproduce and share analysis steps; the current procedures are far too messy and error-prone. Nipype looks to require a fair bit of programming skill (python), however, as you're coding at a step above the underlying routines/programs: to do it properly you'd need a decent understanding of both nipype and the programs it's calling (from fsl or whatever).

Nipype strikes me as most useful to a skilled programmer wanting to run several versions of an analysis (e.g. with different motion-correction algorithms) or the same analysis many, many times (lots of subjects). It makes it more straightforward to make an established set of analysis steps (such as preprocessing) available to other people. It's hard to tell without trying it out, but it looks like a nipype program could be set up to be run by someone that doesn't know the underlying routines (e.g. an RA running the same preprocessing steps on a new group of subjects).

I have concerns about missing errors, specifying options, and stability. It's often necessary to check initial output before moving to the next step in the pipeline; these errors would need to be caught somehow (go back and check intermediate files?). Most routines have many options (e.g. FWHM, interpolation algorithm). Does Nipype set the default values? Making every single option visible through Nipype seems daunting, which is one of my concerns about stability: Nipype will need to be updated any time one of the underlying programs is updated. And installation would likely be non-trivial (all the necessary dependencies and paths), unless it's run in NeuroDebian.

A few final thoughts: First, I've had terrible troubles before with left/right image flipping and shifting centerpoints when changing from one software package to another (and that's not even thinking about changing from one atlas to another). Trying to make packages that have different standard spaces play nicely together strikes me as very non-trivial (I don't mean code-wise, I mean keeping track of alignment). Second, would there be a benefit to using Nipype if the pipeline only includes one program (e.g. why Nipype instead of spm8 batching)? It seems in this case it might just be adding extra complication: the need to learn the program options, plus the Nipype code.

So will I recommend we try Nipype right now? No. But I'll keep an eye on it and am certainly willing to revise my opinion.


Gorgolewski K, Burns CD, Madison C, Clark D, Halchenko YO, Waskom ML, & Ghosh SS (2011). Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in python. Frontiers in neuroinformatics, 5 PMID: 21897815

but replications are possible

Considering the posts on all the reasons that fMRI findings can be dodgy and replications unlikely, I'll share some recent experiences I've had in which proper replications succeeded.

I can't go into any detail (yet), but this image is a few slices showing the results with the first dataset in red and the second in blue. We definitely tweaked the analysis of the first dataset a bit; I did as much of the tweaking as possible with a control classification rather than the real one, but I can't claim that all the choices were made on a priori principle. But I would claim that the choices were made without any data peeking for the second dataset: the blue blobs are the very first set of across-subjects results; just the analysis we used for the first dataset. And I'm extra-pleased by these results because they are properly independent: different scanner, different people, even different stimuli (but the same task and experimental questions, of course).

Within the last week I've actually had another case of replicating results in properly independent datasets; a completely different set of analyses than the blobs above. It's an anatomical ROI-based analysis, so I don't have pretty pictures to show, but the tables are very lovely.

So do not give up hope: replications are possible. But we must be very vigilant in our methods: we are all easy to fool.