What happened in this fMRI run? ... happened again.

Back in July I posted about a strangely-failed fMRI run, and yesterday I discovered that we had another case not quite two weeks ago. This is the same study, scanner (3T Siemens Prisma), headcoil (32 channel), task, and acquisition protocol (CMRR MB4) as the July case, but a different participant. I've contacted our physicists, but we probably can't investigate properly until after the holidays, and are hampered by no longer having access to some of the intermediate files (evidently some of the more raw k-space/etc. files are overwritten every few days). 

I've asked our experimenters to be on the lookout, and while hopefully it won't happen again, if it does, I hope they can catch it during the session so all the files can be saved. If anyone has ideas for spotting this in real time, please let me know.

A possibly-relevant data point: the participant asked to have the earbuds adjusted after the first task run. The technician pulled the person out of the bore to fix the earbuds, but did not change the head position, and did not do a new set of localizers and spin echo fieldmaps before starting the second task run (the one with the problem). I've recommended that the localizers and spin echo fieldmaps be repeated whenever the person is moved out of the bore, whether they get up from the table or not, but the technician for this scan did not think it necessary. What are your protocols? Do you suggest repeating localizers? No one entered the scanning room before the problematic July run, so this (pulling the person out) might be a coincidence.

Here's what the this most recent case looks like. First, the three functional runs' DICOMs (frame 250 of 562) open in mango, first with scaling allowed to vary with run:

Then with scaling of 0 to 10000 in all three runs, showing how much darker run 2 is:

And finally the SBRef from run 2:

In July the thinking was that this is an RF frequency issue, possibly due to the FatSat RF getting set improperly, so that both fat and water were excited. But this seems hard to confirm from the DICOM header; this time, the Imaging Frequency DICOM field (0018,0084) is nearly identical in all three runs: 123.258928, 123.258924, 123.258924 (runs 1, 2, and 3 respectively), which is very similar to what it was in July (123.258803).

mastodon

Like many, I've started a mastodon account: @JosetAEtzel@FediScience.org. My twitter account (also @JosetAEtzel) is still active, but I plan to be more on mastodon than twitter going forward. This blog is staying around, too ... who knows, maybe I'll get some of my backlog of posts written! 😅

Update: US researchers CAN guarantee privacy post-Dobbs

The two previous posts described my concerns about the NIH Certificate of Confidentiality exceptions post-Dobbs; the vagueness of the "federal, state, or local laws" "limited circumstances" formulation is troubling, since it seems that it could apply to something like a state-level prosecution for pregnancy termination.

I am happy to relay that the "federal, state, or local laws" exemption is clarified in the "When can Information or Biospecimens Protected by a Certificate of Confidentiality be Disclosed?" section of the What is a Certificate of Confidentiality? | grants.nih.gov site: 

[update 7 June 2023: the NIH website has changed a bit, but the key text below is still present, now under section  4.1.4 Confidentiality > 4.1.4.1 Certificates of Confidentiality]

"Disclosure is permitted only when: 

• Required by Federal, State, or local laws (e.g., as required by the Federal Food, Drug, and Cosmetic Act, or state laws requiring the reporting of communicable diseases to State and local health departments), excluding instances of disclosure in any Federal, State, or local civil, criminal, administrative, legislative, or other proceeding;  [emphasis mine]
• Necessary for the medical treatment of the individual to whom the information, document, or biospecimen pertains and made with the consent of such individual; 
• Made with the consent of the individual to whom the information, document, or biospecimen pertains; or 
• Made for the purposes of other scientific research that is in compliance with applicable Federal regulations governing the protection of human subjects in research."

The highlighted clause is the key clarification: the "federal, state, or local laws" exemption would not apply to something like a state-level prosecution for pregnancy termination, because that would be a criminal proceeding. And our data isn't only protected from criminal proceedings, but from civil, administrative, legislative, and others as well.

I am relieved by this exclusion, and encourage all universities and groups covered by the Certificate to include it, not only the "*Disclosure of identifiable, sensitive information (i.e., information, physical documents, or biospecimens) protected by a Certificate of Confidentiality must be done when such disclosure is required by other applicable Federal, State, or local laws." formulation. 

While I am relieved by this exclusion and find it sufficient guarantee that our participants' data is protected from disclosure, we will continue to minimize the amount of pregnancy-related information we collect, and use indirect phrasing in our screening questions whenever possible. Privacy and sensitivity are always important, but are especially critical now in the United States and when reproduction is involved.

UPDATE 16 September 2022: Many universities already use the longer (with the exclusion) explanation on their HRPO websites when describing the Certificate of Confidentiality protections. A google search for "excluding instances of disclosure in any Federal, State" found many, including Michigan State University, the University of Pittsburgh, the University of Washington, Virginia Commonwealth University, and North Dakota State University. Hopefully these examples can serve as templates for other institutions.

The NIH's example consent language also includes it: "This research is covered by a Certificate of Confidentiality from the National Institutes of Health. This means that the researchers cannot release or use information, documents, or samples that may identify you in any action or suit unless you say it is okay. They also cannot provide them as evidence unless you have agreed. This protection includes federal, state, or local civil, criminal, administrative, legislative, or other proceedings. An example would be a court subpoena."

UPDATE 28 September 2022: Washington University in St. Louis changed the Certificate of Confidentiality description to include the exclusion.

Tracking US universities' post-Dobbs research privacy guarantees

UPDATE 9 September 2022: Good news! The "federal, state, or local laws" exemption is clarified in the "When can Information or Biospecimens Protected by a Certificate of Confidentiality be Disclosed?" section of the What is a Certificate of Confidentiality? | grants.nih.gov site. 

This post is now less relevant, so I put it below the "jump".

research in the United States after the fall of Roe v. Wade

UPDATE 9 September 2022: Good news! The "federal, state, or local laws" exemption is clarified in the "When can Information or Biospecimens Protected by a Certificate of Confidentiality be Disclosed?" section of the What is a Certificate of Confidentiality? | grants.nih.gov site. 

The NIH Certificate of Confidentiality is sufficient to protect researchers from being forced to release data if one of our participants is charged with abortion, which is great news. However, there are still many ethical concerns about collecting sensitive data unnecessarily, and I believe it is prudent to be extra aware of how pregnancy-related questions are being asked (e.g., in a phone screen), and minimize direct questions whenever possible.

Previous post:

This post is an essay-style, expanded version of messages I’ve posted on twitter (@JosetAEtzel) the last few weeks, responding to the Dobbs v. Jackson Women's Health Organization decision overturning Roe v. Wade in the United States, and Missouri’s subsequent trigger law outlawing abortion except in dire emergency. I hoped these issues would rapidly become outdated, but unfortunately that is not the case; if anything they are compounding, and I very much fear no end is in sight. I am not willing to be silent on the topic of protecting participants, or university ethics more generally.

I am a staff scientist at Washington University in St. Louis, Missouri, USA, and have been here twelve years now. It’s been a good place to do research, and I have great colleagues. I work with data collected on humans, mostly task fMRI. I generally spend my time at work on analysis and hunting for missing or weird images in our datasets, but the last few weeks I’ve spent substantial amount of time hunting for pregnancy-related information in our procedures and datasets, and seeking answers to how the legal changes affect us and our participants.

Our fMRI consenting protocols require the use of screening forms that ask if currently pregnant; high-risk studies (PET-MR) require a pregnancy test be performed immediately before the scan. These signed and dated screening forms are retained indefinitely by the imaging center at the hospital and/or our lab. Imaging studies routinely include pregnancy questions in the phone screening to determine eligibility.

An additional source of pregnancy information in our datasets is via studies using passive sensing data collection (e.g., via an app installed on participants’ phones). These can include GPS and other forms of tracking, which could e.g., show whether the participant spent time at a place where abortions are possible or searched for abortion information. Previous data breaches have happened with this type of research software, and the collection of any GPS or other tracking information raises serious privacy concerns, but my focus here is the security of this data after it is in the researchers’ hands.

We need guarantees that we will never be asked to release this data, even in the (appalling but not totally unprecedented) case that someone is charged with abortion and we are asked by a court to disclose whether the participant said that yes, they were pregnant on a particular date.

NIH Certificates of Confidentiality protect participants’ information from disclosure, but have exceptions in “limited circumstances”. “*Disclosure of identifiable, sensitive information (i.e., information, physical documents, or biospecimens) protected by a Certificate of Confidentiality must be done when such disclosure is required by other applicable Federal, State, or local laws.” At Washington University in St. Louis (as of 11 July 2022) we are being told it might not be sufficient to rely upon the Certificate of Confidentiality; that it is not "bulletproof" for state-level abortion-related lawsuits. University counsel here is still investigating, as I assume are those elsewhere.

I have been hoping that Washington University in St. Louis and other research universities would promise to protect participant (and patient) pregnancy-related information; announcing that they would fight attempts to force disclosure in any abortion-related lawsuits. So far, this has not occurred. Universities often have strong law departments and a pronounced influence on their communities, both as large employers and venerable, respected institutions. Ethics-based statements that some laws will not be complied with could have an outsized influence, and serve as a brake on those pushing enforcement and passing of ever more extreme abortion-related laws.

Since we currently lack pregnancy-related data confidentiality guarantees, in our group we have begun efforts to lessen the chances of our participants incurring extra risks from being in our studies – or even from being *asked* to be in our studies. Reducing our collection of potentially sensitive information to the absolute minimum is one step: even if subpoenaed or otherwise requested, we will not have potentially harmful records to disclose. Concretely, we have submitted changed our screening procedures, such that the participant is asked if any of a group of several exclusion criteria apply, only one of which is pregnancy (rather than asking about pregnancy separately). The participant then does not have to verbally state that they are pregnant, nor the experimenter note which of the exclusion criteria was met.

Participants will still need to complete the screening form immediately before scanning, but presumably anyone that reaches this stage will respond that they are not pregnant; if they are pregnant, the scan is cancelled and the screening form destroyed. This procedure reduces risk if we assume that recording “no, not pregnant” on a particular date has less potential legal trouble for the participant than a “yes, pregnant” response, which hopefully is the case. However, it is not unimaginable that an abortion lawsuit could have proof from elsewhere that the participant was pregnant on a particular date before the experiment, in which case their statement (or test result, in the case of studies requiring one) of not being pregnant on the experiment date could be relevant and damaging. At this time we can’t avoid using the forms with the pregnancy questions, but may start warning participants in advance that they will have to respond to a pregnancy question, and that we can’t guarantee their response will be kept private and only used in the context of the experiment.

The impact of the Dobbs decision (and in our case, Missouri state abortion trigger laws) on non-reproduction-related human subjects research is only a small subset of the harm from these laws, of course, but it is a new risk US-based researchers should consider. Human subjects protections are not trivial and must not be brushed aside, even if we hope no more abortion-related legal actions will occur. As scientists, our ethics, honor, and integrity require us to follow not just the letter but also the spirit of guidelines like the Declaration of Helsinki; we must work towards the absolutely best and strongest participant protections.

I hope that this essay has caused you to consider what data you are collecting, whether it puts your participants at new legal risk, and what you can do to minimize such risk in the short and long terms. Immediate actions such as changing how pregnancy is asked about or stopping collection of especially sensitive information seem to me the minimum ethically appropriate action; stronger, legally-binding guarantees of confidentiality may be needed soon for many types of human subjects research to continue responsibly in the United States.

UPDATE 3 August 2022: Our screening changes were approved, so I edited the relevant text and added a link to the tweet showing the approved version.

Yesterday I tweeted that a source I trust (and in a position to know) told me that Washington University in St. Louis counsel/administration told them that pregnancy-related research records will not be released in an abortion-related lawsuit, regardless of whether the NIH Certificate of Confidentiality is sufficient protection. That is good news, but I am troubled that it came via word of mouth; my source said I shouldn’t “hold out” for an official statement. It is hard to be confident without something concrete; even a technically-phrased memo or HRPO website note would be encouraging. It seems that we are being asked to act as if nothing has changed post-Dobbs, and trust that everything will be fine, but that's an awfully big ask for issues this consequential.

UPDATE 17 August 2022: Last month Jeannie Baumann wrote an article at Bloomberg Law discussing questions about Certificates of Confidentiality protections, including, "It’s unclear how the state law versus certificates would play out because it hasn’t been tested in court."

UPDATE 26 August 2022: Tamara J. Sussman and David Pagliaccio published Pregnancy testing before MRI for neuroimaging research: Balancing risks to fetuses with risks to youth and adult participants 

What happened in this fMRI run?

This is one of those occurrences (artifacts?) that is difficult to google, but perhaps someone will recognize it or have a guess.

This run is from a session in which a person completed four fMRI runs of a task sequentially. They did not get out of the scanner between these runs, nothing was changed in the protocol, no one entered the scanner room. Later participants (with the same protocol, scanner, etc.), have been fine. This study uses CMRR MB4 acquisitions, so we have an SBRef image for each run; the artifact is the same in the SBRef and functional run.

Runs 1 (not shown), 2, and 4 are normal, but run 3 is much darker than the others and has an obvious ghost-ish artifact, here are the DICOMs from each run's SBRef, allowing mango to adjust the contrast in each:

And here they are again, with contrast set to 1-15000 in all three images:

The functional run's DICOMs are also dark and have the prominent artifact; here's a frame:

When the run is viewed as a movie in mango the blood flow, small head movements, etc. are plainly and typically visible. The artifact does not appreciably shift or change over the course of the run, other than appearing to follow the (small) overt head motions (when the head nodded a bit, the artifact shifted in approximately the same way). The two surrounding runs (2 & 4) are typical in all frames (no sign of the artifact).

Given that this artifact is in the DICOMs, it's not introduced by preprocessing, and I am assuming this run is unusable. I'd like an explanation, though, if nothing else, so we can take any steps to reduce the chance of a recurrence. Our best guess at this time is some sort of transient machine fault, but that's not an especially satisfactory explanation. 

Any ideas? Thanks!

update 13 July 2022:

In response to Ben and Renzo's suggestions, I skimmed through the DICOM headers for fields with large differences between the three runs; if there are particular fields to look for, please let me know (this is a Siemens Prisma); I am not fluent in DICOM header! The most obvious are these, which I believe are related to color intensity, but I'm not sure if it's reporting a setting or something determined from the image after it was acquired.

run 2 (typical)

(0028,0107) Largest Image Pixel Value 32238

(0028,1050) Window Center 7579

(0028,1051) Window Width 16269

(0028,1055) Window Center & Width Explanation Algo1

run 3 (dark/artifact)

(0028,0107) Largest Image Pixel Value 3229

(0028,1050) Window Center 1218

(0028,1051) Window Width 3298

(0028,1055) Window Center & Width Explanation Algo1

run 4 (typical)

(0028,0107) Largest Image Pixel Value 31787

(0028,1050) Window Center 7423

(0028,1051) Window Width 15912

(0028,1055) Window Center & Width Explanation Algo1 

 

And here's yet another view from the three functional runs, in which I played with the contrast a bit. There's definitely a difference in which structures are brightest between the three.

 

now available: DMCC55B rejected structurals and ratings

We received several requests for the rejected DMCC55B structural images, since comparing images of different quality can be useful for training. Thanks to the assistance of Rachel Brough, we have now released T1 and T2 images for the 13 DMCC55B people whose initial structural scans we rated of poor quality, as well as our ratings for both sets of images (the initial rejected images and better repeated ones). 

The rejected structural images (with session name “doNotUse”) and ratings are in a new sub-component (postPublication_rejectedStructurals) of the DMCC55B supplemental site, rather than with the released dataset on openneuro, to avoid confusion about which images should be used for processing (use the previously-released ones available at openneuro).

troubleshooting run-level failed preprocessing

Sometimes preprocessed images for a few runs are just ... wrong. (These failures can be hard to find without looking, one of the reasons I strongly suggest including visual summaries in your QC procedures; make sure you have one that works for your study population and run types.) 

Here's an example of a run with "just wrong" preprocessing that I posted on neurostars: the images are all of the same person and imaging session, but one of the runs came out of the preprocessing seriously distorted.

And here's another: the images from the run at the lower right are clearly tilted and squashed compared to the others:

The above images are temporal means made after fmriprep preprocessing, including transformation to the MNI template anatomy (i.e., _space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz);  see this post for more details and code. 

How to troubleshoot this type of partial failed preprocessing?

First, note that this is not due to the entire preprocessing pipeline failing: we have the expected set of derivative images for the participant, and everything looks fine for most runs. This suggests that the problem is with not with the pipeline, or that something is unusual about the participant.

fmriprep (and I think most preprocessing pipelines?) is not fully deterministic: if you run the same script twice with the same input files and settings the output images will not be exactly the same. They should be quite similar, but not identical. We have found that sometimes simply rerunning the preprocessing will correct the type of sporadic single-run normalization/alignment failures shown above.

If repeating the preprocessing doesn't fix the failure, or you want to investigate more before rerunning, I suggest checking the "raw" (before preprocessing; as close to the images coming off the scanner as practicable) images for oddities. Simply looking at all the images, comparing those from the run with failed preprocessing and the other (successful) runs from the same session can often make the problem apparent.

Look at the actual functional images of the problematic run (e.g., by loading the DICOMs into mango and viewing as a movie): do you see anything strange? If the problematic run seems to have more-or-less the same orientation, movement, visible blood vessels, etc. as the non-problematic runs for the person, it is unlikely that the source of the problem is the functional run itself and you should keep looking. (If the functional run itself is clearly unusual/full of artifacts, it is most likely simply unusable and should be marked as missing.)

If the functional run seems fine, look at all of the other images used in preprocessing, especially any fieldmaps and single-band reference (SBRef) images. Depending on your acquisition you may have one or more fieldmaps and SBRef images per run, or per set of runs. For the DMCC we use CMRR multiband sequences, so have an SBRef image for every functional run, plus a fieldmap each session. Both the fieldmaps and SBRef images are used in preprocessing, but differently, and if either has artifacts they will affect the preprocessing.

How an artifact in the fieldmap or SBRef affects the preprocessing can be difficult to predict; both can cause similar-looking failures. In the two examples above, the first was due to artifacts in the fieldmaps, the second in the SBRef.

This is a screen capture of three SBRef images from the session shown in the second example. The numbers identify the scans and are in temporal order; image 41 is the SBRef for the affected run (a "tilted" brain); 23 for the correctly-preprocessed run above it. There are dark bands in parts of scan 41 (red), and it looks a bit warped compared to the other two; below is how they look in coronal slices:

All SBRef images look rather odd (here, there's some expected ear dropout (yellow) and encoding direction stretching), but the key is to compare the images for runs in which preprocessing was successful (23 and 32) with those for which it failed (41). The SBRef for 41 is obviously different (red): black striping in the front, and extra stretching in the lowest slices. This striping and stretching in the SBRef (probably from movement) translated to tilting in the preprocessed output above.

What to do about it?

Ideally, you won't collect SBRef or fmaps with strange artifacts; the scanner will work properly and participants will be still. If the images are checked during acquisition it is often possible to repeat problematic scans or fix incorrect settings. This is of course the best solution!

However, sometimes an artifact or movement is not apparent during data collection, the scan can't be repeated, or you are working with an existing dataset and so are stuck with problematic scans. In these cases, I suggest doing something like in this post: look at all of the scans from the session (and other sessions if relevant) and try to determine the extent and source of the problem. 

In the case of the fieldmap artifacts, every fieldmap from the scanning session was affected, but fieldmaps for the same person from two other sessions were fine. We "fixed" the failed preprocessing by building a new BIDS dataset, swapping out bad fieldmaps for good ones and changing the filenames accordingly. Before trying this I checked that the person's head position, distortion, etc. were quite similar between the runs. I do not really recommend this type of image swapping; things can go badly wrong. But it is something to consider if you have similar artifact-filled and good images from the same person and acquisition. With the SBRef image we have another option: SBRef images are not required, so we could delete poor ones (here, scan 41) and repeat the preprocessing without them. 

Neither workaround (swapping or deleting) should be used lightly or often. But it has produced adequate quality images for us in a few cases. To evaluate I look very closely at the resulting preprocessed images and BOLD timecourses, both anatomy and comparing the positive control analysis (see also) results for runs/sessions with normal and workaround preprocessing. For example, confirm that the changes in BOLD timed with your stimuli in particular visual parcels are essentially the same in runs with the different preprocessing. If different visual parcels show the stimulus-caused changes in BOLD depending on preprocessing, the workaround was not successful.