MVPA Meanderings: "just enough" BIDS 3: questionnaire data following NDAR format

This is the third (and last?) in a series of posts (and a talk) describing what I think of as "just enough" BIDS (and advocating for its use). The first post introduces BIDS and describe how organizing datasets into six top-level directories can simplify and improve sharing, storage, and management. The second post goes into more detail about the BIDS standard, and how to use it in the rawdata subdirectory. This third post has the least BIDS of the series, though adds discussion of a different standard.

"questionnaire" data?

The terminology is imprecise; by "questionnaire data" I mean individual difference questionnaires or surveys, the sort of standardized tests given to each participant at the start of an experiment to measure personality or other traits. The BIDS standard calls these Phenotypic and assessment data, and categorizes it as an optional modality agnostic data type.

In my opinion, questionnaire data is one of the most complicated data types to share in a useful way because there are so many variations and arbitrary-seeming (but critical) details. For example, several studies I've been involved with asked participants to complete the Mindful Attention Awareness Scale, MAAS. To share the data I could tell you that a participant answered 4 on the first MAAS item, but that doesn't mean anything unless you know which question I asked first and what answer "4" corresponds to.

It helps if I share a description of the questionnaire I used along with the data, including its questions, possible answers, and scoring. For example, I could create a .json file:

{  
  "MeasurementToolMetadata": {  
   "Description": "Mindful Attention Awareness Scale, MAAS",  
   "TermURL": "https://doi.org/10.1037/0022-3514.84.4.822"  
  },  
  "maas1": {  
   "Description": "I could be experiencing some emotion and not be conscious of it until some time later.",  
   "Levels": {  
    "1": "Almost Always",  
    "2": "Very Frequently",  
    "3": "Somewhat Frequently",  
    "4": "Somewhat Infrequently",  
    "5": "Very Infrequently",  
    "6": "Almost Never",  
    "7": "I prefer not to answer"  
  },

This sort of .json listing the questions and responses is what's recommended in the BIDS standard; the above could be saved as the first part of /phenotype/maas.json and the participants' responses in /phenotype/maas.tsv (one row per participant, the responses to question 1 in the maas1 column).

Data Dictionaries and NDAR

Releasing a .json describing each questionnaire along with the responses is certainly better and more useful than sharing the responses alone, but not a complete solution, in my opinion. The underlying problem is that there's no single obvious "correct" way to describe questions and responses even in as simple a case as the above MAAS question 1. I wrote the .json using integers for the response coding, with 1 the most, 6 the least, and 7 for no answer, but it seems equally valid to have used other numbering schemes (e.g., 0 for no answer, 1 for "almost never"), letters (e.g., "AA", "VF"; "A", "B"), or strings (e.g., "AlmostAlways", "VeryFrequently").

Thus, each researcher is likely to come up with a somewhat different definition for the same questionnaire, and we have the problem of how to translate between the different schemes: code I wrote to score items with 1 = "Almost Always" will need to be changed if 1 = "prefer not to answer" in your dataset. In bad cases it can be difficult to even tell that we gave the same questionnaire, not helped if copyright or other issues mean we can't include the actual items.

A more complete solution is for everyone to use the same data dictionary (aka codebook) for each questionnaire: instead of each of us inventing our own item and response coding, we use the ones from the agreed-upon data dictionary. This avoids the need for translation: if we've both used the same definition, we know that 1 corresponds to "Almost Always" for both of us.

The most complete public set of data dictionaries I know of are those of the NIMH Data Archive, (aka NDAR or the NDA), backed up by Chris Markiewicz (thanks!) at codeberg, so I advocate using them whenever possible.

Continuing the example, the MAAS is in NDAR as maas01, and the definition can be viewed online or downloaded as a .csv.

Comparing this definition to my example .json above, we can see that the first question is named "daytoday_1", not "maas1", but the question itself and response mapping match. I personally think that maas1 is a nicer label than daytoday_1, but we use daytoday_1 in our files because that's what's in the data dictionary.

Strategy 1: use NDAR definitions for the phenotype jsons

You could follow the BIDS standard Phenotypic and assessment data section, writing the .json and data .tsv to match the NDAR _definitions.csv. If each phenotype .json clearly states which NDAR dictionary entry it corresponds to, the problem of making questionnaire data interpretable across studies is largely solved.

Strategy 2: use NDAR definitions and file format

I recommend storing questionnaire data in a derivatives subdirectory, in NDAR file format (csv):

I do not have a strong opposition to putting them under /phenotype/, but it seems most natural to think of the questionnaire results as a type of derivative: they each contain data for multiple participants, and the response files were processed and reformatted to match the NDAR definitions. It's also easier to collect all the files most often needed by collaborators for their analyses under the single (derivatives) subdirectory.

My preference for NDAR format csv over BIDS tsv is similarly pragmatic: I and many collaborators are familiar with it since we've previously needed it to make files for official NDA data submissions. Since we already had procedures for making and analyzing questionnaire data with NDAR format files, we've kept using them, even for studies without a requirement to submit data to the NDA.

So, I don't really like using the NDAR format as much as I dislike "reinventing the wheel": if a more-or-less reasonable standard for working with a data type exists, I want to use it. But the temptation to rename the src_subject_id field to the BIDS participant_id is strong.

Musings

A lot of questionnaires are in the NDA Data Dictionary, but of course not all of them, which brings us back to having to write a new definition file. If you're working on a project which is required to submit data to the NDA, the admins will review your new definition and add it to the public dictionary. But I'm not aware of a public repository for non-NDAR project definitions. Something like for TemplateFlow would likely work well; I should probably make a formal suggestion to the NiPreps group (unless someone knows of an alternative?).

For now, we're saving the definition and template files in /code/NDAR_templates/, appending _custom to files we either wrote ourselves or substantively edited. In the below example, the cdrs01, ffmq01, and maas01 questionnaires are unchanged from the NDA data dictionary, medquality is a new survey (not in the dictionary), and we had to add a field to dacssf01's definition. The _notes.txt files give information about each custom template; when needed we include pdfs or other supporting material.

Here's another implementation example, showing a bit of one of the participant summary files from /analyses/questionnaireKnitrs/ (knitr being my favorite R markdown flavor); the knitr code reads the /derivatives/questionnaires/ NDAR-format .csv files, so are straightforward to use for multiple studies.