Doing research at TTS
Research isn’t only usability testing. Testing our assumptions by actively engaging with the world around us is at the heart of how TTS works. We do research when we meet with stakeholders to learn about a project, when we investigate and compare tools and systems, and when we find members of the public to tell us about their experiences with the federal government. Research can include questionnaires and surveys, as well as analytics.
These guidelines should be familiar to anyone who's done research in the private sector or in academia. However, as public servants, we need to make sure that we're following a few basic principles.
- Respect: We honor people's opinions and choices, only blocking actions that are obviously going to harm others. We make sure that anyone participating in our research is doing so of their own free will, and that they have enough information to make that decision responsibly.
- Beneficence: We do our best to further people's best interests, and we avoid actions that might hurt them. This goal extends to society at large: before doing research, we need to consider whether what we're doing is in the best interests not just of individual participants but of the organizations and the country that we serve.
- Justice: We do our best to ensure that we compensate research participants in some way for the time and energy that they contribute. We don't have to pay money, but we do have to give something back to participants. Sometimes that means just saying "thank you" and letting them know how much their input has helped improve government services. Other times, it can mean making a card or handing out cookies or fruit. Participation incentives, which are common in the private sector, are more tricky here in government. So please ask one of the Research Guild leaders if you think they're necessary for a specific study.
While only scholarly researchers are strictly obligated to follow these principles, they are widely accepted in the United States as , external,a set of overarching values guiding all investigations involving humans.
This document explains two main tasks we do to operationalize our basic principles: informing and getting consent, and managing personally identifiable information (PII).
Informing and getting consent
TTS researchers have an obligation to ensure that our participants know what it means to participate in our research and how we plan to use the information we collect. Before getting consent, ensure participants understand:
- They cannot expect to receive compensation for their participation (unless otherwise agreed to with our agency partners — note that this is a legal issue!)
- The types of information we may record (this could be name, direct quotations, photos, screen captures, etc.)
- How we will share information we collect. Usual options include: No one ever (off the record), within the immediate project team, on a need-to-know basis (our default), and with the general public.
Getting consent
TTS gets explicit consent from anyone who participates in our moderated research, generally by asking that our research participants sign a "design research participant agreement". TTS maintains a , external,TTS-only, template for internal use, and an , external,example participant agreement for sharing with interested parties. Please make a copy of the participant agreement document template, put it into your project folder, and edit the highlighted text for each of your research studies. If your participants are likely not fully literate in English, consider having the form translated to the correct language or working out a process for verbal consent using a translator.
When sending participant agreements electronically, please use our , external,TTS-only, participant agreement email template. This template clearly specifies how people can opt out of a study and/or request that we do not contact them in the future.
Compensating user research participants
Background
Compensation of research participants has been a long-standing issue for teams at TTS. Although we've investigated a variety of compensation methods over the years, conflicts with laws and practices related to appropriations and regulations made it difficult or impossible to compensate participants. Until relatively recently, we could conduct user research, but only with participants who were willing to donate their time.
Fortunately, we now have an approved process to compensate research participants. We'll continue to update this page as we streamline and refine our practices.
Should I compensate research participants?
In general, if you're conducting research with members of the public, you should compensate them for their time and expertise. Offering compensation is critical to reaching the people most in need of government services. Without compensation, we're likely to hear only from the people who can afford to volunteer their time. Compensation makes it easier to recruit participants, especially from underserved communities, and helps build trust that their input is valued.
What should I do if I cannot compensate participants?
If you can’t provide compensation, There are other ways to recognize the value of participants’ knowledge and experience. These can include:
- Resources or additional information on the topic
- Offer to find out answers to questions they may have
- Sharing outcomes of the research, and how the research impacted the final product
What process do I need to follow?
TTS User Research Participant Compensation Justification Checklist: Each project team must document their need to compensate user research participants by filling in a , external,TTS-only, User Research Participant Compensation Justification Checklist template. Teams can use this , external,TTS-only, sample justification for reference.
Teams don't need to submit this justification, but they must keep a copy in the , external,TTS-only, user research compensation justifications folder. This is part of the expected authorization process so OGC can confirm that we are consistently basing a decision to provide compensation according to the guidance Counsel has provided. The documentation is also necessary in case there is a future question about a decision to offer compensation.
The justification answers the following questions:
- Does this work directly advance GSA’s or the client’s statutory mission and objectives?
- Does the benefit to the government outweigh the benefit to the individual?
- Has the agency attempted to accomplish its goals through means other than paid user research?
Interagency Agreement (IAA) Attachment B - User Research Compensation Authorization: If your project has an Interagency Agreement (IAA), the IAA must include an , external,TTS-only, Interagency Agreement (IAA) Attachment B - User Research Compensation Authorization. If Attachment B is not part of the IAA, work with your account manager to get it signed by an authorized official at the partner agency.
In the agreement, the partner’s OGC will need to provide TTS a concurrence that “the benefit accruing to [the requesting agency] in obtaining this feedback outweighs the personal nature of this expense. [The requesting agency] has also determined that the feedback from this user research will directly advance the statutory mission and objectives of [the requesting agency]."
How much compensation should I offer?
While TTS is working to standardized base rates, the IAA , external,TTS-only, Attachment B - User Research Compensation Authorization sets a rate of $75 an hour for moderated research sessions.
From 2020 research on third-party recruiters, the median range was between $50 to $70 per person per hour (this does not account for recruiting expenses an organization might incur). Recruitment and support averaged about $125 per participant, which means that the total cost for a moderated session would be about $195 per person.
In 2022, TTS considers $75/hour as a starting point for compensation. Teams should consider factors of equity and complexity of the research when determining compensation for participants on their project.
How do I actually distribute the compensation to research participants?
There are a few different ways to distribute compensation to research participants.
Pay participants via gift card: Compensation can be distributed to participants via digital/e-gift or physical gift cards through , external,GiftBit. Gift cards can be purchased by submitting a , external,TTS-only, purchase card request. The total amount of the purchase must be under $10,000 per project.
For more information, see how do I complete a micropurchase request? in the handbook.
Use usertesting.com: Teams can also use usertesting.com, if that’s applicable. , external,TTS-only, There are instructions available on how to use usertesting.com for research. Access to usertesting.com varies by business unit. Consult your supervisor or manager to see if your team is able to use this service.
Step-by-step checklist for compensating participants
Follow these steps when you are ready to begin the compensation process.
- Complete the , external,TTS-only, TTS User Research Participant Compensation Justification Checklist. Save it in your project folder, and save a copy in the , external,TTS-only, user research compensation justifications folder.
- If working with a partner agency, confirm the IAA includes , external,TTS-only, Attachment B - User Research Compensation Authorization.
- Plan your method of compensation. For gift card purchases, see the , external,TTS-only, TTS PeopleOps process for purchasing gift cards for user research compensation ($10,000 or less)
Managing Personally Identifiable Information (PII)
We work in the open, but we need to be mindful of discussing agency partners, collecting any PII, or accidentally disclosing PII.
What is PII?
PII is information that can be used to distinguish or trace an individual's identity, either alone or when combined with other information. Because there are many different types of information that can be used to distinguish or trace an individual's identity, the term PII is necessarily broad. The definition of PII is not anchored to any single category of information or technology. Rather, it requires a case-by-case assessment of the specific risk that an individual can be identified using information that is linked or linkable to said individual. For that reason, Social Security Numbers, Alien Registration Numbers, and driver's license numbers are always PII. Email addresses, phone numbers, and mailing addresses are sometimes PII, and when presented in combination with a first name, often become PII. Photos of people's faces are almost always PII, and that's why we always ask before taking photos and get explicit statements about if we can share them.
Sensitive PII is information which, if shared, could seriously harm or embarrass someone. Unique identifying numbers and biometric data is always sensitive: In general, combining:
- Citizenship or immigration status
- Ethnic or religious affiliation
- Medical information
- Sexual orientation
- Criminal history
with PII always makes that PII sensitive. However, determining what is and isn't PII, and what is or isn't sensitive PII can often require some reflection.
Much of our research involves federal employees. When we are talking to federal employees (not contractors, and not vendors) about their work, they generally have no reasonable expectation of privacy. After all, the material may have to be disclosed pursuant to a judge's order, a Congressional request, or a FOIA request.
As researchers, however, we need to follow the principle of beneficence. So just following the legal guidelines on PII isn't enough if you're collecting information that you know might harm or embarrass your participants if it became generally known, whether or not those participants are federal employees. Different agencies and organizations may have different levels of sensitivity and exposure concerns about types of personal information (identifying or not) you collect and how you use it. Attributing quotations with agency affiliation, for example, may be more sensitive than first name and photo. In those cases, the principle of beneficence demands that we think carefully about protecting participants before sharing working notes or finished reports, even if we have satisfied the law.
Storing PII and sharing research data
If you are collecting PII in the course of research, the best place to store it is in a locked-down folder on Google Drive that is only accessible to the core research team, as Google Drive is a GSA-approved place to store PII. (The only other GSA-approved software for storing PII is Salesforce, which we don't use for research). Raw interview notes, which may contain PII, should be stored in this folder. The core research team should also create a document that gives each participant a pseudonym (eg. a codename or number such as "PO1"). Analysis & synthesis documents and research reports, which are typically accessed by additional members of the project team, should only contain those pseudonyms or codes, unless you have obtained explicit permission from a participant to use, for example, their likeness in a photo or video clip. If you have any paper documents with PII, they always go into a locked file cabinet.
A good rule is to check with your project lead before sharing information outside the immediate team. What has been approved by GSA may not be approved by partners. Check the Design chapter page for detailed information about who can see what, when. The social media, collaboration, and security classes in GSA's Online University can also be helpful in managing access.
Be especially mindful about using Slack during interviews for sidebar conversations. No PII should go into Slack channels unless your participant has explicitly consented to have PII shared with the general public. And even then, be aware of the potential level of visibility any information will have once you post it in Slack.
Carefully restricting access to personally identifiable information is a matter not just of people's right to respect but of their right to , external,privacy. For more information, please see , external,TTS-only, this 30-minute overview of privacy as it relates to research narrated by the GSA Privacy Office.
Bottom line: If you have questions about sharing information, just ask. See the page on sensitive information. If you're not sure if you're collecting PII or need help with policy guidance, you can ask on Slack in , external,TTS-only, #g-research. If you are not sure where the right place might be to store any given file, or what access permissions to grant, you can ask the Tech Portfolio.
Join the research guild!
- Learn more about the research guild in our , external,TTS-only, README.
- Learn more about guilds at TTS.