Something went “bump” in the night (or the day)? This document explains what to do when responding to a security incident. See What is an incident? if you need help determining whether something counts as an incident.
You must report IT security incidents or suspicious activity
An “incident” or “information security incident” is a violation - or an imminent threat of violation - of information security or privacy policies, acceptable use policies, or standard security practices.
If you detect any unusual or suspicious activity on your computer, DO NOT turn off your computer. By turning off the computer, valuable evidence may be lost. If you observe or suspect prohibited material or programs on GSA systems, or inappropriate use of GSA systems, report it immediately to the [GSA IT Service Desk]
It is critical that you notify GSA IT within 1 hour of suspected incident and provide all available information to assist the response team with triage. If email is unavailable, contact them another way.
Potentially sensitive data must never be shared in Slack, GitHub, or transmitted via email. Include the hyperlink to the GSA Google Drive folder in the top summary of the GitHub Issue. At this time, GSA Google Drive is the only approved method of secure data transmission during an active incident
If classified information is part of the incident, do not attach the information to your report. Wait for instructions from the GSA Incident Response (IR) team. If you do not receive a timely acknowledgement of your report, you can phone the IR team via the numbers listed in Section 3.1.1 of the GSA IT Security Procedural Guide.
Remember: it’s totally OK — and encouraged — to fail towards the side of reporting something. Organizations with healthy incident reporting systems see a lot of false alarms, and a lot of low severity reports. This is good, because it indicates that people feel comfortable reporting day-to-day issues. The more we do it, the better we’ll get at it. And this is ultimately the goal, because then when something really serious happens, we’ll be practiced at handling it smoothly and efficiently.
For questions about GSA’s Incident Response Program, contact the GSA Incident Response (IR) Team at email@example.com.
If you receive a suspicious email, follow these steps:
- Do not click any links in the email. Do not delete it yet. You may mark it as spam.
- Follow the “To report a suspicious email” directions on the Phishing Emails and Scams InSite page.
- This should instruct you in how to use the Cofense tool (the little fish icon that should be on the right) to report the phish.
- GSA IT will triage and let you know what they need you to do, if anything.
If you also clicked on a link in a phishing email, follow these steps to report to GSA’s Incident Response (IR) team:
- Send an email to firstname.lastname@example.org and email@example.com. Please include Security Incident in the subject line, along with a brief description of the issue (Ex. Clicked on link in phishing email).
Reporting other incidents
To report a security incident, follow all of the steps below:
Please note that incidents need to be reported within one hour of being identified. This isn’t “within an hour of happening” but “within one hour of you becoming aware of the incident.” The idea is to make sure we’re promptly looping in the right people. So, as soon as you’re aware of a problem, follow the above steps.
- Send an email to firstname.lastname@example.org, email@example.com, and firstname.lastname@example.org within 1 hour of identifying an incident. Please include Security Incident in the subject line, along with a brief description of the incident (Ex. security token committed to repo).
- When emailing GSA’s Incident Response (IR) team, please include as many details as possible, including relevant URLs, repos, and a link to related GitHub issue (if applicable).
Open a GitHub issue in the security-incidents repository, tracking the status of these steps. This will automatically populate the #incident-response Slack channel that is monitored by the TTS Tech Portfolio.
Do not delete any potential evidence or modify the evidence without instruction from the GSA's Incident Response (IR) team.
- Check the system’s specific Incident Response Plan
Following notification to GSA, the GSA’s Incident Response (IR) team will contact you requesting more information.
- Examples of questions that will be asked are 1) What systems do these secrets allow access to? 2) How long were the secrets available on the exposed system? 3) Were these secrets publicly available? If not, how many people had access to the secrets and what were their roles on the project? 4) How quickly can/were the secrets rotated? 5) Has the message - or file - containing the secrets been deleted (and removed from archive) in the exposed system?
What is an incident?
First: it’s always OK to err on the side of reporting! TTS’s and GSA’s incident response teams are good at their job, and they are totally used to false alarms. You’ll never get in trouble for pinging them about something that turns out not to be an issue! Indeed, you’ll never get in trouble for pinging IR at all. The most effective security early warning system is attentive staff, so report early and report often!
What is an incident? An occurrence that actually or potentially jeopardizes the confidentiality, integrity, or availability of an information system or the information the system processes, stores, or transmits or that constitutes a violation or imminent threat of violation of security policies, security procedures, or acceptable use policies.
See GSA’s Insite: Report IT Security Incidents or Suspicious Activity if you need help determining whether something counts as an incident.
Responders to an incident (as defined above) are entitled to take steps to remediate the immediate problem without worrying about whether or not the work is billable for up to eight person-hours of work per incident. Fix the problem, and then worry about how we will account for your time.
TTS tailored examples of incidents
- Putting sensitive data in a public Slack channel
- Exposing a secret token in a GitHub repository
Preventing future incidents with Git Seekrets
If an incident involved exposing environment variables or private configuration data, consider adding a rule to the Git Seekrets installation to prevent further incidents across TTS.
Setting up My Drive for sensitive information sharing
Ensure you’re creating the folder as part of My Drive and not within a pre-existing folder. Use this GSA Google Drive folder for any potentially sensitive data and/or files. Add static files, Google Docs, or Google Sheets as appropriate, and add a comment to any information you think is critical to the investigation.
Means the availability of the services we provide. If an app goes down or something we expect to be running stops running, those are availability issues. This only refers to production systems (it’s fine if your demo app crashes), and only to unexpected downtime. If you decide to shut something down temporarily for maintenance — go for it, not an incident.
think: secrets. Personal information (PII) — names, phone numbers, social security numbers, etc — is one kind of secret, but so are your passwords, service credentials, internal non-public documents, etc. Any time you suspect that confidential information may have been leaked outside TTS, you should open an incident.
The soundness/fitness of purpose of our systems or information. So if a backup was lost, or an app stopped logging for a while, or some documents got deleted, those are integrity issues. Sometimes these can indicate deeper incidents (like an attacker deleting logs to cover their tracks), so it’s important to report these.
- Incident? Report in the #incident-response Slack channel
- Questions? Ask in the #g-security-compliance Slack channel
- You can use this link to quickly send an email to IT Service Desk, GSA IR Team & TTS Tech Portfolio at the same time.
- cloud.gov: cloud.gov support team or #cg-incidents Slack channel
- cloud.gov: #login-incidents Slack channel
- Need to find the Information System Security Officer (ISSO) or Information System Security Manager (ISSM) for a specific system? See the directory of contacts for GSA systems. (You’ll need to be on the GSA network or VPN to access the directory.)