A few thoughts about Uber's breach
source link: https://cendyne.dev/posts/2022-09-19-a-few-thoughts-about-ubers-breach.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
A few thoughts about Uber's breach - 2022-09-19
Allegedly, an eighteen-year-old spammed an employee with two-factor authentication via push notifications on an employee with a known password. They got into the VPN and scanned for servers, found a file share without any access controls, and a script that could access break-the-glass credentials. With the highest level of credentials available, they then got effective root access to Slack, AWS, Google Suite, and active directory at Uber.
🍿for big tech company tonight
Someone in IT did not take security awareness training.
This is not developers fault at all.
Officially:
We are currently responding to a cybersecurity incident. We are in touch with law enforcement and will post additional updates here as they become available.
Allegedly:
Apparently there was an internal network share that contained powershell scripts...
"One of the powershell scripts contained the username and password for a admin user in Thycotic (PAM) Using this i was able to extract secrets for all services, DA, DUO, Onelogin, AWS, GSuite"
Seriously though, ya'll should review the risk you take with the zero trust flat network dream you've been sold.
Definitely enable number matching/verified push for all logins regardless. These kind of attacks have been happening all year, at very large orgs.
Update: A Threat Actor claims to have completely compromised Uber - they have posted screenshots of their AWS instance, HackerOne administration panel, and more.
They are openly taunting and mocking @Uber.
Who's not at fault?
Is the employee who got tricked at fault?
We're talking about a human here, who was intentionally harassed late at night; the easy way out was to follow the attacker's prompt.
Our security needs to be resilient to humans that make mistakes, and our security needs to be supportive of humans who need compassion.
If phishing a single employee can lead to everything in your infrastructure being compromised that easily, that employee is not to blame
The fact that this keeps happening and is enabled by the biggest companies out there needs to be recognized and changed.
Literally how to exclude accessibility needs people, old people, fatigued people, someone who just wants a burger but can't figure out why their card is rejected people, you know: a human?
This issue is not new, it is not a surprise; the abuse that opened the perimeter at Uber has been perpetuated through big companies selling insecure security.
Seeing an increasing amount of abuse of MFA prompt "push" notifications. Attackers are simply spamming it until the users approve. Suggest disabling push in favor of pin, or something like @Yubico for simplicity. In the meantime, alert on volume of push attempts per account.
1. After @jack was infamously hacked through a SIM-swap attack 2019, much of the world moved away from SMS-based one-time passwords (OTPs)
The apparent @uber hack may lead to a similar decline in Magic Links... here's what you need to know 👇
@ClerkDev @jack @Uber Surely the real answer is to require FIDO2? That would've stopped this entirely.
Break the glass
Allegedly, the attacker got break-the-glass credentials, which gave them access to literally everything their most senior IT personnel should have access to on a limited basis.
"Break the glass" credentials and accounts are used in rare circumstances outside the normal day-to-day operations.
Here's an example of an emergency where a break-the-glass activity is fine. As an IT administrator, you were just locked outside your account, and something requires immediate attention that cannot wait. All other IT administrators of equal or greater authorization are unavailable and you are unable to reset your account's status. You reach to break-the-glass to restore your own access, and then you return to your account to continue performing your duties.
Here's an example of a non-emergency where a break-the-glass activity is fine. As an IT or DevOps administrator, you need to provision a new AWS account for a new engineering team product. The company only creates a new product once a year and master payer account / management account is otherwise limited and off limits. You break-the-glass to access the management account to create a new account for the product, you document the new account's root user in a secured place, and set up SSO access to this account for the new team. Afterwards, you log out of the management account and continue performing your duties.
Credentials to break-the-glass accounts are arguably the most important secret material to secure for an organization. It allows an organization to respond to an emergency and to respond to infrequent requirements that require risky access to complete.
How do you protect emergency credentials?
If you want a product, the key words to search are: Privileged Account Management.
Uber specifically relied upon Thycotic. There are others too, but in short these technology solutions have at least these features:
- Unified repository for shared account credentials with excessive permissions.
- Independent (or SSO) authentication of employee access.
- Independent authorization to credentials.
- Auditing of all access to each credential.
A less technological solution would be to store a static password on a Yubikey and throw it in a safe. Yes, people do this, and I think it is far better than leaving break-the-glass credentials in a public location like Uber.
Click this area to load an embedded youtube video.
KSK Key Signing Ceremony (16 Jun 10) by ICANN
Crypto is bound to repeat every mistake learned over the past hundred years in finance
Should you go with a safe, have some sort of immutable log strategy in place where you document the date, the reason you accessed the key material, and which key material you accessed. Maybe even go old school and fax evidence of your activity to a third party who files it away in case of forensic analysis.
Lastly, and out of good practice, whoever has access to the keys on an ongoing basis should be separate from the person who momentarily uses the keys.
If you need help in person, it is usually up to the Chief Information Security Officer (CISO) to provide privileged access to you for a short amount of time. In my opinion, a CISO should not be using the credentials. Their authority should be to delegate access, not to act with access.
So, in a way, Thycotic is a virtual extension to CISO. As a Privileged Account Management product, Thycotic is fulfilling the role of delegating authorized access to an authenticated person and auditing the action.
So is Thycotic at fault? I think not.
How did Uber protect emergency credentials?
Again, this is based on alleged information.
Here's what I would expect a sane CISO to implement with a privileged account management (PAM) product:
- Users authenticate with the PAM through an SSO provider.
- All staff besides the CISO and their immediate deputies have limited authorization to resources within the PAM.
- Any disaster recovery credential for the PAM is stored securely in a place that can only be accessed by the CISO and their immediate deputies.
Instead, Uber had a potato-for-brains PAM admin put their credentials in a script for Thycotic on a file share that any employee with legitimate VPN access could have accessed and then abused.
Simple graph mapped to MITRE ATT@CK and TA TTPs used to breach UBER
https://whimsical.com/uber-breach-7JNtVoq4Tu73kBXzoisuiQ
A "fun story"
There are similarities. My breach and Uber's 2022 breach had unrestricted credentials in the clear where any employee could have found them.
Now I am the one who delegates authority to our AWS account. The rest of the company's IT infra is outside of my reach under the CIO. I may not be a CISO, but I think my experience and responsibilities are getting close. I am also interested in this, but I'm not sure yet on if I should jump into it.
Uber's security history
RM @FrankPallone's statement on Uber's failure to protect the data of 57 million consumers 📱🚗 💳 👎
.@Uber: "Oh hey, Mark had to testify at Congress because he didn't address their data breach... Maybe we should send our users a quick email?"
@TPioreck So, you mean the same as is going on right now?
I mean, I literally was called to testify before Congress last year regarding a breach cover-up done by Uber, with the extortion/hush money paid via a complicit bug bounty platform.
Corps already underreport, it's why the regs exist
BREAKING: Former Uber CSO charged for covering up the company's 2016 security breach
https://www.zdnet.com/article/former-uber-cso-charged-for-2016-hack-cover-up/
Former Uber CSO Faces New Charge for 2016 Breach https://j.mp/33ZyEor
Uber has been hacked, and it looks bad. The hacker got in through social engineering and allegedly found a network share full of Microsoft PowerShell scripts that included Uber admin usernames and passwords to let them breach AWS, G Suite, and more 🥲 https://www.theverge.com/2022/9/16/23356213/uber-hack-teen-slack-google-cloud-credentials-powershell
This AM, I’m listening to the trial of former Uber exec who’s facing criminal obstruction charges for his role in paying hackers who breached the company in 2016.
Uber’s CEO is expected to testify today.
ICYMI, Uber is now responding to another breach: https://www.wsj.com/articles/uber-says-responding-to-cybersecurity-incident-11663299504
In short, Uber had a breach, the CISO at the time covered it up and paid a ransom, and then got thrown under the bus as a criminal.
Now Uber is having another breach because the CISO or someone with equivalent access left their credentials in the clear.
Lots of screenshots going around about Uber but this one shows how wide the hack is.
"Security Response Break Glass Service Account" password 🔥
What Uber could have done better
Perimeter security would be improved by using WebAuthn and hardware tokens for all employees and contractors. The use of push-based two-factor authentication is dangerous because the human can be harassed.
Device profiles should be a requirement on the perimeter. A certificate specific to the device (and hopefully bound to its trusted platform module) will reduce the likelihood of perimeter compromise.
Account lockouts should be the norm when a two-factor fails several times in a row. I recommend five failures. Yubikey FIPS security tokens lock out WebAuthn after three failures. Allegedly the failure was performed over a hundred times to harass the employee who gave in and approved the attacker's access.
Hard-coded credentials must be avoided. IT and engineering should be educated that a better alternative is to prompt for credentials in the script or to perform a brief SSO handshake. Ongoing education for this situation should be regularly covered and reference examples available to use.
IT should adopt source control and use tools like GitGuardian to scan and detect credentials in scripts so the credential is invalidated as soon as possible.
Human made credentials must be avoided, especially for disaster recovery and emergency use. It is not unheard of that these credentials might even be single use. A culture of using generated passwords should be adopted at all depths. Secret scanners can detect and alert high entropy generated passwords. Use this to your advantage.
Credential rotation should be regularly performed for shared accounts, including disaster recovery accounts (and a backup account in case that is destroyed). This should involve a documented process with verification steps to ensure it is smoothly executed and meets expectations. Documentation of credential rotations should be performed and preserved.
Privileged Account Management break-the-glass credentials must be protected and upon use queued for immediate rotation. The use of these credentials should set off alarms and be very visible.
Penetration tests should be regularly performed by multiple vendors with access to the internal and application network. Uber should place multiple flags (capture the flag style) in their infrastructure, services, source code, application containers, logs, chat rooms, etc. to measure coverage that each vendor achieved as a simulation of what an attacker could have found.
Honeypot credentials should be included in the privileged account management product, their retrieval should cause an on-call system to page / call the CISO, the on-call, and a situation manager.
Conclusion
The industry needs something like OWASP that provides ongoing risk assessment and guidance for CISOs and CIOs in informing their security policies. Turns out, we have MITRE! Unfortunately, I personally find MITRE to be overwhelming. Its incredible trove of techniques, detection strategies, and mitigations could be made more accessible to senior officers.
As an example, T1611 - Escape to Host eloquently describes the issue, examples of tools or malware that causes an issue, mitigations like read-only containers and pod security policies, and detection strategies like process spawn monitoring on container hosts.
I can understand this because of my background as a developer, DevOps administrator, and AWS infrastructure subject matter expert. Not every CISO has that technical knowledge. Likewise, I could not gauge anything in this repository that involves Active Directory. Some CISOs do have that experience.
I have heard from G Mark Hardy on CISO Tradecraft that CISOs often lack staff. Given the spectrum of issues on MITRE, I believe that we need guidance and recommendations on what specialties should report to a CISO to discover, formulate, enact, and verify security policies from information and application security.
Just as a CEO has reports for marketing, finance, sales, human resources, and so on, there are too many possible issues for one person as CISO to successfully grapple with security. This needs to be recognized and supported by chief officers, investors, and executive boards if we are to see an improvement.
Uber's response
Security is expensive. It is seen as a cost center. Until something like this happens. Unlike Patreon, maybe Uber learned that they need to hire more security people.
Whoa @Patreon laid off their ENTIRE security team.
Wouldn’t trust my data there. Also there’s some amazing talent to scoop up.
Firing your security team is dumb.
You cannot outsource your risk.
Any sufficiently large business should have security staff to reduce risk.
"Oops we got busted and leaked all our customer data" cannot be followed by "Because so and so at another company didn't do their job"
Fact: an organization's budget for cyber security increases only after an incident 🥹
#cybersecurity #hacking
You cannot delegate or outsource responsibility for risk. It is on you to protect the value of your organization and the privacy of your employees and customers. Today, it is on you to convince investors and the board that security teams are worth the expense. Tomorrow, maybe investors and the board will see things differently. Who knows.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK