Here’s a scary story about the government’s preparations for the 2020 census and how it was exposed to “potentially catastrophic risk” when it lost some passwords.
Costs for the census have soared in the last few decades. After the 2010 census, the Census Bureau decided to embrace technology in 2020 to make the count more accurate and keep costs down. For the first time, households will be able to fill out the census form online, and field workers will record information on a smartphone app.
It’s a large project that requires computing power and storage space. The Census Bureau signed up with Amazon Web Services to get the secure cloud servers they needed.
So far, so good. Many of the world’s governments and large companies have outsourced their computing to Amazon. Amazon Web Services (AWS) powers far more of the world than you realize. AWS GovCloud (US) is designed for security and compliance, to the point that Amazon promises it is “managed by U.S. citizens on U.S. soil.” It’s used by US government agencies like the Department of Justice and Department of Homeland Security and many more.
When a new AWS environment is set up, a “root user” is created. It’s a crucial moment.
One admin to rule them all
Let’s take a break and think about administrators.
There has to be somebody who has ultimate power.
• Your computer You are probably the administrator of your computer. You can install programs. You can add another user. You can delete files and programs. You can even wipe it out and erase everything.
• Company network If your company has a network, it probably has a domain administrator. That’s the account with unlimited power to do anything – add users, change passwords, decide which computers have permission to read files, and even destroy the network. There might be other admins with limited powers; for example, if you’re in a big company, help desk staff can install a program on your locked-down computer, but they don’t have permission to delete your account if you’re rude to them.
North Korea reportedly stole the credentials of a system administrator at Sony Pictures, one of the ways they got access to install malware and take down the entire company in 2014.
• Cloud services There is a global administrator for your business accounts with Office 365 and Dropbox and Box and Salesforce. Again, there has to be one account with unlimited permission to reset passwords and set up a new user and determine which folders are restricted.
You get the idea. The “root user” or “domain administrator” or “global admin” can do anything to a network. Someone with the credentials to log in as a global admin can lock everyone out of the system, install viruses, and leave hidden traps for spying later.
Back to the Census Bureau
The Census Bureau created root users for its eight AWS environments. The root users are all-powerful. They can get into every database, install backdoors, set bombs.
The Census Bureau lost the passwords.
You know that feeling when you write a complicated password on a Post-It and then you lose the Post-It? It was like that except it was the password for the entire US census.
Government agencies and large companies start with an all-powerful root user. The very next task is to set up elaborate, complex chains of authority to decentralize things – it’s too dangerous to have a single password that unlocks everything. Do you know those scenes in movies (and Stranger Things) where the bomb can only be set off by two people with two keys turning at the same time? It’s like that.
Once the chains of command are in place, the initial root user is disabled because it’s too risky to have a single login name and password with unlimited power over, say, the servers running the US Census. “AWS, industry best practices, and the Bureau’s security documentation all recommend that GovCloud root user keys be disabled after initial environment setup.”
The root user accounts were not disabled. Each one was accessed at some point – somebody had the passwords – but no one at the Census Bureau noticed that the passwords had been lost for as long as two years.
In 2018, the Office of the Inspector General did a security audit and discovered that no one had control of the passwords for the active root user accounts, meaning the entire 2020 Census infrastructure had an unlocked door and no way to discover if anyone had tested the knob.
It took six long weeks for the bureaucracy to sort it out with AWS and secure the accounts. In a report about the security lapses, the Inspector General notes that the delay “demonstrates the Bureau’s inability to have stopped a potential attacker with stolen root keys from modifying or destroying all cloud system resources hosted in its GovCloud environments.”
During those two years, the Bureau had no way to know if its data was secure, if the servers had been hacked, or if the Russians had planted back doors to use later.
The Inspector General’s report states: “Fortunately, we did not find evidence of the lost root keys being used maliciously. However, the Bureau could not know if they had been stolen or sold and, having lost the root user keys, would have been powerless to stop an attacker from causing irreparable harm to the cloud environments. Therefore, we conclude that the Bureau exposed the 2020 Census preparations to potentially catastrophic risk by not securing the root user accounts.”
There are two lessons:
(1) Don’t lose your passwords. Especially your LastPass master password.
(2) Admin passwords are important. In the next article, I’ll give you some tips about protecting global admin access to your business Office 365 account.
Trackbacks/Pingbacks