How the “Solar Winds” hack went down

Image Source: Republican Policy Committee, US Senate

Nearly every day in our digital lives, we see a multitude of apps on our phones or laptops get updated – innocuous maintenance updates that invariably include words to the effect of “This release addresses maintenance updates, stability improvements, and bug fixes.” It was during one of these routine releases late last year that triggered one of the more immensely damaging and wide ranging hacks the US has ever seen, commonly known as the Solar Winds hack. It is named after the company of the same name, and was triggered by an update of their widely used Orion software that helps companies and major technology operations in the US Government monitor network activity. Like a tiger laying in wait, the seemingly boring update triggered what many believe is one of the largest and most damaging hacks the US has ever seen. For many who analyze this sort of thing, the true impact of this will not be able to be calculated for months if not years.

In the months since it happened, NPR has been doing its research to learn more about how exactly it went down and where the checks and balances failed. As was reported when it happened, the brilliance and sophistication of the hack can not be understated.

Network monitoring software is a key part of the backroom operations we never see. Programs like Orion allow information technology departments to look on one screen and check their whole network: servers or firewalls, or that printer on the fifth floor that keeps going offline. By its very nature, it touches everything — which is why hacking it was genius…

The SolarWinds attackers ran a master class in novel hacking techniques. They modified sealed software code, created a system that used domain names to select targets and mimicked the Orion software communication protocols so they could hide in plain sight. And then, they did what any good operative would do: They cleaned the crime scene so thoroughly investigators can’t prove definitively who was behind it. The White House has said unequivocally that Russian intelligence was behind the hack. Russia, for its part, has denied any involvement.

“The tradecraft was phenomenal,” said Adam Meyers, who led the cyber forensics team that pawed through that tainted update on behalf of SolarWinds, providing details for the first time about what they found. The code was elegant and innovative, he said, and then added, “This was the craziest f***ing thing I’d ever seen.”

Dina Temple-Raston, NPR

The first foundation blocks of the hack started in late 2019 when the hackers inserted a seemingly simple line of code into the software that would indicate to them if the server used a 32-bit or 64-bit processor. Once the hackers were able to see a response to that simple query, they knew they could wreak some havoc. And five months later, they set down further foundational blocks by inserting code that would inform them whenever there was an impending software update.

Under normal circumstances, developers take the code out of the repository, make changes and then check it back in. Once they finish tinkering, they initiate something called the build process, which essentially translates the code a human can read to the code a computer does. At that point, the code is clean and tested. What the hackers did after that was the trick.

They would create a temporary update file with the malicious code inside while the SolarWinds code was compiling. The hackers’ malicious code told the machine to swap in their temporary file instead of the SolarWinds version. “I think a lot of people probably assume that it is the source code that’s been modified,” Meyers said, but instead the hackers used a kind of bait-and-switch.

But this, Meyers said, was interesting, too. The hackers understood that companies such as SolarWinds typically audit code before they start building an update, just to make sure everything is as it should be. So they made sure that the switch to the temporary file happened at the last possible second, when the updates went from source code (readable by people) to executable code (which the computer reads) to the software that goes out to customers.

Dina Temple-Raston, NPR

Similar to the situation that happened in the days and weeks leading up to the 9|11 attacks, there were small signals and flags that were picked up by random people in random places, however none of those pieces were put together to demonstrate that something nefarious was going on. It was only after a network administrator at FireEye discovered that there was a listing for two phones for a single employee that they realized that there was a hacker within their network.

Another reason why this hack was such a ‘work of art’ was that all the normal trackers look for ‘normal techniques’ which usually account for 90 to 95% of all attacks. This one was so unique and so stealth that it completely bypassed all normal checks.

Like other catastrophic failures of all shapes and sizes, this hack had its warning signs. There was just no one there looking at the big picture that could have put these pieces together to see what may be happening just beneath the surface. The NPR article is a really great read, and probably worth a second read, just to really grasp the level of sophistication some of these nefarious hacking organizations have.

Asleep At the Computer

From the NY Times on the recent hack of pretty much every security department system in the US Government:

Over the past few years, the United States government has spent tens of billions of dollars on cyberoffensive abilities, building a giant war room at Fort Meade, Md., for United States Cyber Command, while installing defensive sensors all around the country — a system named Einstein to give it an air of genius — to deter the nation’s enemies from picking its networks clean, again.

It now is clear that the broad Russian espionage attack on the United States government and private companies, underway since spring and detected by the private sector only a few weeks ago, ranks among the greatest intelligence failures of modern times.

Einstein missed it — because the Russian hackers brilliantly designed their attack to avoid setting it off. The National Security Agency and the Department of Homeland Security were looking elsewhere, understandably focused on protecting the 2020 election.

NY Times

Once again, while the US Government is playing checkers, our adversaries are playing chess when it comes to cyber-security. Let’s not lose sight of the fact that even as the US Government has spent billions to set up their ironically named “Einstein” cyber-security system, it wasn’t that system that detected the hack. It was a private company – the US Government vendor FireEye – that actually detected it and alerted US authorities.

Updated Hack: Send to Flickr via Picasa

I, like many folks, am a regular user of Google’s Picasa. I am also an avid fan of Yahoo’s Flickr. And being that both of the parent companies of these two fine services are locked in a steel-cage death match for worldwide Internet dominance, I am not going to hold my breath for a plug-in that will enable me to easily upload photos from Picasa to Flickr.

In searching for a straight forward work around to more easily upload photos from Picasa to Flickr when a) I just want to share and b) am not too concerned about photo quality, I found a python script via Lifehacker, which is way too much advanced coding for me to deal with, and a work around where I can email photos to Flickr via Gmail (emailing photos is a default option in Picasa), a seemingly far easier option. In reading the email option article, I went and tried out this option with great success (more details on Flickr’s email uploading functionality can be found here, here and here). It seemed far easier than dealing with Python code. I think it degrades the quality of the photo slightly, but for purely sharing purposes, it does the trick.

I also saw a couple of things that I’d like to humbly add on to this article:

  1. Brilliantly, Flickr has now enabled you to tag photos that you upload via email. Simply include the syntax in your subject line or body: tag: tag_1 tag_2 tag_n and automagically, your photos will be uploaded and tagged on Flickr. Slick, very slick.
  2. By default, the Picasa/Gmail email solution reduces the photo sizes to 480 px wide. To upload photos to Flickr as close to the original size of your digital photo, fire up Picasa, go to Tools > Options > Email and where it says “When sending more than one photo, resize to:” select 1024 px. You will still see a slight dip in photo quality when your photo is uploaded to Flickr, but I’m not going to lose any sleep over it. The photo won’t be the original size but you really don’t need massive 6 megapixel photos sitting up on an online photo sharing service anyway, unless you are a pro or semi-pro photographer.
  3. Finally, now that you can effectively use Gmail to upload photos to Flickr, a minor issue arises in that you can rapidly use up lots of disc space sending these big files via Gmail. Now, Gmail does have endless disc space, but for those that are find it important not to waste useful disc space, here is a little add on pointer: Set up a filter in Gmail, where all emails sent To your personal Flickr email are automatically Deleted. Simply go through the Gmail Filter wizard, putting your Flickr upload email into the “To” field, and then select “Delete It”. From there, every time you send a bunch of photos to Flickr via Gmail, the “sent” email is automatically routed to the Delete tag in Gmail. All the information contained within the email is brilliantly copied and integrated into your Flickr account, so there is no need to keep those emails. Next time you clear out your old emails, those memory hogging emails to Flickr are quickly and painlessly deleted as well.

So those are my few additions to the articles I found on hacking Picasa and Flickr with Gmail. It’s not the prettiest solution in the world, but for a quick and easy way to get photos from Picasa to Flickr, it’s pretty hard to beat.