Through what can only be called a comedy of errors, “we” (not me) managed to brick nearly 1,000 laptops at work. The details aren’t particularly important (nor am I in a position to discuss them), but let’s just say that a unique combination of outdated operating systems, older third party disk encryption solutions, and Microsoft’s latest patches combined to create the perfect storm for these specific machines. All of these things came together in a way that left the machines in a non-bootable state, effectively turning them into “bricks.”

A quick response was needed, so management requested that those machines be mailed to Oklahoma City where an impromptu team of approximately thirty computer specialists gathered to remediate the issue. Roughly one third of the team members like myself live here in Oklahoma; the rest have come from Texas to Washington DC to lend a hand. Over the past several days I’ve had the opportunity to catch up with some old friends and make some new ones.

Because of the large scope and quick turnaround of this effort, many kinds of volunteers were needed. We have people receiving and inventorying the machines. We have people dealing with obtaining disk decryption keys. We have people physically swapping hard drives. We have people reimaging machines, performing data recovery, and making sure all the required software has been reinstalled before the laptops are repackaged, tracked, and shipped out. My contribution to the project has largely been in automating some of the processes. I wrote the scripts that automate the backup process, updates the BIOS, and performs other tasks — and when we found that PowerShell couldn’t operate in the environment, I reverted back to ol’ DOS batch files (yes, in 2020). If I had more time I could have written more graceful code, but this is a quick-and-dirty operation, and I wrote some quick-and-dirty scripts to move things along. There have been hiccups along the way, and the scripts are still being tweaked as I get feedback from the technicians about more parts of the process that can be automated.

Beginning last Wednesday, I’ve been working twelve hours a day (as have most of the volunteers). Wednesday morning we had volunteers but no tools, no new hard drives, no USB sticks (needed for imaging) and, most important of all, no donor machines. Looking at a glass half full, that gave us time to create a skeleton process (which to be sure has changed several times as bottlenecks were identified). By Friday, machines were moving through the process as smoothly as could be expected.

Today, Saturday, we’ll be working from nine a.m. until we run out of machines. If we can get the stockpile down to zero today, we’ll get Sunday off and begin again on Monday. Sometime next week, the first wave of traveling volunteers will return to their duty locations and a new wave will arrive. Hopefully there will be enough crossover time for the old group to pass along the process to the incomers. The official ETA for completion is “when all the machines are done,” so we’re hoping within another week or two the mass wave of dead machines will have been processed, and any future machines can be handled by one of our normal imaging teams. Until then, it’s all hands on deck. I’m tired and sore, but also excited and determined.