Microsoft/CrowdStrike IT Outage

By now the world is aware of the widespread impact of the Microsoft/Crowdstrike technology failure. While this has not been categorized as a malicious event, the operational impact may be covered by the dependent or contingent business interruption coverage found in your cyber policy. As you navigate the disruption to systems and business practices, Lockton has the following tips for aiding in recovery:

  • There may be Dependent or Contingent Business Interruption coverage available in a cyber policy

  • Keep track of all expenses, invoices and costs. Many policies cover normal operating expenses for the duration of the outage, as well as extra expenses incurred to mitigate the disruption.

  • Proof of loss can be challenging in these situations; delayed revenue does not always mean irrecoverable revenue. Most policies contain funds to engage a forensic accountant to assess the impact of this event.

  • Any vendors that are engaged usually need to be pre-approved or on a carrier’s panel. Coordinating the response with the Lockton Cyber Claims team will ensure that any conditions for coverage are met during the response process.

Details:

CrowdStrike is aware of reports of crashes on Windows hosts related to the Falcon Sensor. This is an operational issue with a software update with a driver. The update caused a failure on the windows computer.

CrowdStrike has confirmed the issue has been resolved with a software update. The issue was not a malicious cyberattack but stems from a defect found in a content update CrowdStrike pushed out to its customers that use Microsoft Windows. There is a manual recovery process for clients to implement.

Action Plan:

If hosts are still crashing and unable to stay online to receive the Channel File Changes, the following steps can be used to workaround the issue:

Workaround Steps for individual hosts:

  • Reboot the host to give it an opportunity to download the reverted channel file. If the host crashes again, then:

    • Boot Windows into Safe Mode or the Windows Recovery Environment

    • Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory o Locate the file matching “C-00000291*.sys”, and delete it.

    • Boot the host normally.

Note: Bitlocker-encrypted hosts may require a recovery key.

Workaround Steps for public cloud or similar environment including virtual:

Option 1:

  • Detach the operating system disk volume from the impacted virtual server

  • Create a snapshot or backup of the disk volume before proceeding further as a precaution against unintended changes

  • Attach/mount the volume to to a new virtual server

  • Navigate to the %WINDIR%\\System32\drivers\CrowdStrike directory

  • Locate the file matching “C-00000291*.sys”, and delete it.

  • Detach the volume from the new virtual server

  • Reattach the fixed volume to the impacted virtual server

Option 2:

  • Roll back to a snapshot before 0409 UTC.


Workaround Steps for Azure via serial

  • Login to Azure console --> Go to Virtual Machines --> Select the VM

  • Upper left on console --> Click : "Connect" --> Click --> Connect --> Click "More ways to Connect" --> Click : "Serial Console"

  • Step 3 : Once SAC has loaded, type in 'cmd' and press enter.

    • type in 'cmd' command

    • type in : ch -si 1

  • Press any key (space bar). Enter Administrator credentials

  • Type the following:

    • bcdedit /set {current} safeboot minimal

    • bcdedit /set {current} safeboot network

  • Restart VM

  • Optional: How to confirm the boot state? Run command:

    • wmic COMPUTERSYSTEM GET BootupState


CrowdStrike Support:

Find answers and contact Support with our Support Portal. Anyone accessing this page needs a working CrowdStrike support portal account.

Claims

If you believe your business operations have been impacted, please contact your Lockton Cyber & Technology Claims Advocates to discuss notification process with insurance carriers.

Cyber Vigilance

Cyber criminals are taking advantage of this situation and are standing up phishing sites impersonating Crowdstrike, Microsoft offering “fixes”. It is prudent to be vigilant of IT Helpdesk impersonation social engineering attempts at employees, employee impersonation attempts at company IT Helpdesk.

Organizations may follow strict ID validation process for calls into their IT Helpdesk, especially related to password reset and access requests. It may be prudent to remind employees to stay vigilant of social engineering attempts. Organizations should monitor and mitigate as needed.

Further Updates

If we receive further updates, we will advise accordingly. If you have any further questions, please reach out to Lockton’s Cyber & Technology Team.

For general inquiries regarding this matter, please contact cyber@lockton.com (opens a new window)