Patching – The continuing challenge for SysAdmins
According to Wikipedia ‘Patch Tuesday’ started as an unofficial term to describe Microsoft’s regular updates generally landing on a Tuesday. It was formalised in October 2003 with updates landing on the second and sometimes fourth Tuesday of each month.
This means that for most IT environments, sysadmins have to evaluate if the updates will have any effect on their systems, and if not get the patches applied. Occasionally, there are changes to Dynamic Libraries (DLLs) which will affect other products. That could introduce some risk of uinintentional side-effects. It’s no surprise that risk-averse Admins, might choose to skip an update to reduce the chance of impacting the business. But this soon becomes an issue where organisations find themselves running older versions which will require significant effort to upgrade. Eventually, there will be a critical need to upgrade to address a serious security vulnerability or implement change requests, user training, internal API re-writes etc.
This is why sysadmins have to know what patch-level their systems are at, and scan in the new updates for possible clashes before actually implementing the updates. The whole process has to be performed against all servers and devices and means a lot of privileged account usage, a lot of passwords moving around, and that human vulnerabilities can be exploited.
Of course, Patch Tuesday has given rise to ‘Exploit Wednesday’. This is because attackers also get to see the patches on the Tuesday and the race is on to see if un-patched systems can be breached for either data exfiltration or ransomware.
Automating Patch Tuesday
At Osirium, we’ve been using automation for years with ‘Privileged Task Automation’. These days we use our own PPA (Privileged Process Automation) product to scan our Windows Server estate. It’s fast and delivers comprehensive reports and means we can patch faster, and be done before Wednesday.
Of course the time-savings are dependent on the size of your Windows estate, but in our case it amounts to several hours every ‘Patch Tuesday’. It takes less than 30 seconds to set up a run, and the run completes in around 5 minutes.
For Osirium, our PAM (Privileged Access Management) product holds all the connection details and credentials for our Windows estate. PPA uses a secure API to bring these details into the sandboxed container of the PPA Appliance. This means that every time this task runs no credentials ever go through the user, or even enter the user’s workstation.
The generated emails are a great asset for audits. We can be asked the patch level of any machine at any given date and have the answer in less than a minute!
We also have similar task for our Linux estate, and as soon as Matt has some spare time we’ll get that demonstration to you. Of course, as always if you’d like to know more – please get in touch.