Software Patching Best Practices – 18 Must Do Tips

Why Is Software Patching So Important?…Failure to Patch Can Be Very Costly

Failure to patch vulnerabilities in computer code can lead to losses of information that can cost more than your company can afford to pay. A recent example of  software patching failure came from a small non-profit in Anchorage Alaska. The small, five-facility nonprofit is paying federal regulators $150,000 in fines for allowing their system to be breached by not patching their software. That penalty is on top of the costs that they will additionally incur in now patching the systems they failed to patch in the first place – the legal fees for defense, the costs associated with getting credit monitoring services for those affected by the breach, the ongoing compliance monitoring by the Office of Civil Rights, and all other costs incurred by this very public debacle. Click HERE to see the Resolution Agreement signed by the non-profit. You don’t want to be forced to sign one of these, too.

There are many other benefits to applying software patches, including adding features, and fixing bugs that make the software run slow or not work right. All software needs to be patched. Whether the software sits on a disk and runs on a server, resides on a chip within a firewall, or is an app that is in your tablet devices, it all needs to periodically be updated and patched in order to be secure.

The following list of 18 software patching best practices is what we follow at Alvaka Networks when delivering on our Patchworx Patch Management service. It is important to note that all these steps are important; however, they are not always all utilized, or they can be utilized in different ways depending upon the needs of the client. Like us, you will need to decide what your patch management plan needs to look like to best suit your needs.

18 recommended best practices for patching your software:

1. First, identify all the software you are using. Today’s IT systems present a challenge because most systems run dozens of different software titles. You can’t know what you need to patch until you know what you have. You have operating systems, server applications and desktop applications.

2. You need to patch servers, PCs and mobile devices.

3. Remember, you need to have a strategy to patch many of your hardware based appliances, like firewalls, routers, SANs and more.

4. Set a regularly scheduled routine every month to patch your systems. You can do it most efficiently all in one big event over a weekend, where all systems are patched. Or, you can elect to do 20% of them at a time over the course of the month, to mitigate impacts from unexpected patching problems. There are many other approaches you can take. You need to decide what is best for your business.

5. When dealing with multiple servers, you need to identify if you have dependencies that require a certain server reboot in order for everything to work right on restart. For example, it is best practice to bring down a multi-tier system by starting with the presentation tier (web server), then the application tier, and then lastly, the database tier. Your systems should be brought up in reverse order.

6. Read the release notes or ReadMe files to learn more about the implications of deploying a set of patches. You should also review software user forums to see if anyone else is reporting problems with the new patches.

7. It is good to apply patches in a timely manner, but unless there is an imminent threat, don’t rush to deploy the patches until there is an opportunity to see what effect it is having elsewhere in similar software user communities. A good rule of thumb is to apply patches 30 days from their release.

8. Before applying patches to your production system, you should test the patches out on a test environment. This can be difficult and expensive for most companies, since it requires buying a lot of extra hardware and software to build the test environment. If you don’t have a budget to do this, there are alternatives. There are companies that provide patching services. Testing those patches before deployment to production systems is one of the tasks they perform.

9. During the testing process in the test environment, determine if the computers require a reboot or if they do so automatically. If so, then you need to plan for a maintenance window in which to apply the patches to the production system, so you don’t have unexpected system reboots that hurt your business operations or do damage to databases, etc. You can expect 90+ percent of your patch deployments to require reboots.

10. After you have applied patches, utilize a smoke testing procedure to make sure all applications and services are back online and running properly when servers and PCs restart.

11. Change Management is important, but often overlooked. You need to involve other stakeholders in the organization before making the changes. Often they will let you know of system or organizational demands that will have an effect on your patch deployment task. Read our blog, “What is Change Management and Why is it Important?”

12. Notify your end-user community of your planned time frame for patch deployment, so they know what to expect. When patching workstations, remind the users just prior to patching to save all documents, close all applications, and logout of their workstation, but DO NOT SHUT THE PC DOWN. Let them know what they should do if they encounter a problem after the patch deployment.

13. Have a good roll-back plan. A roll-back plan allows you to quickly reverse the patches and go back to the pre-patched system if there is a significant problem with the deployment. Good patching tools and procedures will allow for a roll-back of patches.

14. Have a good backup of all your systems and, if possible, take an image snapshot of your servers right before your patch deployment.

15. Are there any auto-scheduled maintenance jobs running to do maintenance, such as for a SQL database? If yes, be sure to put them on hold, as they can really mess things up if left running.

16. Use a service or automated tools whenever possible. Don’t use tools like Auto-Update, as you cannot control when patches are applied, you cannot test applications before patches are applied, there is no smoke testing procedure post-patching to determine everything is running fine, and there is no patch deployment reporting that is required to show yourself, management, auditors and regulators that you are running a securely patched operation.

17. Review the patching report after deployment and look for patches that failed to deploy. Investigate why they failed to deploy, develop your remediation plan, and then redeploy.

18. Make sure you accommodate your exceptions. Sometimes certain servers or applications cannot be upgraded or patched in order to maintain compatibility with a critical application that is in use. When this happens, you need to make sure you have an alternative strategy for securing that system from the vulnerability left exposed by the inability to patch the software.

A recent Center for Strategic and International Studies (CSIS) study made a strong case for patch management. The study, conducted over a three-month period, found that simply applying the most recent patches to six software packages on Windows machines could prevent 99.8 percent of malware infections!

Click HERE to see the HHS Office of Civil Rights partial list of companies paying heavy fines; some are in the millions of dollars, for breaches of their IT systems. These companies were already victimized once by the breach. I bet they felt victimized a second time when the fines were levied. Don’t let that happen to you. The moral of this blog is that you have to patch eventually one way or another, so make sure you do it in a timely and professional manner to avoid terrible impact on your business. It will bring you much peace of mind.

Calculate Your Costs to Patch Using Our Simple Monthly Software Patching Cost Calculator!