This blog aims to explain what hotfixes are and considerations when applying them.
What are hotfixes
A hotfix is a small patch to update one or more objects outside the normal patch release mechanism. Hotfixes are provided to resolve a specific problem, updating just the necessary objects to provide the solution. When hotfixes may be provided is indicated in the "Sage X3 UKI Business Partner Handbook").
Be aware that hotfix patches such as described in this article do not go through the same level of QA and test procedures as a full scheduled patch updates would. You therefore need to satisfy yourselves from your own testing there are no unexpected effects introduced by applying a hotfix . Also note that code changes in a hotfix will automatically be included in the next scheduled patch set. I firmly recommend implementing the appropriate scheduled patch set as soon as feasible, as this will provide a high degree of testing as part of its QA process.
Hotfixes or patches provided by other organisations are outside the scope of this article but may still be required for Sage X3 to function correctly. e.g. SQL Server, JDK, Elastic Search.
There are two separate types of hotfixes provided by Sage, either or both which may be required to resolve a particular reported issue:
- Technology Hotfix
This applies to the Sage supplied technology components. i.e. Syracuse, Runtime, AdxAdmin, MongoDB, Print Server, Console, X3 Services, VTWebServer. Web Scheduling patches works slightly differently, so will be outside the scope of this article.
Although called a “hotfix” Technology component hotfixes are complete component installations, so are applied by running the installer which overwrites the existing version.
The file format is “product-version.jar” where product is the technology component, such as “syracuse_server” and version is the version of that component, such as “12.19.12.2” e.g. “syracuse-server-12.19.12.2.jar”
- Application hotfix
Application hotfixes are the X3 objects themselves that require updating, such as 4GL code (adx files in TRT directory), Crystal reports, or metadata such as table definitions, screen definitions, etc.
These are provided as one individual file for each bug fix, which may contain multiple X3 objects to be updated.
The hotfix filename convention is the form: WX_123456_R090_034.dat where “WX” designates Sage X3 product, “123456” is the hotfix number, “R090” confirms the patch is for Version 12 and “034” designates the X3 patch level the file is designed for. You should only apply “WX” hotfixes which match your X3 version and patch level.
To see what has been included in a hotfix patch you can use Patch Finder, although this only applied for fixes up to the last release version. If the hotfix is not included in patch finder data, you can review the patch file with a text editor such as Notepad if you need to do so, as the file is an ASCII file apart from when there is an ADX file included which is binary data within the file.
Before applying hotfix
Planning and testing
Both technology and application hotfixes impact ALL folders of your X3 instance. You certainly need to plan on doing testing before applying any hotfix, which should be on a separate test instance. If you do not have a specific test instance this will introduce considerable business risk.
Your planning should include the steps:
- Confirm the hotfix you are applying does not have any additional dependencies that also need to be applied, this mainly concerns the Technology Components. You can check this in the "Sage X3 Latest Patches" article.
- What testing activity needs to be undertaken by the patch installers, power users and/or general user population. You should test ALL critical business functions to ensure they perform well, even when a hotfix only seems to impact a small area. You can consider using Automated Test Platform to automate this testing process.
- Run these test steps before the hotfix is applied and document the results to confirm what the expected results are for your re-test after the hotfix has been applied.
- If you have external interfaces, how can these be tested.
- What downtime is required to install and test the hotfix and does this fit into the available downtime for when LIVE instance is patched.
- What exact steps are needed to apply the hotfix, e.g. what services need to be shut down before patching.
- If the patching fails, how can it be rolled back/abandoned.
Backups before patching
The recommendation is to have a full cold backup of your entire X3 instance where possible. “Hot” snapshot backups in a virtual environment could be used, but if you need to restore using such backups you will lose in-flight transactions.
Downtime whilst patching
The recommendation is to prevent all user activity whilst any kind of patching is undertaken, as you don’t want to be updating things that are being used! This includes shutting down Batch Server and SOAP pools, stopping external interfaces, as well as shutting down any appropriate technology components service. In multi-server instances, only MongoDB clusters provides a mechanism to do rolling upgrades, but as you need to shutdown Syracuse anyway this doesn’t really help.
Technology hotfixes
Syracuse - shutdown Syracuse service before applying hotfix.
MongoDB – Shutdown Syracuse first, then MongoDB service.
X3 services – Stop the X3 services service and uninstall the service. Described in https://online-help.sageerpx3.com/erp/12/staticpost/sage-x3-services-installation/
Runtime and AdxAdmin – with V12 these two components have a hard dependency so should both always be applied together to keep them in sync. Shutdown Runtime, AdxAdmin and Print Server services. Needs to be validated in the X3 console after applying the hotfix.
VTWebServer – Shutdown VTWebServer Apache and Java services. Needs to be validated in the X3 console.
Print Server – Shutdown Print Server service. Needs to be validated in the X3 console.
X3 Console – Make sure X3 console is not running. There is no need to take all users out of the system if this is the only component you are patching.
Application hotfixes
Get all users out of the system, batch server and SOAP pools shutdown and any other external interfaces disabled. You could consider temporarily changing the Syracuse port number in use to guarantee no users will login whilst a patching process is underway.
How to apply hotfixes
Technology hotfixes
The process is basically the same for the initial install, upgrades and “hotfixes”. Launch the component jar file installer and complete the information requested in the subsequent screens. You just need to make sure you select the “Modify installation” option rather than “New installation” when prompted.
Application hotfixes
You can either copy your hotfix file to the X3 runtime server (normally to folder “..\folders\X3\PATCH”), or can keep the hotfix file on the client. The advantage of putting hotfix files on the server is that you can apply multiple files in one run, if you have multiple hotfix files to apply.
Once you are ready to apply the hotfix file(s) you take the following steps:
- Login to X3 as an administration user. (Your user needs access to all folders)
- Connect to the X3 folder.
- (Optional) Navigate to Development, Utilities, Patches, Patch test (PATCHT)
(This step is optional) Allows you to see the impact of the patch you are about to apply.
If using “client” destination type, you will need to pick the hotfix file, or “server” you will pick the server folder/filename.
Leave all folders in the grid, you should always apply all patches to all folders.
Click “OK” to execute.
Check the resulting log file for any messages and resolve any issues reported.
4. Navigate to Development, Utilities, Patches, Patch Integration (PATCH)
If using “client” destination type, you will need to pick the hotfix file, or “server” you will pick the server folder to apply all contained patches or can select one filename.
- Patch integration = Must be checked to update, otherwise is run as a simulation.
- Descriptions in English = When checked always generate log file in English language.
- Deferred validation = If checked this will speed up patch application by skipping any Window or Screen validation, if the patch includes these objects. In this case, the validation is run when the Window or Screen is first accessed, causing a delay to the user for a few seconds.
- Link synchronisation = Syncronise the links between Classes and Representations. Normally leave checked as default. Refer KB article "Patch Integration: What is the purpose of field Link synchronization (field ALNKSYNC) ?"
- Synchronise windows = If checked, any Windows or Screens in any of the selected folders that need to be validated will be validated as part of the patch (i.e. not just the objects in the patch). This can potentially take some considerable time.
Leave all folders in the grid. This will apply the patch files to the X3 folder and cascade appropriate objects down the child folders. This is the only supported procedure, as you should always apply all patches to all folders.
Click OK to proceed.
When prompted “Have you read the ADXPATCH.htm file” select “Yes”
The patch may take some time to complete.
Once the patch has finished, review the generated log file and check for any errors or messages that indicate action needs to be taken.
You can also review the raw log file in the “..\folders\X3\TRA” directory if you have access.
What if I need to apply the same hotfix patch again?
If there is some error that you have fixed and need to apply the patch again, you can do so by running the patch as described above. You may get an error saying the patch is already applied, but can close the log file and continue anyway.
After applying hotfix
You need to complete your testing to ensure you get back the expected results before allowing users onto the system.
You can also review the applied patches in Development, Utilities, Patches, Patch Inquiry (GESAPT). The patches should be the same across all folders in the instance.
Reverting a technology hotfix
If you have an issue with the new version and need to revert to the previous version, the recommended/preferred option is to restore to backup, which provides the only guarantee to be back where you were.
In some cases, if you have gone past the restore window you can consider to re-install the previous version which will overwrite the newer version, although there is risk in this approach and may need some messing about with a failed install. I would certainly recommend speaking to your local Sage X3 Support team for advice before attempting to revert in this situation. If you have allowed users to enter data in between times, it increases the risk there may be some data corruption or unexpected behaviours even after reverting.
Syracuse – install your previous version over the top of the new version.
MongoDB – use the MongoDB procedure ( https://www.mongodb.com/docs/v4.4/release-notes/4.4-downgrade/ ) to revert if needed.
X3 Services – same process as upgrading. Remove the service, install the version you need. However, you must take account of the Syracuse/X3 patch level dependencies.
Classic components (Runtime, AdxAdmin, Print Server, VTWebServer) will require you to unpublish, unconfigure and then uninstall – followed by doing a fresh install of the original version. There is significant risk and time needed to complete this sequence, so restoring is generally going to be the best approach rather than doing these steps.
Reverting an Application hotfix
The only supported mechanism to revert an application patch is to restore from your backups. This is why testing before releasing to users is so important.
For smaller hotfixes it may be technically feasible to put back an individual file manually, but is not recommended and you may have data corruption to resolve and reverting in this way doesn’t update the metadata stored in X3 relating to patches applied. This is certainly not something to be undertaken lightly, and again would certainly recommend speaking to your local Sage X3 Support team for advice before attempting to revert in this situation.
Additional Reading
Understanding and troubleshooting Sage X3 "Updates" patching mechanism
Index page: Sage X3 Technical Support Tips and Tricks (March 2021) Overview of patching X3 and supporting technologies
Conclusion
I hope this article has enlightened you on what to do with hotfixes and why. Either way, feel free to provide your feedback in the comments below.
