Load Testing Allscripts Enterprise EHR™
- A feeling of worry, nervousness, or unease, typically about an imminent event or something with an uncertain outcome.
- Desire to do something, typically accompanied by unease.
If you have ever worked on a project that has required a significant change to your Allscripts Enterprise EHR™ infrastructure then you certainly know the feeling. You’ve carefully planned your infrastructure, done extensive functional / unit testing and now comes the day of go-live. How will your new infrastructure handle a production load?
This question is far too common, but I have recently worked with a major health system that made significant infrastructure changes and used HP LoadRunner to simulate load given common resource intensive workflows like Note & Results Verification. Here are few lessons learned to share about this experience:
- Value – Licensing around load testing software is expensive and is normally based on the number of users in which you want to simulate load – the more load you want to put on your system the more it costs.
- Risk – Using HP LoadRunner requires a few minor configuration changes to every web servers where load will be tested. This will require a server admin to make changes to the web servers which will then need to be reverted after load testing is completed. Please use caution when scheduling your load testing to ensure that these changes can be reverted and a subsequent smoke test can be executed in a timely manner.
- Define your workflows: Simulation testing is 100% based on capturing and scripting workflows which can be very timely – in some cases it can take days to script out just one workflow in your load simulation software. In this client’s case, we used 4 common workflows including a Nurse Note for Urgent Care Visit, Chart download, Results Verification (POC orders) via worklist, and Charge.
- Technical challenge: Using HP LoadRunner we had to programmatically create/copy both test patients and test users. So if your maximum threshold for testing is 1000 users then you will need to create 1000 test users with unique logins and 1000 test patients.
- What is your organizations baseline? Load testing is based on concurrent users, but Allscripts has a variety of different user roles all which have varying demands on the system. In the aforementioned client’s case, it was decided that a clinician role would be used where common workflows could be utilized. Once you have your role selected you will need to create a tiered approach where you are starting with an average number of system users and slowly ratchet up the simulation testing against your capacity (and even beyond).
- Add a little chaos… Once you have captured your baseline numbers rerun the simulation just short of your capacity using real testers. Click around and measure the impact to your load testing given workflows that aren’t being accounted for in your load simulation. Compare these results with your baseline numbers.
John, I read this with great interest: I work for a competitor to HPs Load Runner load testing software and Allscripts happens to be one of our customers. Your concern about cost is certainly valid – but there are much less expensive options out there.
I’m surprised you needed to modify the web servers. As I mentioned, we’ve tested with Allscripts and never had to do this when testing with our software. Could you elaborate on how/why Load Runner required you to modify the web servers? We do have a monitoring component that can be (optionally) installed on servers, but it is designed to be installed and run with no discernible effect on a running server…we do this with our customers live systems on a regular basis.
Your difficulty in building testcases for that system is not unusual. It can be a challenging system to test. We’ve been doing it long enough we can usually develop a specific testcase in a day, which is still a lot longer then is typical for us.
Hi Chris. Thanks for the great feedback! I cannot comment specifically on the selection of HP LoadRunner in this particular case, but I am glad to hear that there might be other options out there. I have worked with Allscripts Enterprise EHR™ for several years and load simulation testing is always a tricky piece around upgrades, etc.
I would be happy to chat with you at some point to compare some notes.
Thanks for writing this article.
I have successfully tested Allscripts EMR for multiple clients since version 4.5 using Loadrunner. In our experience, the core functionality is generally implemented via Citrix or Remote Desktop publishing protocol. We recently finished up testing version 6.0 and the testing engagement paid for itself in MS SQL Server licensing savings alone (right sizing the database verses the default specs from Allscripts). The same best practices that apply to ALL software packages apply to Allscripts:
1. Have a test environment that is like production or test on the new production servers before it goes live. Be able to make changes and take down this environment without affecting current production servers.
2. Stage the data (user logins, patients and data) and set up the application to make it easier to automate the business workflows.
3. Allow at least 4 to 6 weeks to do performance testing prior to go-live and have “reaction time” built in to make changes based on findings. Don’t do performance testing while others are inthe same environment making changes and fixing functional defects. Giving performance the appropriate attention can really be a huge cost savings.
LoadRunner can be expensive, but there are alternative licensing models which allow for this cost to be minimized. It provides the largest set of protocol support (web, Citrix, Remote Desktop, and others) and built in monitoring of the servers.
Full Disclosure: I represent an HP software partner/reseller
What minor configurations changes were required on the web servers to allow the use of LoadRunner that you mention under your Risk?
Hi Daryl. This work was done on version 11.2.3, and we had to adjust the web.config file in C:\Program Files\Allscripts Healthcare Solutions\Touchworks Web\ ISAPI.
If you search the file for the word ‘binary’. It should bring you to a key labeled ‘SerializationFormat’. Change the value from binary to XML and you will need to cycle IIS.
There are other adjustments required on the LoadRunner side, but I omitted them from the original article as they posed no risk. I can send you those as well, if you need them.
Hope this helps.
If you could share the other changes that were required on the LoadRunner side I would appreciate that.
I am currently attempting to script against 11.4.1 but am having some issues staying ‘authenticated’ and am not sure if there is something that I am missing or if anyone else has come across some similar troubles.
Specifically it appears that the login is successful but when the GetAppGroupInfo is posted I get a 500 error “Access Denied. The request could not be authenticated”. 500 errors continue after that point.
I was not encountering this issue in versions 11.2.3 and 11.1.7.
Daryl. The following registry keys were adjusted on the LoadRunner server:
HKEY_Current_User\software\Allscripts Healthcare Solutions\ConfigurationSettings\SystemSetting.SerialFormat=XML
HKEY_Current_User\software\Allscripts Healthcare Solutions\ConfigurationSettings\SystemSetting.Compression=0
HKEY_Current_User\software\Allscripts Healthcare Solutions\ConfigurationSettings\SystemSetting.UploadCompression=0
Hope this helps.