Scripting into Non-Traditional Windows Applications – Citrix (Part 2)

Posted on behalf of Paul Actis, Sr. VP, Research & Development
In part one of our Scripting to Citrix series, we discussed some of the issues regarding reliability and performance in automating a remote hosted environment.
In part two, we will elaborate in more detail how that can be accomplished, the best practices, and troubleshooting when (as always happens) things do not go as expected.
The old way of connecting to a Citrix application involved trying to convert the image to text rendering.  This would often fail as the conversion simply could not interpret various font types, and did a poor job distinguishing numbers from letters.
A better way would be to look at how the user interacts with the application and make decisions based on what they see in front of them, not what some backend engine is trying to extrapolate.
In Summit Scripting Toolkit (SST) 8.0, the new Citrix stream has been updated to show the window being worked on, instead of its text interpretation.   The view below shows connecting into a Meditech dictionary screen.

A couple things to review here:

  1. The stream view updates any changes to the application with the latest snapshot
  2. You can select a subset of the screen and that allows for various options around timing commands, text reading, and mouse actions
  3. The highlighted portion shows the button in a disabled state. As a user I now know I will not be able to save the data on the screen until the button is enabled.  The same logic can be used to distinguish between selected / not selected items and warnings (red vs yellow shading).

The general concepts around automation remain the same, review the application, enter data, react to any changes.   Because all the rendering and processing for OCR is done internally performance has been greatly increased.
A big upgrade to previous ways of reading text is the ability to set Binary Threshold and Character Whitelist on the selected image.  These options greatly increase the accuracy of rendered text.   The screen below shows how these can be manipulated.

Here we can see the original image (on top) and when making changes to the Threshold that then changes the text output (bottom right).
Lastly, there will always be those instances where the script fails. This can be caused by taking an update, data validation or workflow changes.  In these situations, the ability to distinguish what caused the failure can greatly help in resolving it in a timely manner.
Below, the Pause for Rectangle command has failed (red line item). The Image Comparison form displays the Expected and Actual images.  In this example we can see that reason the script didn’t go forward is that it was waiting for all three buttons to be visible while the current workflow is only capturing part of the first button.   Lastly, the Run Time View shows the total screen with the image highlighted that it expects.

Workflow automation has its own predefined difficulties when scripting to a thick client.  Dealing with a Citrix or Remote Desktop environment brings additional demands.  You’re basically writing to a blank canvas without a safety net.
Fortunately, with SST 8.0, those demands have been addressed.  Happy scripting!