Using C # and UI Automation to Capture Content of an Unknown Control Type

Question

In the image below there is an area that has an unknown (user) class. This is not a grid or table.

I need to be able to:

• select rows in this area
• to get the value from each cell

The problem is that this is not a general type of element - I have no idea how to solve this problem or solve it myself. So far, the code is as follows:

Process[] proc = Process.GetProcessesByName("programname"); AutomationElement window = AutomationElement.FromHandle(proc [0].MainWindowHandle); PropertyCondition xEllist2 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomListClass", PropertyConditionFlags.IgnoreCase); AutomationElement targetElement = window.FindFirst(TreeScope.Children, xEllist2); 

I already tried to threaten this Region as a text field, as a grid, as a drop-down list, but so far I have not been able to solve my problem. Does anyone have any tips on how to capture data from this area and iterate over rows?

EDIT: Sorry, I made the wrong assumption. Actually, the header (column 1, column 2, column 3) and the "lower half" of this area are different types of controls !!

Thanks to Wininspector, I was able to get additional information about these types of controls:

• The header has the following properties: HeaderControl 0x056407DC (90441692) Atom: # 43288 0xFFFFFFFF (-1)
• and in the lower half: ListControl 0x056408A4 (90441892) Atom: # 43288 0x02A6FDA0 (44498336)

The code I showed earlier is just a List item, so this is an update:

 Process[] proc = Process.GetProcessesByName("programname"); AutomationElement window = AutomationElement.FromHandle(proc [0].MainWindowHandle); //getting the header PropertyCondition xEllist3 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomHeaderClass", PropertyConditionFlags.IgnoreCase); AutomationElement headerEl = XElAE.FindFirst(TreeScope.Children, xEllist3); //getting the list PropertyCondition xEllist2 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomListClass", PropertyConditionFlags.IgnoreCase); AutomationElement targetElement = window.FindFirst(TreeScope.Children, xEllist2); 

After further thought, I tried to get all the column names:

 AutomationElementCollection headerLines = headerEl.FindAll(TreeScope.Children, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.HeaderItem)); string headertest = headerLines[0].GetCurrentPropertyValue(AutomationElement.NameProperty) as string; textBox2.AppendText("Header 1: " + headertest + Environment.NewLine); 

Unfortunately, in debug mode, the count element in "headerLines" is 0, so the program throws an error.

Edit 2:. The answer to this question is below - I installed Unmanaged UI Automation, which has better features than the default UIA. http://uiacomwrapper.codeplex.com/ How do you use an outdated template to capture data from an unknown control type?

 if((bool)datagrid.GetCurrentPropertyValue(AutomationElementIdentifiers.IsLegacyIAccessiblePatternAvailableProperty)) { var pattern = ((LegacyIAccessiblePattern)datagrid.GetCurrentPattern(LegacyIAccessiblePattern.Pattern)); var state = pattern.Current.State; } 

Edit 3. IUIAutoamtion approach (not working at the moment)

  _automation = new CUIAutomation(); cacheRequest = _automation.CreateCacheRequest(); cacheRequest.AddPattern(UiaConstants.UIA_LegacyIAccessiblePatternId); cacheRequest.AddProperty(UiaConstants.UIA_LegacyIAccessibleNamePropertyId); cacheRequest.TreeFilter = _automation.ContentViewCondition; trueCondition = _automation.CreateTrueCondition(); Process[] ps = Process.GetProcessesByName("program"); IntPtr hwnd = ps[0].MainWindowHandle; IUIAutomationElement elementMailAppWindow = _automation.ElementFromHandle(hwnd); List<IntPtr> ls = new List<IntPtr>(); ls = GetChildWindows(hwnd); foreach (var child in ls) { IUIAutomationElement iuiae = _automation.ElementFromHandle(child); if (iuiae.CurrentClassName == "CustomListClass") { var outerArayOfStuff = iuiae.FindAllBuildCache(interop.UIAutomationCore.TreeScope.TreeScope_Children, trueCondition, cacheRequest.Clone()); var outerArayOfStuff2 = iuiae.FindAll(interop.UIAutomationCore.TreeScope.TreeScope_Children, trueCondition); var countOuter = outerArayOfStuff.Length; var countOuter2 = outerArayOfStuff2.Length; var uiAutomationElement = outerArayOfStuff.GetElement(0); // error var uiAutomationElement2 = outerArayOfStuff2.GetElement(0); // error //... //I've erased what followed next because the code isn't working even now.. } } 

The code was implemented thanks to this problem:

Reading cell elements from data grid in SysListView32 of another application using C #

As a result:

• countOuter and countOuter2 lengths = 0
• cannot select items (rows from a list)
• impossible to get ANY value
• nothing works.

Answer 1

You might want to use the basic user interface automation classes. This requires that you import the DLL in order to use it in C #. Add this to the pre-build event (or do it only once, etc.):

 "%PROGRAMFILES%\Microsoft SDKs\Windows\v7.0A\bin\tlbimp.exe" %windir%\system32\UIAutomationCore.dll /out:..\interop.UIAutomationCore.dll" 

Then you can use IUIAutomationLegacyAccessiblePattern.

Get the constants you need for calls:

C: \ Program Files \ Microsoft SDK \ Windows \ v7.1 \ Include \ UIAutomationClient.h

I can read Ultragistics Ultragrids this way.

If this is too painful, try using MSAA. I used this project as a starting point for MSAA before moving on to the entire core of the ISA: MSSA Code Example

----- Edited 6/25/12 ------

I would definitely say that finding the right “identifiers” is the most painful part of using MS UIAutomation material. What helped me a lot was to create a simple form application that I can use as a “location logger”. Essentially, all you need are two things:

• a way to keep focus even when you are out of the form focus placement window

• calling the ElementFromPoint () element using the x, y coordinates where the mouse is located. This is an implementation in the CUIAutomation class.

I use the CTRL button to tell my application to capture mouse coordinates (System.Windows.Forms.Cursor.Position). Then I get the element from the point and recursively get the parent element until I get to the desktop.

  var desktop = auto.GetRootElement(); var walker = GetRawTreeWalker(); while (true) { element = walker.GetParentElement(element); if (auto.CompareElements(desktop, element) == 1){ break;} } 

----- revision 6/26/12 -----

Once you can find automation identifiers and / or names recursively, you can easily change the code here: http://blog.functionalfun.net/2009/06/introduction-to-ui-automation-with.html , which will be used with Core UI Automation classes. This will allow you to create a line in a repeated procedure that can be used to identify a control embedded in an application with XPath style syntax.