I have been spending a lot of time exploring automation testing frameworks that can be driven using NUnit. I am exploring the following:
- Roll Your Own
- NUnit Forms
- Ranorex
- Microsoft UI Automation
Microsoft's UI Automation Framework is NOT fit for command.
Don't get me wrong... there are a lot of easy-to-use systems in .NET. BUT, I find the Code DOM easier to use than the UI Automation tools. If you have ever used the Code DOM, you know that I am saying a lot with this.
Here are my issues with the UI Automation Framework:
- UI Spy Doesn't Ship with Visual Studio 2008. If you want UI Spy, you need to download the SDK for Windows Vista. Don't try to use this framework without it. You will fail.
- The UI Spy tool is buggy and crashy and slow. Did I mention that this tool is essential?
- As much as I research it, I can't figure out how to do the first two things I tried to do: Click on a LinkLabel and enter text into a RichTextBox. I know how to click on a button and enter text into a TextBox, but the more complicated versions of this are too difficult, aparently.
- The API is terribly difficult to write AND read. Not intuitive at all!
- Finding controls using the NameProperty doesn't use the "Name" property. It uses the "Text" property. If you want to search for a control using the "Name" property, you need to use AutomationIdProperty. Did I mention that the API is awful? This is why you need the UI Spy tool.
- The framework itself is slow
- You need a separate thread to catch modal dialogs. Have you ever tried to write tests that utilize multiple threads?
Sorry, Microsoft. This framework has failed.
5 comments:
I agree about the Microsoft UIA framework. The White framework (http://www.codeplex.com/white) does a good job of abstracting the UIA framework by providing a UI element-oriented framework (as opposed to MS's pattern-oriented one) built on top of UIA. White gives you classes of objects to work with, such as Buttons, TextBoxes, ComboBoxes, etc.
However, UIA doesn't live up to what I was hoping for. I wrote some NUnit tests that automated some of my regression tests. By using UIA, UISpy, and White, I came up with a nice set of automated tests, or so I thought I did. The tests work very well in Windows Vista (UIA successfully finds all of the UI elements I ask it to look for, etc.). However, when I run the same tests in WinXP, UIA fails to find some trivial UI elements (e.g., tool strip items and grid rows). Since I need to test our software in both Vista and XP, having to write around these OS-sensitive issues is not acceptable.
Maybe MS will roll out a UIA 2.0...
Yours was the first result as I looked for clicking a link label. After a bit more searching I found this page: http://msdn.microsoft.com/en-us/library/system.windows.forms.linklabel.system.windows.forms.ibuttoncontrol.performclick(VS.80).aspx
(LinkLabel as IButtonControl).PerformClick();
I've run in to this many times before, the framework seems very good allowing typing transitions, why they don't just inheret the same function I don't know.
itjstagame: I think you have misunderstood what I am talking about. Your solution is great if you have a reference to the actual LinkLabel.
I am talking about the UIA framework, which doesn't give references to the actual objects. It gives you an automation controller. From there, you can't do what you are referring to.
I believe the PerformClick is not available on the LinkLabel directly because the interface is implemented explicitly. Casting does the job.
I don't think this framework has failed. It is unfair to assume that UIA will just work on all 3rd party applications. I think it was designed to work by using compatible provider / client implementations. By that I mean a target application that implements UIA successfully by way of server provider where required (for custom wpf controls or any pre-wpf applications where accessibility is lacking) and a client that ensures at least .NET 3.0 framework exists in the machine where the automation is executed (Jeff, your "os specific" problem. It is not os specific, you are simply lacking the correct .NET framework. For any other functionality you simply need to ensure that you have implemented server / client provider to cater for the controls lacking in accessibility features you require. This will only need to be done once.
Shawn,
After re-reading my post, I feel a bit sheepish for saying that the framework has "failed". I regret that. It was unprofessional of me. I could have toned down my rhetoric quite a bit in that post and I apologize.
That being said, I still maintain that this framework is extremely difficult to use. I have used many automation frameworks and this ranks up there as one of the hardest to use.
In addition, as someone who writes a lot of tests for his code, I have a problem with being required to implement providers in my code to leverage a test harness. This is not a requirement for UI testing.
I have found that a roll-your-own approach to UI testing (for .NET) is much easier and more flexible than using UIA in its current form.
I will be very interested to see what advances UIA makes in .NET 4.0.
Post a Comment