Why I think keyword testing is not that big of a deal

Back around 2005 'keyword driven testing' use to be a big buzz phrase.  Although the idea hasn't died out yet, and still pretty popular, I have several reasons against it.  There are frameworks such as Robot Framework that still evangelize the practice as well as many popular off the self automation tools that promote this as a feature.

But as an SDET who has taken apart of rebuilt several frameworks for various companies I've worked for, I noticed that it changed how people approached testing and looked at testing, and not for the better.  While I do see some of the benefits of keyword driven testing, I also do see how they become crutches and promote negative thoughts about test design and implementation.  I'd like to take some time to highlight some of the arguments and counter points I see with keyword testing.

Not much gained

Below are some of the popular arguments for keyword-driven-testing from wikipedia, https://en.wikipedia.org/wiki/Keyword-driven_testing#Pros
I'd like to take time to point out some of the flaws in the arguments for keyword-driven-testing.

Test cases are concise and readable to stake holders

This can easily be achieved by properly architecting code.  Main reason why coded test cases are not precise is the lack of separation of high level and low level logic.  Simply separating your test code in to  3 separate packages: test behavior, flows, and page/screen objects.  You'll have achieved a consistently concise test cases without losing any benefits of working with code.  The idea is high level behavior is described in your top level test behavior package, the low level UI interactions are described in your page/screen objects, while multi-page flows where many page/screen objects are involved can be described as in the flows package.

The idea code has to look ugly and unreadable to business folks is ridiculous.  A properly layered approach, a test can look as follows.

public void testSearch() throws Exception {
 //Test that the search results contains our search text.
 GooglePage google = openGoogle()
 SearchResults results = google.search("what is keyword driven testing")
 verify(results.contains("what is keyword testing"))
}

And a badly written keyword test can look like.

NavigateTo	http://www.google.com
FillText	id=q	what is keyword testing
Click	name=btnk
GetPageText	$PageTextVariable
VerifyContains	$PageTextVariable	what is keyword testing

The only real difference between code and keyword testing is just how you name your functions and what logic level abstractions you have used.  Notice that although the code isn't as close to English, simply wrapping up calls into more readable functions and objects, you can make it read pretty well.  Also notice that keyword scripts can look very bad if you don't use proper discipline in creating higher level keywords to abstract away the lower level details.

Test cases easy to modify, new tests can reuse existing keywords.
This argument plays into the idea that 'testers' are not coders.  They can just simply rearrange keywords on a spread sheet to create new tests.  Like the above point illustrates, this isn't a by product of keyword testing, but rather a by product of levels of abstraction.  

Not dependent on a specific tool or programming language
This is one benefit that writing tests in code does not give you.  But in my opinion having actually done large ports of test cases across languages, the effort saved isn't that big when you are porting a properly architected projects.  Changing languages and technology is fairly common in the programming world, and there are many tools that help you do it.  Usually the problem isn't the language barriers, but mostly the lost knowledge of 'What does this code do?'.  This pretty common problem in porting keyword frameworks to other tools when the keywords themselves are not properly documented and lack the proper abstraction level.

Division of LaborTest case construction needs stronger domain expertise - lesser tool / programming skills
This plays into the idea that Programming is very hard to learn.  The fact is, the basic mastery of programming languages themselves is ridiculously easy to learn.  You're talking about people who are already capable of following technical instructions, creating written procedures, and dealing with different technologies such as SQL queries, setting up their test environment, running commands on a terminal or command prompt.  Most people can understand the basic building blocks to a program in just a few hours.  You can look at a code example, finding out what a certain line of code does, and repeat that pattern.

What really separates a 1st year computer science major from a 3rd year who's seen a software design course, is that their level of mastery of organizing their code, data structures, and making their code reliable.  These are all problems that do not magically go away because of a keyword based test structure.  You still have to properly design and architect your keyword structure to best represent a model of what you are testing and in a way that is easy to maintain.

One of the problems I have with the idea any 'tester' can build together tests using keywords like lego pieces without these bigger considerations is it can lead to people who shouldn't be writing automated tests in doing so.  If a test is badly organized, it will cost the team more in time to maintain the test in the long run.

What is lost with keyword testing?

One of the main weaknesses of using a keyword testing approach is you will tend to flatten out your data structures.  This in essence will reduce the level of abstraction and effectively mute the power of object oriented programming benefits. One of the biggest benefits of having objects instead of just pure single level function calls is the ability to group the data, related methods, and data structures into one representative object.  This is one of the first things that get lost in Keyword Driven Testing Approach.

For example, say I have an object called a 'InsuranceClaim'.  It contains setters, getters, multiple constructors, 1 for generating it with default data, 1 for generating given a file.  In keyword testing, you have to create 4 separate keywords that are not grouped at the object level, basically a Keyword for each operation which takes in the variable to store some sort of dictionary or map object reference, and to perform the action.  That in itself is not bad, but from a scaleability standpoint this is horrible.  Let's say you start getting variations of similar objects, then you have a situation it becoming difficult to figure out which keywords work with which data types.  In the object oriented programming world, since the operations are built into the data objects, this is all known at compile time.

Another detriment of keyword testing is you lose the ability to work with some of the advanced programming patterns that require object oriented approach.  For example, the Command Chain pattern that is commonly used in conjunction with Selenium Page objects.  Imagine expressing the following code in Keyword driven approach testing:

google.search("keyword testing").openResult(1).validatePageContains("keyword testing")

What makes this pattern very effective with PageObjects is each page action returns the PageObject of the expected Page, then using that you can call the methods of those PageObjects in a chain.  What makes this pattern very nice to work with is it makes you test very easy to refactor.  Anytime there is a flow change, this chain will break at compile time and you can easily identify and fix your tests.  With keyword testing, each becomes a separate keyword / parameter entry, which are not tightly coupled with the data type.  This makes compile time issues harder to spot.

Keyword testing also loses a lot of expressiveness when working with Multi-Threaded and Event Driven Tests.  Imagine how do you express something as simple as handling unexpected dialogs that only occur within a scope of a single page/screen.  In keyword testing, you will need to develop a keyword which then tells the framework to change it's error handling routine or global unexpected dialog handler.  In the object oriented world, you can spawn a separate thread or build into to your page/screen object, and have that object or thread tear it self down after the scope of it's need has passed.  Now imaging if you will having to keep creating exception cases and removing them during the course of the test.  Where this is easily managed in the object programming world with simple destructors, in the Keyword Testing world, this is a nightmare to deal with.

Dealing with page/screen variants is also another trivial thing that can be done in an object oriented way, which is difficult to do in Keyword Driven approach.  In an object oriented world, you can just create an interface and abstract factory for dealing with your Page level abstraction.

driver.get("http://www.myhost.com/search-variant1");
ISearchPage search = AdvancedPageFactory.constructPage(driver,ISearchPage.class);
ISearchResult results = search.search("Keyword Testing")
assertTrue(results.contains("Keyword Testing")

driver.get("http://www.myhost.com/search-variant2");
ISearchPage search = AdvancedPageFactory.constructPage(driver,ISearchPage.class);
ISearchResult results = search.search("Keyword Testing")
assertTrue(results.contains("Keyword Testing")

With a keyword testing approach, you'll end up dirtying that code using a bunch of if/else statements to handle all the different variations.  Handling things at an abstract level is what will allow you to make your tests more maintainable in the long run.  A general rule of thumb, if you have to use a nested if statement inside your test, that means you did not separate high/low level logic.  This occurs quite often with keyword tests.

Conclusion

IMO, there really isn't much gained by Keyword testing.  Keyword testing does force people to create reusable keywords, however because of Keyword testing's lack of abstraction and ability to apply advanced programming patterns, it becomes another low level language that lacks the power of an object oriented language.

I think it's best to take an object oriented approach, and approach test automation like any real programming project that needs to be maintained.