Thoughts on what should be automated

It's bee traditional thinking to...

1. 1-to-1 parity between manual tests and automated tests.
2. Trying to have as much requirements and test case coverage as possible.
3. Automated tests should be able be reproducible manually.

But over the years, I've found that having 1000's of test cases I tend to cause even more problems.

1. Test code is harder to maintain that production code.

Besides the technical aspects like having to bend over backwards to accommodate changes in the software under test, it is also the lack of support from business and development for test code.  For example, very few shops will hold up a release or delay development if there are broken tests or flaky tests.  The default is generally to comment those tests out..

2. The need for quicker feedback.

What tends to happen with 1000's of test cases is many of the tests cases will get moved into some sort of nightly or weekend stage.  But instead of catching bugs the moment they are checked in, they are caught much later.  What ends up happening is you'll have to first investigate to make sure that issue is not a flaky test, then reproduce that step across different environments to make sure its not an existing bug.  By the time you gone through all the work, the developer probably have already moved on to the next task and story, and even worse, some other developer probably already pulled/rebased his code to include the bad code.  

3. When writing many tests, you may not have spend as much effort writing them for debug-ability

Tests are generally hard to debug.  Any number of things can cause them to fail, and from the test's point of view, an unexpected element or an expected element might be missing.  But the cause could be a slow render, or a misstep in an earlier setup, an upgrade window getting in the way of the test, a tree falling on the data center, etc...  

Solution? 

1. Write fewer tests
2. Focus on performing complete workflows over single actions.  Testing single actions in isolation will lead to number of tests creeping up.  Working workflows will keep it at a smoke test level.
3. Write code with the intention of debugging every single line.  Pay close attention to your logging and exception handling.  Make sure a failure at any point will return you a reason.
4. Use good programming patterns and test architecture to make it easier to refactor.