How to kill UI regression testing: git source control


This is third part in series how to kill UI regression testing. It is about git source control system and its web application part, github, bitbucket or gitlab. I have already wrote about this topic in Product moving parts as a source for test strategy, and Micro tracking changes for regression test. In this post, I will extend my thoughts from those two posts.

So git helps you to determine which part of application feature could be affected by the code change. Precondition is that you need to be able to read the code. Just read and understand functionality in that code. Many software testers started in software testing because their are afraid of code. I think that to be able to read the code could be great booster for software tester career.  And by reading pull request, you have very valuable mentor, the author of code himself.

Soon, you will find different code change patterns. For example, change in function signature in helper module could break more features than change in one module method. You can learn about that very quickly, the most important is that you should stop be afraid of reading the code.

And basically, you are doing code review. You will have questions and by asking them you could find issues before code is deployed and put in action.

Next post will be about Jenkins as an example of continuous integration tool.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to kill UI regression testing: development framework tests


This is second post about idea how to kill UI regression testing. In first post I described symptoms for UI regression testing that I would like to kill. I will describe development framework tests that are precondition for killing UI regression testing.

Development framework tests (DEFRATE) are automated tests that are written by developers. Here are reasons why I think that should be the case for every development project.

You have a developer that is a “rock star” in some development framework, lets take for example Ruby on Rails (you can replace that with development framework on your project). Lets see what Ruby on Rails framework offers in testing front.

Here is official documentation on testing Ruby on Rails application. Here is additional ruby testing toolbox. Every software developer that claims Ruby on Rails rockstar title, should be master in any of those ruby testing frameworks.

But which tests should developer create?

Developer test should create tests in DEFRATE that CHECK their code design.

Developer should check all flows that he created in its mind. Those code flows must PAST through the RUNTIME (lets say processor instructions). Tests should follow DEFRATE practice and use all features. For example, developer wrote some code that communicates with database. Essential part of test should be code that mocks database. Good DEFRATE enables easy mock code creation. And when somebody changes code that is mocked in a test, mocked code will fail and detect REGRESSION in code. Lowest and most quickest way to detect regression change.

Side effect of DEFRATE tests are tools that enables you to have code coverage by running those tests. Code coverage information is excellent source of information for you test strategy on UI level. And remember, code line coverage is just one testing coverage out of 101 testing coverage metrics. Here is excellent resource about testing coverage [Kaner: Software Negligence and Testing Coverage]

Google testing blog is also excellent resource for DEVELOPERS how to write good DEFRATE tests.

And the best way to learn testing framework is to read a book about it and do ALL the exercises listed in a book.

But my developers do not write DEFARTE tests! So I must stick with my UI regression testing. Ok, but now you know metrics that enables you to provide information about software quality in your project. And you have directions how to make quality in your project much better.

In next post, I will discuss source control tools that can help you to kill UI regression testing.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to check font size on a web page?


In this short blog post I will share with you a tool that could help you in checking font size on a web page.

I had to check application change that involved changing font sizes on application web page. In BBST foundations course I learned how to create test strategy. My test strategy was that I need to find a tool that will help me to do that check.

I searched Chrome extensions, and found WhatFont extension. My heuristic to quickly evaluate how good is that tool:

  1. simple and intuitive user interface
  2. more than 1000 ratings
  3. rating average > 3.5 starts

If those three are satisfied, chrome extension is a good tool. For WhatFont those three criteria are true.

After you enable extension, you just click on a font and you will get all needed information about it.


Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to kill UI regression testing: the simptoms


This is first post in what should be series of post. Idea is: How to kill UI regression testing. Idea started at CAST 2016 lighting talk and I extended it more at Testival 2016 open session slot where I get excellent feedback from the audience. The goal is to write material that will try to explain this idea in constructive, thoutfll way with experiments feedback that will suport or fail it.

First step, lets detect the symptoms, or reason why you are using UI regression testing.

You have a test plan that only contains a huge list of test cases. Every test case has useful name, and not so useful steps description. Context is a document with a hundred of pages. You are proud, because you wrote that document, and you are the only person that read it.

You have a automated test suite of selenium web driver tests, that have more that a hundred automated test cases, and in order to run it, you need several hours and several Jenkins instances. You have a number of failed tests, and those fails are false positives (test failed but this does not indicate problem in your product).

If this is the case, that means that developers DO NOT WRITE TESTS in their development framework (e.g. Angular, Ruby on Rails, Django,…). Note here that I do not mention unit tests, but DEVELOPMENT FRAMEWORK TESTS (DEFRATE).

There are many DEFRATE broken developers. They know only part of their framework, an usually they are not interested in testing part.

And because of that, you are developers safety net, and you are not doing software testing.

Because software testing is:


Every physician has first to determine the symptoms,  in order to determine disease. I defined the symptoms for  disease called UI regression testing.



Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Report on Testival 2016


This post is my report on Testival 2016 that was held last weekend in Rijeka STEPRI center.


First, a few metrics.

We had in total 32 participants, with female/male ratio 12/20!

As this is free event, dropout count is important metrics for us. 15 registered testers did not show, and did not let us know that they will not show. Which means that we wasted a lot of food.

Testival was first time international event:

Serbia: 4

Hungary: 1

Austria: 2

USA: 1

Liberland: 1

We are very excited about that metric!

Here are links to Facebook photo albums (you need to be log in in Facebook): Irja, Zeljko, Zeljko, Zeljko and me.

Here is my observation. Testers were not shy, they jumped in open session proposals without fear and hesitation:


My impression is that several testers problems spawned through all themes.

I got a chance to further model my thinking around idea: “How to kill regression user interface automated testing”, and got excellent feedback and deeper understating of the problem. New material for blog post.

For me, new thing was Splunk product.

And this event introduced for the first time context driven testing school by Ben Simo, that has years of experience of applying context driven principles in his daily work.

Mirjana introduced exploratory testing, and how to transition in daily work from test case step descriptions towards exploratory testing.

My opinion what could be done better for next Testival:

Get proactive help in organising process.

Switch Testival to be Thursday/Friday event.

Add workshops on Thursday day.

More coffee.

Add additional sponsors.

In order to fight against food waste, symbolic price for registration for which value would be given back in form of software testing book or something similar.

Here is audience feedback:



Again, without our sponsors, this event will not be possible:


Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to get full stack trace during gunicorn exception investigation


This blog post will explain how to get full exception stack trace during the gunicorn start up issue. gunicorn is python native application server for running django application.

So you changed something in your django configuration, and gunicorn failed to start. In my case, I did not get full exception stack trace, which means I did not know what was the root cause for this issue.

Traceback (most recent call last):

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 196, in run

web_1 |     self.sleep()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 346, in sleep

web_1 |     ready =[self.PIPE[0]], [], [], 1.0)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 231, in handle_chld

web_1 |     self.reap_workers()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 506, in reap_workers

web_1 |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)

web_1 | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

web_1 |

web_1 | During handling of the above exception, another exception occurred:

web_1 |

web_1 | Traceback (most recent call last):

web_1 |   File "/usr/local/bin/gunicorn", line 11, in <module>

web_1 |     sys.exit(run())

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/", line 74, in run

web_1 |     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/", line 192, in run

web_1 |     super(Application, self).run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/", line 72, in run

web_1 |     Arbiter(self).run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 218, in run

web_1 |     self.halt(reason=inst.reason, exit_status=inst.exit_status)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 331, in halt

web_1 |     self.stop()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 381, in stop

web_1 |     time.sleep(0.1)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 231, in handle_chld

web_1 |     self.reap_workers()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/", line 506, in reap_workers

web_1 |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)

web_1 | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
 Google led me to this usefull blog post.
gunicorn slweb.wsgi:application --preload
 And official gunicorn documentation explains –preload option:
Load application code before the worker processes are forked.
Because first exception describes worker spawn failure, loading application before worker spawn may trigger more detailed exception.
And that was true:
django.core.exceptions.ImproperlyConfigured: 'oracle' isn't an available database backend.
Try using 'django.db.backends.XXX', where XXX is one of:

    'base', 'mysql', 'oracle', 'postgresql_psycopg2', 'sqlite3'
 The root cause was wrong name for oracle driver.
Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Android back arrow icon UX issue

In this post I will present a UX issue from a very famous engineer cultured company.

Every user interface should be developed in such manner that it is self descriptive. It should be self documented, so menu items or input fields should have labels that clearly describe their purpose. Tooltips must not be some fancy technology, but they should contains small pieces of business logic that describe how is this part of form connected with business logic.

Icons should also have that purpose. If I want to save something, please put floppy disk icon (thank you Microsoft for that mnemonic).

And then we have Android back icon. What is first mental image in your head when you see that icon? What is the purpose of that icon?

For me: I will first go right in menu, and when I get there, Android will AUTOMATICALLY make a U turn and then I will go to the left in application menu.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Feedback on my objection to gherkin language


In this post I will explain how not to engage in twitter war, but with simple politeness, provoke useful feedback about important topic.

Twitter war is very serious thing. Neil Studd talked about this problem at CAST 2016 lightning talk sessions. During his talk, simple idea popped up in my head. It was time for the experiment.

In my post I constructively object to Gherkin language I expressed my opinion on Gherkin language (which is first part on objection on using UI automation as regression purpose thing).

I always tweet my blog post, and this time I mentioned mr.  George Dinwiddie, because he asked my opinion, also via Twitter.

Here was his first reaction:


He used word strawman, and as I did not know that word, i looked it up:

A straw man is a common form of argument and is an informal fallacy based on giving the impression of refuting an opponent’s argument, while actually refuting an argument that was not advanced by that opponent.[1][Wikipedia]

So I learned new english word, and he objected that I am using gherkin for wrong purpose.

Here was my response:


So I did not engage in twitter war, I politely asked for a blog post. And we have thoughtful response about gherkin language: Using a good tool for the wrong thing. And that was experiment about idea during the Neil lighting talk.

And Ben Simo also responded:


So now we know true purpose of Gherkin language.

And in my original post I forgot to mention how I get information from business people, what product should do so I can create my testing mission. I get them at the beginning of github issues, as several issue comments, written in plain english language, with references to google docs and pictures about the design of the feature.

Using that feedback, here is my follow up on gherkin objection.

We have product examples that could be automate and run so we have immediate evidence that product actually behaves  as this example documents. Which is reasonable purpose.

The problem is:

  • when only those examples are used in testing the product.
  • they are used for regression testing.
  • they are used for UI regression testing (end to end testing) as only automated tests on project

Here is what I suggest:

  • do not stop testing using “manual” (highly intellectual activity)
  • create test mission “using tools at our disposal, lets find out what exactly changed in product from previous release” and do “manual” testing in that part of product. There is no need for automated regression testing on UI level.

As I find this matter highly important for testing community altogether, I would really appreciate your feedback on this matter.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

I constructevly object to Gherkin language


In this post, I give constructive example why using Gherkin as communication tool  is not productive. At CAST 2016, I gave 5 minute lightning talk “I constructively object to UI automation. This first part is about using Gherkin language as communication tool.

I sad in lighting talk (thanks Neil):

Screen Shot 2016-08-31 at 10.32.22 PM

That statement went to Twitter, and I got reasonable question:

Screen Shot 2016-08-31 at 10.32.34 PM

I consider “pretty sloppy” remark as “highly creative”.

I am giving example why I think that Gherkin language is bad for communication. And I welcome you to prove me wrong.

It is important to state that James Bach already have spoken his mind about this topic: Behavior-Driven Development vs. Testing

So here is example. Let’s write start of Shakespeare Hamlet in Gherkin:

Feature: ACT I SCENE I. Elsinore. A platform before the castle.

Scenario: FRANCISCO at his post meets BERNARDO

Given that Francisco is at his post

When Bernardo approaches in dark

Then Francisco asks who is there

Scenario: Bernardo replaces Francisco at his post

Given that is midnight

When Bernardo offers to replace Francisco at his post

Then Francisco accepts this greatfully

Scenario: Type of guard

Given that Francisco had quiet guard

When Bernardo asks him about his guard

Then Bernardo confirms that he had quiet guard.

And here is original play:


SCENE I. Elsinore. A platform before the castle.

FRANCISCO at his post. Enter to him BERNARDO
Who's there?
Nay, answer me: stand, and unfold yourself.
Long live the king!
You come most carefully upon your hour.
'Tis now struck twelve; get thee to bed, Francisco.
For this relief much thanks: 'tis bitter cold,
And I am sick at heart.
Have you had quiet guard?
Not a mouse stirring.

Notice the difference?

When we try to fit interesting user stories in Gherkin Given When Then format, we lose a lot of beautiful and important information. I read a lot of blog posts “How to effectively written scenarios in Gherkin format”.

For example: Describe every scenario in ONLY three steps, Given When Then !?

But scenario, not feature, is core of every application. They describe how application features, building blocks of scenario, are combined together to provide VALUE for the USER. This is how application will be used by REAL users. Do you really think that users think in Given When Then format?

This is like we have a hammer and we will make Eiffel tower with only hammer, here are the instructions.

Hamlet in gherkin is not a Hamlet. Why not read actual Hamlet in order to be able to test it?

Another point. Gherkin was introduced as communication tool. Gherkin is documentation communication tool.

So here is alternative that I propose. As I am testing various projects, for me the best sources of communication are:

  • application under test
  • tester that provides information about application and is happy to share it with others
  • developers that implemented that information into the application
  • git (github, bitbucket, gitlab) – to find out what changed in application

And when I communicate with them (or just listen their conversations or read slack chats) I always get UP TO DATE, LATEST information about application under test.


Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Stairs help for blind people


This post is follow up on my post about traveling to CAST 2016 as software tester. In that post I mentioned that I noticed at Paris Charles De Gaulle Airport metal pattern at the top of the stairs. Using Google, I tried to find out the purpose of that pattern, but with no result. Then community kicked in and I got response about the purpose of that pattern (tanks to another_one and Bruno Prsa).

That this pattern helps blind people to identify where stairs begin. This  blog post describes testing techniques that could help me to get to that conclusion by myself.

Michael Kelly article: Taking a tour through test country presents application tours that helps to get to know with the application. Two tours could have helped me to identify purpose of the pattern: user and scenario tours.

Kelly states:

The first is the user tour. In this tour, you attempt to imagine five users for the product and the information they would want from the product or the major features they would be interested in. The second tour is the scenario tour. Here, try to imagine five realistic scenarios for how the users identified in the user tour would use this product.

In BBST Test Design, you can learn about James Bach Heuristic Test Strategy Model where one of test technique is user testing.

Are you ready to enhance your testing craft?


Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Blog that makes software testing interesting and exciting.