Report on Testival 2016

TL;DR

This post is my report on Testival 2016 that was held last weekend in Rijeka STEPRI center.

 

First, a few metrics.

We had in total 32 participants, with female/male ratio 12/20!

As this is free event, dropout count is important metrics for us. 15 registered testers did not show, and did not let us know that they will not show. Which means that we wasted a lot of food.

Testival was first time international event:

Serbia: 4

Hungary: 1

Austria: 2

USA: 1

Liberland: 1

We are very excited about that metric!

Here are links to Facebook photo albums (you need to be log in in Facebook): Irja, Zeljko, Zeljko, Zeljko and me.

Here is my observation. Testers were not shy, they jumped in open session proposals without fear and hesitation:

img_1739

My impression is that several testers problems spawned through all themes.

I got a chance to further model my thinking around idea: “How to kill regression user interface automated testing”, and got excellent feedback and deeper understating of the problem. New material for blog post.

For me, new thing was Splunk product.

And this event introduced for the first time context driven testing school by Ben Simo, that has years of experience of applying context driven principles in his daily work.

Mirjana introduced exploratory testing, and how to transition in daily work from test case step descriptions towards exploratory testing.

My opinion what could be done better for next Testival:

Get proactive help in organising process.

Switch Testival to be Thursday/Friday event.

Add workshops on Thursday day.

More coffee.

Add additional sponsors.

In order to fight against food waste, symbolic price for registration for which value would be given back in form of software testing book or something similar.

Here is audience feedback:

img_1754

 

Again, without our sponsors, this event will not be possible:

screen-shot-2016-09-06-at-9-08-52-pm

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to get full stack trace during gunicorn exception investigation

TL;DR

This blog post will explain how to get full exception stack trace during the gunicorn start up issue. gunicorn is python native application server for running django application.

So you changed something in your django settings.py configuration, and gunicorn failed to start. In my case, I did not get full exception stack trace, which means I did not know what was the root cause for this issue.

Traceback (most recent call last):

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 196, in run

web_1 |     self.sleep()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 346, in sleep

web_1 |     ready = select.select([self.PIPE[0]], [], [], 1.0)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 231, in handle_chld

web_1 |     self.reap_workers()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 506, in reap_workers

web_1 |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)

web_1 | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

web_1 |

web_1 | During handling of the above exception, another exception occurred:

web_1 |

web_1 | Traceback (most recent call last):

web_1 |   File "/usr/local/bin/gunicorn", line 11, in <module>

web_1 |     sys.exit(run())

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/wsgiapp.py", line 74, in run

web_1 |     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/base.py", line 192, in run

web_1 |     super(Application, self).run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/base.py", line 72, in run

web_1 |     Arbiter(self).run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 218, in run

web_1 |     self.halt(reason=inst.reason, exit_status=inst.exit_status)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 331, in halt

web_1 |     self.stop()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 381, in stop

web_1 |     time.sleep(0.1)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 231, in handle_chld

web_1 |     self.reap_workers()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 506, in reap_workers

web_1 |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)

web_1 | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
 Google led me to this usefull blog post.
gunicorn slweb.wsgi:application --preload
 And official gunicorn documentation explains –preload option:
Load application code before the worker processes are forked.
Because first exception describes worker spawn failure, loading application before worker spawn may trigger more detailed exception.
And that was true:
django.core.exceptions.ImproperlyConfigured: 'oracle' isn't an available database backend.
Try using 'django.db.backends.XXX', where XXX is one of:

    'base', 'mysql', 'oracle', 'postgresql_psycopg2', 'sqlite3'
 The root cause was wrong name for oracle driver.
Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Android back arrow icon UX issue

In this post I will present a UX issue from a very famous engineer cultured company.

Every user interface should be developed in such manner that it is self descriptive. It should be self documented, so menu items or input fields should have labels that clearly describe their purpose. Tooltips must not be some fancy technology, but they should contains small pieces of business logic that describe how is this part of form connected with business logic.

Icons should also have that purpose. If I want to save something, please put floppy disk icon (thank you Microsoft for that mnemonic).

And then we have Android back icon. What is first mental image in your head when you see that icon? What is the purpose of that icon?

For me: I will first go right in menu, and when I get there, Android will AUTOMATICALLY make a U turn and then I will go to the left in application menu.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Feedback on my objection to gherkin language

TL;DR

In this post I will explain how not to engage in twitter war, but with simple politeness, provoke useful feedback about important topic.

Twitter war is very serious thing. Neil Studd talked about this problem at CAST 2016 lightning talk sessions. During his talk, simple idea popped up in my head. It was time for the experiment.

In my post I constructively object to Gherkin language I expressed my opinion on Gherkin language (which is first part on objection on using UI automation as regression purpose thing).

I always tweet my blog post, and this time I mentioned mr.  George Dinwiddie, because he asked my opinion, also via Twitter.

Here was his first reaction:

screen-shot-2016-09-10-at-6-26-07-pm

He used word strawman, and as I did not know that word, i looked it up:

A straw man is a common form of argument and is an informal fallacy based on giving the impression of refuting an opponent’s argument, while actually refuting an argument that was not advanced by that opponent.[1][Wikipedia]

So I learned new english word, and he objected that I am using gherkin for wrong purpose.

Here was my response:

screen-shot-2016-09-10-at-6-36-01-pm

So I did not engage in twitter war, I politely asked for a blog post. And we have thoughtful response about gherkin language: Using a good tool for the wrong thing. And that was experiment about idea during the Neil lighting talk.

And Ben Simo also responded:

screen-shot-2016-09-10-at-6-40-12-pm

So now we know true purpose of Gherkin language.

And in my original post I forgot to mention how I get information from business people, what product should do so I can create my testing mission. I get them at the beginning of github issues, as several issue comments, written in plain english language, with references to google docs and pictures about the design of the feature.

Using that feedback, here is my follow up on gherkin objection.

We have product examples that could be automate and run so we have immediate evidence that product actually behaves  as this example documents. Which is reasonable purpose.

The problem is:

  • when only those examples are used in testing the product.
  • they are used for regression testing.
  • they are used for UI regression testing (end to end testing) as only automated tests on project

Here is what I suggest:

  • do not stop testing using “manual” (highly intellectual activity)
  • create test mission “using tools at our disposal, lets find out what exactly changed in product from previous release” and do “manual” testing in that part of product. There is no need for automated regression testing on UI level.

As I find this matter highly important for testing community altogether, I would really appreciate your feedback on this matter.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

I constructevly object to Gherkin language

TL;DR

In this post, I give constructive example why using Gherkin as communication tool  is not productive. At CAST 2016, I gave 5 minute lightning talk “I constructively object to UI automation. This first part is about using Gherkin language as communication tool.

I sad in lighting talk (thanks Neil):

Screen Shot 2016-08-31 at 10.32.22 PM

That statement went to Twitter, and I got reasonable question:

Screen Shot 2016-08-31 at 10.32.34 PM

I consider “pretty sloppy” remark as “highly creative”.

I am giving example why I think that Gherkin language is bad for communication. And I welcome you to prove me wrong.

It is important to state that James Bach already have spoken his mind about this topic: Behavior-Driven Development vs. Testing

So here is example. Let’s write start of Shakespeare Hamlet in Gherkin:

Feature: ACT I SCENE I. Elsinore. A platform before the castle.

Scenario: FRANCISCO at his post meets BERNARDO

Given that Francisco is at his post

When Bernardo approaches in dark

Then Francisco asks who is there

Scenario: Bernardo replaces Francisco at his post

Given that is midnight

When Bernardo offers to replace Francisco at his post

Then Francisco accepts this greatfully

Scenario: Type of guard

Given that Francisco had quiet guard

When Bernardo asks him about his guard

Then Bernardo confirms that he had quiet guard.

And here is original play:

ACT I

SCENE I. Elsinore. A platform before the castle.

FRANCISCO at his post. Enter to him BERNARDO
BERNARDO
Who's there?
FRANCISCO
Nay, answer me: stand, and unfold yourself.
BERNARDO
Long live the king!
FRANCISCO
Bernardo?
BERNARDO
He.
FRANCISCO
You come most carefully upon your hour.
BERNARDO
'Tis now struck twelve; get thee to bed, Francisco.
FRANCISCO
For this relief much thanks: 'tis bitter cold,
And I am sick at heart.
BERNARDO
Have you had quiet guard?
FRANCISCO
Not a mouse stirring.

Notice the difference?

When we try to fit interesting user stories in Gherkin Given When Then format, we lose a lot of beautiful and important information. I read a lot of blog posts “How to effectively written scenarios in Gherkin format”.

For example: Describe every scenario in ONLY three steps, Given When Then !?

But scenario, not feature, is core of every application. They describe how application features, building blocks of scenario, are combined together to provide VALUE for the USER. This is how application will be used by REAL users. Do you really think that users think in Given When Then format?

This is like we have a hammer and we will make Eiffel tower with only hammer, here are the instructions.

Hamlet in gherkin is not a Hamlet. Why not read actual Hamlet in order to be able to test it?

Another point. Gherkin was introduced as communication tool. Gherkin is documentation communication tool.

So here is alternative that I propose. As I am testing various projects, for me the best sources of communication are:

  • application under test
  • tester that provides information about application and is happy to share it with others
  • developers that implemented that information into the application
  • git (github, bitbucket, gitlab) – to find out what changed in application

And when I communicate with them (or just listen their conversations or read slack chats) I always get UP TO DATE, LATEST information about application under test.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Stairs help for blind people

TL;DR

This post is follow up on my post about traveling to CAST 2016 as software tester. In that post I mentioned that I noticed at Paris Charles De Gaulle Airport metal pattern at the top of the stairs. Using Google, I tried to find out the purpose of that pattern, but with no result. Then community kicked in and I got response about the purpose of that pattern (tanks to another_one and Bruno Prsa).

That this pattern helps blind people to identify where stairs begin. This  blog post describes testing techniques that could help me to get to that conclusion by myself.

Michael Kelly article: Taking a tour through test country presents application tours that helps to get to know with the application. Two tours could have helped me to identify purpose of the pattern: user and scenario tours.

Kelly states:

The first is the user tour. In this tour, you attempt to imagine five users for the product and the information they would want from the product or the major features they would be interested in. The second tour is the scenario tour. Here, try to imagine five realistic scenarios for how the users identified in the user tour would use this product.

In BBST Test Design, you can learn about James Bach Heuristic Test Strategy Model where one of test technique is user testing.

Are you ready to enhance your testing craft?

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Google maps offline mode scenario testing

TL;DR

As I was traveling to CAST2016 in Vancouver, and roaming cost for 1MB of data traffic using my Croatian operator is 10 US$, I decided to use Google maps in offline mode.

Why you did not buy Canadian sim card with data plan? I investigated that option, and I could not find on web simple explanation how to do that. Also, doing business in Croatia is rather complicated, and putting that expense on my company account would be very complicated. So I decided to go with Google maps offline option, and using wi-fi where possible.

Day before travel day, I downloaded Vancouver map. In iOS Google map application, you need to search for Vancouver, and select in main menu offline areas. Touch big blue plus sign, and hit download.

Offline content is valid for one month.

First surprise is that route feature is only available for Cars option, bus and walk is not available. My heuristics is that this is because of security implications for walk option. Google only wants to guide you for walking using up to date information. For example, you do not want to go through some riots area.

Bus option is not available because bus timetables need to be up to date all the time to have the most accurate routing.

My current location works in offline mode, but only when airplane mode is off.

And one interesting scenario (BBST scenario testing) happened. On Paris airport, I enabled data roaming, because those prices are acceptable (Croatia is part of EU). On plain I switched on airplane mode on, with data roaming enabled. Next stop, Toronto, Canada. And guess what, IT IS NOT POSSIBLE TO TURN OFF DATA ROAMING WHEN AIRPLANE MODE IS ON.

I was afraid that I will get some data traffic after I turn airplane mode off. But, luckily, my phone was not able to connect to any of Canadian mobile networks, so I could turn off mobile data roaming without any cost.

Scenario testing is very important part of professional testing activity. It is unfairly called manual testing, giving the impression that it is low skill activity.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Tester on a journey

TL;DR

I traveled to Vancouver, Canada, in order to attend TestRetreat and CAST 2016. Here is the experience from the tester’s point of view.

First issue I found was at Zagreb Airport. Boarding card reader is only used just before you enter the Gate. Boarding card reader failed, and just in front of me, it was restarted. Scan of my board card returned “unknown flight” error. Despite that, I proceeded with boarding the plane.

In Paris CDG Airport, I took the picture (featured picture of this blog), because every step at the top of stairs section have this metal endorsement. I do not know why, I just documented that pattern.

While waiting for flight to Toronto, I did one exercise from book programming elixir. I was connected to Internet (only wifi connection), and started my Mac terminal. Exercise was a small program that is accessing the github api. I run it, and got ssl security exception, github certificate mismatch. This is security exception for man in the middle attack, and it means that you do not have direct connection with a server.

I opened Chrome and noticed airport wifi provider page where I needed to accept terms of service page. After that, I got direct connection with github api server.

Tester should never be bored during his journey.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Example of fast decision making

TL;DR

It this blog post I will give an example of fast decision making and explain why the skill of making fast decisions could make you better software tester.

Olympic basketball tournament started great for Croatian national team. They won over Spain, one of the best teams in last decade. Decision was made with last ball, when Dario Saric, new signed Philadelphia 76ers blocked one of the best world Players Paul Gasol.

What caught my attention as software tester was Dario’s statement about that block:

I saw that ball is not going to Nikola Mirotic (second center position Spain player), SO I JUMPED  BEFORE PAUL got the ball in order to block him.

That was fast decision (less that 1 second), he gambled a little bit, but with his observation, gambling was very justified.

He blocked Paul Gasol, and Croatia had an excellent start in Olympic tournament.

In order to be a better testers, you have to make a decision (for example, should I deploy to production this code change) in your context that is based on your observations. And it is important to start practicing your decision making by doing observations. You can start with your daily environment, it does not need to be connected with some software product.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Blog that makes software testing interesting and exciting.