Can you give me a test environment?

Graham Maxfield

That is a question which most environment managers will hear all too often. So what is the answer? Yes, would be obvious, but I usually counter with an array of questions such as: What do you understand by “an environment”? What do you want it for? What will be tested on it? Who will manage the testing for the environment? Who will use the environment? When do you need it and how long for? Where do you want it? What should be in the environment? Which interfaces should be used? Do you need stubs? How flexible does it need to be? Who will manage the testdata? How many instances does it make sense to have? Does it need to reflect production? How well does it need to perform? What is the budget? and the list goes on…

Many of the requesters will start to look vague by the third question, and then totally lost by the last. The simple request has become a lot more complex and testing will not likely start within the next hour because the test environment manager wants to understand what is actually required, as opposed to what the tester thought he or she might like.

It is not that I wish to be a pain nor waste time unnecessarily, but getting it right the first time can save money and time in the future. It should be the aim of any environment manager to build stable, manageable and well prepared environments which are fit for purpose and do not contain excess baggage.

As I have previously stated, effective testing relies on reproduction. This can be achieved by keeping changes to a minimum and properly managed.

Knowing exactly what is in the environment and why it is there will allow the manager to better coordinate change. Assessing the impact of environmental changes with the testing team and development whilst understanding the possible effects it might have on the rest of the environment, or other environments for that matter, should reduce the number of unforeseen issues.

Environments have a tendency to get very messy very quickly if inefficiently managed, especially when used in parallel. Timelines, requirements, software, versioning, databuilds, servers, storage, databases and testdata can get out of hand when not well coordinated. People, especially those in development, are quick to question the environments when bugs are found in testing. And not knowing what was where when the software was tested and not being able to reproduce a defect can result in longwinded and expensive analysis phases, giving test environments their bad name.

I have to laugh when requests for test environments to support complex and costly projects are proposed over coffee, without an ounce of thought going into what it is that is actually needed. I also enjoy watching the requester deflate as I question the needs. That is priceless and makes up for a lot of the stress which follows 🙂

redditpinterestredditpinterest

Who needs test environment management anyway?

Graham Maxfield

Ah yes, test environments; the first point of blame for development and testers alike, the black box, where information goes in and information comes out, the annoying, time consuming and mostly boring part of testing. From my experience people either love them or hate them. But regardless of emotional preferences, they are a pretty bloody vital aspect of testing. In fact without a well-managed test environment, you might as well not bother testing at all. And without testing, who needs testers?

… so, where does this leave us?

With one of the crucial aspects in testing being reproducibility, it is key to properly maintain and manage your test environments.

There is of course a distinct difference between managing an environment to test a simple, standalone app, compared to testing a whole system landscape, however the principles and benefits are the same. A test environment should be cost effective, reliable and purposeful and it needs to be managed so that software defects are reproducible.

Test environments are very much a part of testing, hence the name, however they seem to linger somewhere between developers and testers. Developers see them as an extended playground, testers as a misunderstood necessity and project managers as a budget eating nuisance.

The physical role of a test environment manager is in many projects unpractical and unnecessary, but the responsibilities involved should be addressed and practised; especially with the speed, size and capabilities of current technology. And as we know, things are only going to get faster, bigger and more complex in times to come, challenging testers and environments to keep up the pace.

In the World Quality Report 2013-14, 55% of organizations stated a lack of Test Environment Management skills in-house. http://www.capgemini.com/thought-leadership/world-quality-report-2013-14

As a professional Test Environment Manager, these figures do not surprise me and only reflect what I see in so many of my major clients.

redditpinterestredditpinterest

What would be the worst start to a conference? 3/3

Graham Maxfield

Let’s Test Sweden Day 3

Refreshed after about 3 hours of snooze, I tucked into my breakfast and looked forward to the day ahead. The first session was a very well presented workshop from Fiona Charles – We Can’t Know Everything – Promoting healthy uncertainty on software projects.           There was a lot of class debate, allowing me to take away many different ideas. From Eddy explaining The Orders Of Ignorance to Kristoffer Nordström touching on the idea of No Estimates.

Following that managed to spend some time with Panda discussing his business.

I tried to talk to as many people as possible with it being the last day. I finally got to spend some time to talk Erik Davis too. With his bright orange hair and kilt, I was hoping for a fellow Scottish Punk tester. What a flop! He isn’t Scottish and he isn’t a punk :(, but what a great guy. I got on well with him, as I did with everyone I talked to. I got round to seeing Michael Bolton, Ilari Henrik, Ruud Cox, Neil Thompson and a whole heap of other attendees.

The final keynote from Jon Bach was a great way to finish the conference, with a lot of laughs and fun, considering the serious natural of the topic – A Critical Look at Best Practices. Good input from James, Michael and a range of others made it all the more enjoyable.

After that, Let’sTest finally wound down to a close. After saying my farewells, I hitched a ride to the Airport with Ruud and two of his fellow countrymen. Having the navigation system speak Dutch, whilst pronouncing Swedish towns made me love software even more. Thanks for the lift.

With my Germanwings Mobile App informing me that everything was fine with my flight I headed straight to security and proceeded with the annoying process of “I’m not a terrorist, get me out of here”. Once on the other side I saw that, oh so welcoming, information on the monitor. FLIGHT CANCELLED. Apparently a bird hit the turbine. What a pain! I left the secure area and took a ticket to wait for the helpdesk to advise me of my next moves. Funnily/sadly enough, Chris Mann was also hoping to fly to Hamburg, so we had a bit more time to chat.

I finally decided to fly to Düsseldorf that evening and try to take the train to Hamburg Airport to pick up my car. As it turned out, the trains aren’t great at that time of night. Well not the ones from Düsseldorf Airport to Hamburg Airport. As I was already used to meeting new faces by then I agreed to share a hire car and drive to Hamburg. 4 hours later we arrived at Hamburg Airport where I picked up my car and travelled the final 1,5 hours home. I got through the door at 05:00 in the morning totally dead!

What a trip!

After reflecting on my first test conference and finally finishing this blog, I really look forward to next year and will be booking my tickets for another round of Let’s Test as soon as I can. Now knowing many of the attendees, I think 2015 is going to be even better.

A huge thanks to the organisers. Keep up the good work.

 

If you are going to the EuroStar 2014….. I’ll see you there.

redditpinterestredditpinterest

What would be the worst start to a conference? 2/3

Graham Maxfield

Let’s Test Sweden Day 2

After a lovely breakfast the first keynote of the day was opened in a fine display of “who cares how I look”, as the facilitators jumped on stage and shook whatever they had in some strange dance. (You can YouTube this).

Steve Smith then took to the stage to present Whose Ideas Are In and Whose Ideas Are Out. For this, groups of about 12 were formed and given a Flipchart sized sheet of paper. The rules were displayed on the screen and briefly discussed before for letting the teams loose. The object was to have as many people on the paper as possible without anyone touching the floor. The first round didn’t prove too difficult, but each subsequent round, the paper was halved. I spent a lot of time trying to convince our facilitator, Lars Sjödahl, to let us bend the rules by either creating a large paper loop for everyone to stand on, or allowing me to place a group photograph, taken from my mobile phone, on the piece of paper, thus having us all aboard for a whopping 12 points. Naturally the exercise was nothing to do with points, rather how people reaction in teams when decisions have to be made. I’d like to say thanks to my team for making the experience so enjoyable. https://twitter.com/Brickuz/status/471210995877425152

We then presented our findings on stage, where I gave my “first keynote speech”.

Later that morning I visited Panda’s Testers as respected business solvers a true story workshop. I had high expectations after the session he gave on day 1 with James and Panda did not let me down. I was impressed with his spin on Business Driven Testing, taking what I conceived to a new level. I was impressed and definitely wanted to talk to this guy!

After a tasty lunch Carsten Feilberg’s Getting Problems Sorted workshop was on. I was a little disappointed with some of the ideas after such a good start to the conference. It seemed rather over the top to act out situations where communication is the main bottleneck. It may have help some people cope with issues, but I did not take much away from the session.

Follow that I attended Tim Coulter’s Marketing yourself (or why quitting is something you should consider). This was also not really what I had hoped for. I understand that losing the presentation notes for whatever reason can set you off to a bad start, but Tim did manage to pick up the session after a bit of a downer, which saved him. His point about quitting is valid and is something I have done before, but regulations in many European countries don’t allow you to jump ship from one day to the next.I would have liked some new ideas, but I did take away that I am not the only one to have had 10 projects or jobs in 10 years and that others do the same to market themselves.

Bill Mathews did a very interesting demonstration about security testing with SQL injection, where he hacked a prepared site and showed us what to look out for when testing.

That evening I spent my time connecting with as many people as possible. I had a good chat with Richard Bradshaw and sat until 4 o’clock the next morning talking to Dawn Haynes, Huib Schoots, Stephen Blower and Lars Sjödahl. It was already light again by that time so sleeping wasn’t really possible anyway.

redditpinterestredditpinterest

What would be the worst start to a conference? 1/3

Graham Maxfield

What would be the worst start to a conference? Missing your flight? I was pretty bloody close to doing so…

Let’s Test Sweden Day 1

After leaving with what I thought was ample time, I arrived at the airport just 15 minutes before boarding. On a good day in my regular airport, Hannover, that wouldn’t be an issue, but this time I was flying from Hamburg! I got to experience the “Oh f*ck” feeling as I entered the terminal and saw a huge queue, all waiting to pass security. Maybe I should have left at 04:00 instead of 04:30? But, luckily it only took me 35 minutes to get to my gate, just in time for the final call.

So slightly hotter than comfortable, I found my seat and settled in. The last person on the plane was, as usual, some massive guy. The plane really wasn’t that big and my shoulders were already wider than the backrest, so as he neared the empty seat next to me I prepared for “one of those flights”! Luck was on my side again though, as he sat down two rows in front of me, squint-necked to avoid the overhead luggage storage.

The flight was as pleasant as it could have been and with the sun glistening on the lakes of Sweden I was able to relax and once again feel excitement for my first Let’sTest.

I had arranged to meet Eddy Bruin in his terminal (which was not good as the coffee shop’s machine was broken) and share the taxi ride to Rüno. We spent the journey, as one would expect, discussing testing and exchanging experiences.

Once at the conference centre, we were greeted by Jon Bach, before we checked in and found our rooms. The first stop after that was, of course…. the coffee machine. By this time the first keynote was well underway so the building was still rather quiet. As I strolled down the hall heading for a shot of caffeine, I spotted the back of a head, peering over the top of a sofa. Easily identifiable as James Bachand his Satisfice cap, I was pleased to finally meet him in person. Busy with his preparation, he agreed to catch up with me later, then introduced me to Panda – aka. Pradeep Soundararajan and was adamant I spend time with him and his knowledge of Indian testing.

I then enjoyed my coffee with Eddy, who introduced me to Scott Barber. We had an interesting discussion about Swedish developers as both he and I have dev teams in Stockholm. We also touched on one of my favourite topics, performance testing.

The room rapidly filling up signalled the end of Tim Lister’s keynote and after a short break it was time for the first of the day’s seminars. After recently finishing James’s RTI online course and meeting him and Pradeep, I was looking forward to attending the class – Review by Testing: Analzying a Specification by Testing the Product.

A large interest and lack of tables caused for some disruption and chaos for the first few minutes, but we quickly settled in. There was a sense of nervousness and anticipation in the room, as not many people seemed to know each other, which continued into the first task. Then to the surprise of everyone, James sent Panda out of the room and said he would do the session himself.

He gave us the task of analysing a simple spec for a volume control program, noting what we think we should test. Split into groups to do this, the “who will speak first” scenario arose. Being the only native English speaking in my group, I tried to easy the others into sharing their ideas as we decoded the spec. Although we had difficulty fully understanding it, we put together some basic testing ideas.

Panda was called into the room, and received the same spec from James. In an Expert Challenge he was then asked to analyse the document, thinking aloud. This was my first experience of an Expert Challenge and it was a great insight into how others work through their testing ideas and knowledge when confronted with something new.

We then got to test the program and try out our ideas. This led to many more questions based on the spec vs the behaviour of the program. We spent a fair bit of time discussing our findings as a class, having James and Panda tease our minds regarding what we thought we know.

It was a great exercise to show that even a tiny, simple volume control program can generate so many questions, assumptions and misconceptions when testing it.

Another exercise was to test a program before seeing the spec, hence the name of the session. It was a program I had previously tested on James’s RTI course, just a newer version. Due to this, I found it hard to think about how to best test it from scratch. My previous tests kept coming up and my expectations didn’t allow me to really think freely. When we received the spec it totally calmed my thought patterns. The only thing which allowed me to take something away from this task was watching and listening to other people analyse the same program and spec.

I had a similar prejudice with the next software which we were given to test as I had again used it before. It was interesting for me to note, that what I think I know about a program deeply influences how I test it. The purpose of this exercise was however to use/test the software, creating a spec based on what we saw. We were only focusing on one aspect of the program, but we, as testers, still managed to come up with a large wish list of specs.

One of the main points I took away from the seminar was – “The people who write specs are not professional writers. They do not know how to write and they probably hate writing anything, but code”. – James Bach

It was a great day seminar which taught me a lot about how I, and others, analyse programs.

We got lunch, coffee and cakes at the due times and after the classes were finished we enjoyed a hearty evening meal. Following that, the evening fun started.

I headed straight for the TestLab, run by James Lyndsay. With a very early start to the day and hours of James and Panda, I was already suffering from dead brain syndrome. But I still somehow let myself get talked into trying to solve the Black Box puzzle. Luckily I had Sandro Ibig helping me analyse the machines. A while later (and a good few beers later), Neil (the) Studd join us and brought some much needed wind into the round (plus more beer). One of the fabulous female colleagues (I’m sorry, I don’t remember the name) also helped us pick our way through the array of lights and yes, it took some time, but we finally understand what is was that weird little box was doing.

redditpinterestredditpinterest