Transforming to New Test and Assurance Approaches

Organisations used to slower rates of change are adopting new approaches of Agile, DevOps, and Continuous Delivery to achieve more rapid rates of systems change. The Scaled Agile Framework (SAFe) is one approach to transforming to an Agile organisation, but companies are finding the transition from enterprise software development to agile approaches requires more than a process change.

The roles of testing and assurance professionals especially, are usually poorly defined particularly where companies are attempting to shift testing activities to the left.

In this talk, Paul will set out the problems faced by less-agile organisations who are trying to implement agile software practices and identify three areas of transformation that are key to successful delivery:

• The transition from linear to event—driven processes of software delivery

• Understanding how to trust test automation to do what humans used to

• The new skills profile of test and assurance professionals and those who test

Agile transformation in software delivery teams does not automatically fit the culture of non-agile organisations. Paul suggests how test and assurance practitioners adapt to help this transition succeed.

Test Bots and AI Systems are coming: Rise of the Test Architect

Are robots taking the test world over? Will you lose your test job to AI? Is testing dead?

In this keynote, Jon Hagar will explore how you should prepare your test career for the intelligent tester future.

Yes, some categories of test jobs will go away because of concepts such as test automation, bots, and thinking AI systems. However, creative thinking still will be a valued tester skill. Additionally, tester knowledge and expertise will be needed on software projects.

Testers will improve their expertise by knowing about standards, building experience, increasing their flexibility, and practising constant new topic learning. An important tester job category will be the test architect-designer who will have skills in the areas of test including data analyst, planner, modeller, bug hunter, process practitioners, and AI engineer.

Teams will find these skills in schools, conferences, boot camps, exploration, and practice...practice...practice. Testing as a career is not dead, it will be changing over the next many years, and we must change with

Principles of Quality Assurance for AI-based Systems

TBC

The Next Generation Tester: Meeting the Challenges of a Changing IT World

This presentation looks at the future of the professional software tester and how it must adapt to the demands placed on it by a wide range of technological and business factors. By developing an outline for a “Next Generation (NG) Tester” the presentation reflects on the future skills set of the professional software tester. This must evolve by bringing testers closer to their stakeholders in their day-to-day work and enabling them to cope with changes in IT and project life-cycles. Without adapting to this demand, the tester as a profession runs the risk of losing its relevance!

The presentation starts by identifying the key developments in IT which satisfy the demands of business for shorter delivery cycles, better cost effectiveness and increased flexibility. These changes enable business to stay competitive and include significant advances in the digitalization of Industry (“Internet of Things”), increased use of mobile solutions and the adoption of robotics and AI/Knowledge-based concepts. In addition to the technology-driven factors, different ways of working such as Agile and DevOps have been steadily adopted as development life-cycle models. Each of these factors is described and the impact on testing outlined.

In its response to these issues, testing requires increased levels of test automation supported by frameworks and integrated tool sets, the modelling of requirements for “seemless” test automation (Model-Based Testing) and the adoption of test processes which are better aligned to Agile and DevOps development life-cycle models. Examples are given for each of these aspects based on the presenter’s insights into the current-day practices being adopted by industry.

The testing community (be it at overall industry level or within the organization or project) must now agree on what it considers to be good testing practice. This will have an impact (amongst others) on test processes, testing strategies and the tester’s contribution to quality engineering. It will also require the definition of an expanded skill set for the proposed NG Tester role which builds on the current tester role. A summary of this skill set is provided to support these discussions. The presentation concludes with some suggestions on how the adoption of new skills can be enabled. This focusses not only on training and certification possibilities, but also on the overall approach we should consider in helping people make the transition to NG Tester.

Enhancing Customer Experience in the Era of Digital Technologies through Intelligent Software Testing

Approximately 5000 years ago, although computers have technically been in use, it is modern computers that have had the greatest and most profound effect on society. Computers in the form of personal desktop computers, laptops, SmartPhones, tablets and wearable devices have become an inevitable part of our life.

Evolution of computers from 1st generation to fifth generation has led to significant changes in the way we perceive and communicate information. This has also led to changes to customer experience as well. To embrace this change, software quality assurance also requires adoption to new way of delivering to stay relevant to business demands and challenges.

This presentation will address few adoptions which TM has embarked to enhance customer experience in the era of Digital Technologies through Intelligent Software Testing, such as:

• Delivery Quality at Speed

• Introduction to AI in testing

• Security, Performance, Cross Browser Compatibility Testing and Usability Testing

• Embrace Agile and DevOps in QA

• Digital Customer Experience

• Enhancing SQA skills from Quality Assurance (QA) to Quality Engineer (QE)

Trends in Software QA – Adding Intelligence to Testing

Intelligence is a big word. Is intelligence the same as wisdom, knowledge, and smartness? In general, when people talk about intelligence with respect to software testing, they automatically think of Artificial Intelligence (AI). In this talk, Phil examines where AI has applications in testing, what AI can do today and possibly tomorrow. He’ll discuss how AI is currently being used in prediction to optimize where testing can be done most effectively. He’ll also discuss current applications in automation including some current work his company is doing in mobile test automation. Most importantly, he’ll discuss where AI works well and where it doesn’t. In light of the trend towards autonomous vehicles, is a driver still needed for software testing?

Deriving Intelligence from RE for Better Testing

The noun intelligence is defined as “the ability to acquire and apply knowledge and skills” and the Latin root of the word means “to understand”. This brings us to the core of Requirements Engineering (RE) and how intelligence from RE helps the testing process.

Talking about intelligence also touches on information vs. knowledge management. Matthias will provide insights into these terms and how understanding the difference will help to better gather intelligence and testing based on requirements.

Matthias will give an overview of what deriving intelligence from RE means, what approach can help testers in acquiring knowledge and which RE artifacts involved in different software processes (traditional waterfall, iterative and agile processes).

The talk concludes with an outlook on current methods of integrating AI methods into RE.

Topic TBC

TBC

Problem Solving for Testers

In some organisations, it is perfectly fine for testers to report failure as they experience them. To capture the details of behaviour that does not meet expectations, how to reproduce the problem and an assessment of severity and/or priority might provide enough information to allow developers to diagnose and debug the problem.

But in many situations, this simply does not work. For example, in a company that builds hardware and writes their own firmware and application software to diagnose the source of a problem can be a difficult task. Where a device has many, many configurations or connects to a range of other hardware, firmware or software applications it might be impossible to reproduce the problem outside the test lab.

In these situations – and they are increasingly common – the task of the tester is to look beyond the visible signs of failure and to investigate further: to narrow down possibilities; to identify and ignore misleading symptoms and to get to the bottom of the problem.

In this tutorial, Paul explores how we can be deceived by evidence and how we can improve our thinking to be more certain of conclusions. You will learn more about the design of experiments, recognise what you can and cannot control, learn how to systematically diagnose the causes of failure and work as a team to problem solve more effectively.

Problem Solving for Testers

In some organisations, it is perfectly fine for testers to report failure as they experience them. To capture the details of behaviour that does not meet expectations, how to reproduce the problem and an assessment of severity and/or priority might provide enough information to allow developers to diagnose and debug the problem.

But in many situations, this simply does not work. For example, in a company that builds hardware and writes their own firmware and application software to diagnose the source of a problem can be a difficult task. Where a device has many, many configurations or connects to a range of other hardware, firmware or software applications it might be impossible to reproduce the problem outside the test lab.

In these situations – and they are increasingly common – the task of the tester is to look beyond the visible signs of failure and to investigate further: to narrow down possibilities; to identify and ignore misleading symptoms and to get to the bottom of the problem.

In this tutorial, Paul explores how we can be deceived by evidence and how we can improve our thinking to be more certain of conclusions. You will learn more about the design of experiments, recognise what you can and cannot control, learn how to systematically diagnose the causes of failure and work as a team to problem solve more effectively.

Creative Intelligence for Today’s Test Team

Did the conference keynote topic of the morning sessions scare you? Good, but do NOT worry too much.

Robots, automation and even AI are good at some things, but testing will always need creativity, solid thought, experts, and inspiration. It will not happen overnight, but in the world of IT, change is a given constant. This workshop will explore these topics and help you to start to build skills in support of them.

Keys to support creative intelligent thinking include AI, dealing with big data, knowing the process rules (and how to break them), creative thinking with inside and outside of the box approaches, and finally staying flexible while embracing the changes as they happen.

Test team jobs and the supporting skills you will need include test: data analyst, architect-planner, modeller, designer, bug hunter, AI engineer, and manager. While some of these were mentioned in the keynote, in this workshop Jon Hagar will give attendees more hands-on exercises and practices to improve understanding of these topics.

Creative Intelligence for Today’s Test Team

Did the conference keynote topic of the morning sessions scare you? Good, but do NOT worry too much.

Robots, automation and even AI are good at some things, but testing will always need creativity, solid thought, experts, and inspiration. It will not happen overnight, but in the world of IT, change is a given constant. This workshop will explore these topics and help you to start to build skills in support of them.

Keys to support creative intelligent thinking include AI, dealing with big data, knowing the process rules (and how to break them), creative thinking with inside and outside of the box approaches, and finally staying flexible while embracing the changes as they happen.

Test team jobs and the supporting skills you will need include test: data analyst, architect-planner, modeller, designer, bug hunter, AI engineer, and manager. While some of these were mentioned in the keynote, in this workshop Jon Hagar will give attendees more hands-on exercises and practices to improve understanding of these topics.

Software Requirements in Agile

Regardless of whether you are working in a traditional or agile manner; all projects begin with requirements. Specifying software requirements not only provides a framework for the design as well as valuable documentation. In agile projects, well-written user stories promote collaboration and accuracy across all stakeholders including development, testing, and product management. Yet, many projects are derailed by poor specifications. Ambiguously written requirements, insufficient customer interviews, missing stakeholders, and communication failures can all lead to budget overruns, missed milestones, disappointed users and software that simply doesn’t do what it should. This workshop covers the basics your team will need to improve the requirements process: how to create unambiguous requirements, how to explore the solution space, how to discover stakeholders and users, and how to document functions, attributes, constraints, and expectations. With many hands-on exercises, you’ll go back to work with methods that you can use immediately to start improving the requirements process in your company.

How to Improve Mobile User Experience

UX has increased importance on the mobile platform, especially with subscription business models where users can switch applications in a heartbeat. So you've got only thirty seconds for your users to understand how to use your app and get value. How do you do that? A UX that works for and “talks” to the user, understanding what they want and when. To build a deeper understanding of how to improve your own app’s UX, Phil provides examples with key principles for design, test and evaluation of UX. Learn how to incorporate context and trust into your UX and keep your users coming back.

Improve Your Testing Process in Agile Projects

In this session, speaker delve into the scenario where testing fits into the agile project. Participants will learn about principal agile concepts and how these impact testing, understanding the tester's responsibilities and required skills set for testing in agile projects.

Speaker will also touch on how to approach test process improving in an agile project.

Improve Your Testing Process in Agile Projects

In this session, speaker delve into the scenario where testing fits into the agile project. Participants will learn about principal agile concepts and how these impact testing, understanding the tester's responsibilities and required skills set for testing in agile projects.

Speaker will also touch on how to approach test process improving in an agile project.

Building Blocks for Intelligence in Testing – a V&V Perspective

Verification & Validation (V&V) practices, regardless the degree of independence associated to it (i.e. regardless V&V or IVV), involve multiple types of activities – reviews, testing, auditing etc. – and produce various reports and readings. Collectively, these artefacts contain substantial analytics (and intelligence) which contribute to improving your testing practice and make you a better tester.

In this session, Amir will share his vast experiences in leveraging such artefacts as building blocks for adding intelligence in testing.

Practical Application of Intelligence in Testing – a V&V Perspective

Intelligence derived from artefacts and analytics will only be beneficial if applied properly and appropriately. Unfortunately, there is no Hard and Fast Rules in ‘how’ and ‘where’ to apply such intelligence. This can only be learnt from experience.

In this session, Amir will share his experience in leveraging applying intelligence in testing.

Test Modelling and Coverage

Testing is a process in which we create mental models of the environment, the program, human nature and the tests themselves.” Boris Beizer said that in 1990, but the idea that testers use models is much older than that. For almost everything humans do – that involves complexity – we create models to simplify, to scope, to mechanise, to understand.

Models are an essential part of being human. To take a few steps requires us to understand the configuration of all of the larger bones and joints of our bodies and the tensions in around 100 muscles. Our brains must understand all this, send an orchestrated set of nerve impulses to all these muscles to take a single step and calculate, calibrate and recalculate second by second – but much faster than that.

It takes huge processing power to control mechanical robots that simulate human movement. Humans simply don’t have that power, so we must simplify, through mental models. Modelling is essential, innate and human. Our brain is a superb modelling engine. As developers and testers, let’s use it to advantage.

In this tutorial, Paul explores how we think as testers and how we use models to simplify, scope and explain what we do. Consider how our understanding of a problem is in effect, a model. Our approaches to testing are models. We explain what we do as testers as meaningful models – to stakeholders. Paul will demonstrate and explain how successful test design and execution automation is based on models too.

Test Modelling and Coverage (continued)

Testing is a process in which we create mental models of the environment, the program, human nature and the tests themselves.” Boris Beizer said that in 1990, but the idea that testers use models is much older than that. For almost everything humans do – that involves complexity – we create models to simplify, to scope, to mechanise, to understand.

Models are an essential part of being human. To take a few steps requires us to understand the configuration of all of the larger bones and joints of our bodies and the tensions in around 100 muscles. Our brains must understand all this, send an orchestrated set of nerve impulses to all these muscles to take a single step and calculate, calibrate and recalculate second by second – but much faster than that.

It takes huge processing power to control mechanical robots that simulate human movement. Humans simply don’t have that power, so we must simplify, through mental models. Modelling is essential, innate and human. Our brain is a superb modelling engine. As developers and testers, let’s use it to advantage.

In this tutorial, Paul explores how we think as testers and how we use models to simplify, scope and explain what we do. Consider how our understanding of a problem is in effect, a model. Our approaches to testing are models. We explain what we do as testers as meaningful models – to stakeholders. Paul will demonstrate and explain how successful test design and execution automation is based on models too.

Test Intelligence and Data to Drive the Asian Mobile/IoT Future

How many of you have smart devices? How many computers do you use in a day? What are the concepts and skills we should be working on to improve test intelligence for such devices?

Software and computers are everywhere, being used by many people, and many times users are not even of aware they are using computers. We do however notice when these systems fail us or have bugs. These systems will always have bugs, even when we improve such systems with intelligence and drive them with data analytics.

So, working with computing systems and software testing are good career options. The new intelligent test jobs will come slowly but worse, the nature of these jobs will likely change over time. Additionally, there will be many jobs in the mobile, embedded and IoT system space. Jon Hagar will explore the basic engineering ideas of mobile/IoT systems with a tester focus. He will expand these basic ideas with hands-on practice in mobile/IoT devices using data analytics and intelligent test patterns.

ISO Standards, IV&V, and Test Planning for the AI Intelligent Future

Does AI in software mean that we will not need concepts such as standards, IV&V, and testing?

No. It is likely that many of the software systems we create will need these concepts and, in some cases, we may need new ideas. The hard part will be balancing machine learning with human thinking.

Jon Hagar will examine current “standards” for testing and IV&V. Then Jon will consider how these may need to change and be applied when AI and data analytics are incorporated into a project. To support these concepts, skilled and expert testers will be in demand worldwide

One definition of expert that Jon Hagar likes is someone that knows the rules but how to break the rules when they must be broken. Breaking of some rules will be needed for the AI-driven projects of the future. Jon will examine some of the rules of standards and then work exercises where we “break” things.

Risk Management in Agile

Software componentization has made software more unpredictable because unforeseen conditions can cause components to interact in ways we hadn’t imagined. Combined with greater complexity, increased user expectations and our desire to use Agile with ever increasing velocity, uncertainties and risks need to be actively managed. Classic risk management identifies risks and prioritizes them to determine impact to the project, but how does that differ in an agile project? Agile is designed to handle uncertainty in requirements as new features are requested and priorities shift. But what about the other uncertainties outside of requirements changes? Understanding those risks even before the project gets started, and those that can possibly derail the project after delivery are critical. Phil provides insight into the uncertainties and risks involved in agile software projects and supplements classic risk management approaches with how and when to apply within an agile process. Through a hands-on approach, Phil outlines risk management practices at each stage of the agile process. Take home pragmatic knowledge to identify and mitigate risks with the goal of focusing teams on the right things at the right time and producing higher quality software.

Software Quality Metrics the Agile Way (or Not)

When implementing software quality metrics, we need to first understand the purpose of the metric and who will be using it. Will the metric be used for measuring people, the process, illustrate the level of quality in software products, or drive towards a specific objective? QA managers typically want to deliver productivity metrics while management may want to see metrics that support customer or user satisfaction or cost related (ROI) initiatives. With agile development methods, we often lose sight that our primary objective is the same, quality. Many organizations are myopic on the primary objective of velocity. However, velocity means little without quality. Define quality for your organization with an agile looking glass with intermediate metrics that lead to both quality and velocity. Learn to develop and implement software quality metrics with actions toward improving on these two primary objectives.

Soft Skills for Testers and Test Managers

Soft skills are getting more and more important. It's an "old" subject but still very relevant. In this session speakers will cover issues of Appreciating the significance of soft skills in today's projects and principal techniques to apply. Not the least, speakers will also dive into the question: What can go wrong when soft skills are not applied?

Model-Based Testing in a Nutshell

In addition to going through the concept of MBT, the speaker will also elaborate on benefits of using MBT (adding "intelligence" to your testing approach), basics of modelling and how this is useful for testing. He will also address the subjects of tools support MBT the ‘Dos’ and ‘Don’ts in applying MBT

Intelligence-driven risk-based Testing

In risk-based testing, it is crucial to get the prioritization of test cases correct, A thorough risk assessment process or methodology is required. While key business drivers remain as the foundation for critical testing decisions, analytics derived from historical data have been widely used to enhance the risk assessments. With enriched analytics, intelligence gets into play and this is elevating testing to a higher level.

Learn more about it in this session.

Intelligence-driven Software Defect Prevention

As the age-old saying goes, “prevention is better than cure”. The same principle applies in good Software Quality Assurance (SQA) practice. Existing defect prevention strategies are largely based on historical data and lessons learnt from past projects. With enriched analytics, intelligence gets into play.

Coming from a TMMi Level 5-certified organisation where defect prevention is a key differentiator, Amir elaborate more on how intelligence can drive software defect prevention.

SCM and Intelligent Software Engineering

Can you name the critical skills now required for a degree in software engineering? Here are a few found in a quick Google search for software engineering: AI, machine learning, digital systems design, robotics, factory automation, pattern recognition, numerical methods, computer graphics, and database management just to name a few. While these skills will definitely help in the architecture and design of systems, they won’t assist in the process of creating credible software.

One of the critical skills in establishing the credibility of software is understanding and putting into practice SCM—no matter the type of development project or products or role on project. Laura will cover basic SCM skills; measures to ensure software credibility including standards; types of data to demonstrate software validity; collaborations on the Agile/Dev-Ops team with each other, their customers or management. Laura will also offer a discussion on how AI and tools could possibly change how SCM is practiced in the future.

SCM – Defining a Strategy for Product Integrity to Support Intelligence in Testing

How do you know that you can trust any software product you come in contact with (cars, trucks, airplanes, medical devices, entertainment systems, home appliances, etc.)? Does anyone on your team really know what the software is designed to do or how security for the product works? How can your customer trust that your team has produced a valid or reliable product in a known configuration for them?

Laura will provide training on 7 basic SCM concepts and how those SCM concepts support intelligence in testing and product integrity to include a discussion of requirements traceability, software product configuration control, data analysis, and test traceability. Participants should be prepared to ask and answer tough questions about software products and the many implications of poorly designed, developed, or tested software, and how SCM and intelligence in testing can assist in product integrity.

Behavior-driven Testing Technique (using the testing tool Cucumber)

Speaker will provide overview on behavior driven design/ development/ testing and how this technique is used in agile development, how this technique is related to requirements engineering. Specifically, the speaker will be using the Cucumber testing tool as part of the hands-on experience during the session.

Notes:

• No programming language background needed but participants need to bring a laptop (Windows/ Linux/ Mac).

• Cucumber installation will be provided as a Virtual Machine image. Participants need to install the free tool Virtual Box on their Laptop

Usability Testing – a Practical Perspective

This is an introductory workshop on Usability Testing. The session covers offers theories aspects of Usability Testing and hands-on application of key elements of the theories.

Notes:

• Participants would need a smart phone or tablet

• Participants would need to install a simple app used for the usability test

• No knowledge in UI/UX design or testing needed

Intelligent Test list Generator for IoT Systems based on Artificial Intelligence Techniques

Speaker will provide overview on how to create test list based on the combination between all IoT Systems inputs using Artificial Intelligence techniques. Specifically, the speaker will introduce some existing online combinatorial tools as well as give a softcopy of their developed tools. A tutorial of how to use the tool to generate the test list with minimum size that cover the most important combinations in IoT Systems.

Notes:

• No programming language background needed but participants need to bring a laptop (Windows/ Linux/ Mac).

• The tool will be provided as a JAR file. Participants need to install the free tool on their Laptops. JDK is recommended to be installed in the participants laptops to be easy to use the existing online tools.

Topic TBC

TBC

First Step Of Test Architecture Design

For testing large-scale and/or mission-critical systems it is necessary to design test architecture though concepts of test architecture are not yet well established now.

In this workshop, the speaker shares the fundamental concepts, principles and heuristics of test architecture design, while you can try to make and review parts of test architecture design in group works.

First Step Of Test Architecture Design (continued)

For testing large-scale and/or mission-critical systems it is necessary to design test architecture though concepts of test architecture are not yet well established now.

In this workshop, the speaker shares the fundamental concepts, principles and heuristics of test architecture design, while you can try to make and review parts of test architecture design in group works.

Mining Intelligence from QS Certification Report

MSTB`s Quality Software Product Certification Scheme (QS) provides an independent, third-party certification of software products based against international standards. The reports that came along with the QS certification contains “intelligence” and additional insights on how to, other than improve the software product in question; improve quality assurance & testing process and even future development project(s).

We will show how you can maximize the benefits of going on the QS certification journey with us, apart from just getting the QS certificate itself!

Case Study: Leveraging Intelligence from QS Certification

We’ll share with this session participants, experience of the QS certificate holder on the “intelligence” provided pre and during the QS certification journey. They will be sharing their experience on the QS certification journey and on how they utilize the intel provided to them in planning product releases, in improving their software product, supporting quality assurance and testing process, amongst others.

Be sure to secure your seat to get all these insights.