Transforming to New Test and Assurance Approaches

Organisations used to slower rates of change are adopting new approaches of Agile, DevOps, and Continuous Delivery to achieve more rapid rates of systems change. The Scaled Agile Framework (SAFe) is one approach to transforming to an Agile organisation, but companies are finding the transition from enterprise software development to agile approaches requires more than a process change.

The roles of testing and assurance professionals especially, are usually poorly defined particularly where companies are attempting to shift testing activities to the left.

In this talk, Paul will set out the problems faced by less-agile organisations who are trying to implement agile software practices and identify three areas of transformation that are key to successful delivery:

• The transition from linear to event—driven processes of software delivery

• Understanding how to trust test automation to do what humans used to

• The new skills profile of test and assurance professionals and those who test

Agile transformation in software delivery teams does not automatically fit the culture of non-agile organisations. Paul suggests how test and assurance practitioners adapt to help this transition succeed.

Test Bots and AI Systems are coming: Rise of the Test Architect

Are robots taking the test world over? Will you lose your test job to AI? Is testing dead?

In this keynote, Jon Hagar will explore how you should prepare your test career for the intelligent tester future.

Yes, some categories of test jobs will go away because of concepts such as test automation, bots, and thinking AI systems. However, creative thinking still will be a valued tester skill. Additionally, tester knowledge and expertise will be needed on software projects.

Testers will improve their expertise by knowing about standards, building experience, increasing their flexibility, and practising constant new topic learning. An important tester job category will be the test architect-designer who will have skills in the areas of test including data analyst, planner, modeller, bug hunter, process practitioners, and AI engineer.

Teams will find these skills in schools, conferences, boot camps, exploration, and practice...practice...practice. Testing as a career is not dead, it will be changing over the next many years, and we must change with

Principles of Quality Assurance for AI-based Systems

Artificial Intelligence is now the most powerful technology in the world for IT systems such as smart speakers, factory automation and autonomous driving as well as content generation, e.g. conversation, music, images, movies. Quality assurance of them needs innovation because AI services and systems uses machine learning technology, which is quite different from traditional software engineering.

We established a consortium last year named QA4AI together with industry and academia in Japan, and the QA4AI consortium has just released a guideline for organizing principles of quality assurance for AI-based services and systems.

In this presentation, Nishi will introduce the overview of the guideline and discuss the innovation and the principles of QA for AI-based services and systems.

The Next Generation Tester: Meeting the Challenges of a Changing IT World

This presentation looks at the future of the professional software tester and how it must adapt to the demands placed on it by a wide range of technological and business factors. By developing an outline for a “Next Generation (NG) Tester” the presentation reflects on the future skills set of the professional software tester. This must evolve by bringing testers closer to their stakeholders in their day-to-day work and enabling them to cope with changes in IT and project life-cycles. Without adapting to this demand, the tester as a profession runs the risk of losing its relevance!

The presentation starts by identifying the key developments in IT which satisfy the demands of business for shorter delivery cycles, better cost effectiveness and increased flexibility. These changes enable business to stay competitive and include significant advances in the digitalization of Industry (“Internet of Things”), increased use of mobile solutions and the adoption of robotics and AI/Knowledge-based concepts. In addition to the technology-driven factors, different ways of working such as Agile and DevOps have been steadily adopted as development life-cycle models. Each of these factors is described and the impact on testing outlined.

In its response to these issues, testing requires increased levels of test automation supported by frameworks and integrated tool sets, the modelling of requirements for “seemless” test automation (Model-Based Testing) and the adoption of test processes which are better aligned to Agile and DevOps development life-cycle models. Examples are given for each of these aspects based on the presenter’s insights into the current-day practices being adopted by industry.

The testing community (be it at overall industry level or within the organization or project) must now agree on what it considers to be good testing practice. This will have an impact (amongst others) on test processes, testing strategies and the tester’s contribution to quality engineering. It will also require the definition of an expanded skill set for the proposed NG Tester role which builds on the current tester role. A summary of this skill set is provided to support these discussions. The presentation concludes with some suggestions on how the adoption of new skills can be enabled. This focusses not only on training and certification possibilities, but also on the overall approach we should consider in helping people make the transition to NG Tester.

Enhancing Customer Experience in the Era of Digital Technologies through Intelligent Software Testing

Approximately 5000 years ago, although computers have technically been in use, it is modern computers that have had the greatest and most profound effect on society. Computers in the form of personal desktop computers, laptops, SmartPhones, tablets and wearable devices have become an inevitable part of our life.

Evolution of computers from 1st generation to fifth generation has led to significant changes in the way we perceive and communicate information. This has also led to changes to customer experience as well. To embrace this change, software quality assurance also requires adoption to new way of delivering to stay relevant to business demands and challenges.

This presentation will address few adoptions which TM has embarked to enhance customer experience in the era of Digital Technologies through Intelligent Software Testing, such as:

• Delivery Quality at Speed

• Introduction to AI in testing

• Security, Performance, Cross Browser Compatibility Testing and Usability Testing

• Embrace Agile and DevOps in QA

• Digital Customer Experience

• Enhancing SQA skills from Quality Assurance (QA) to Quality Engineer (QE)

Trends in Software QA – Adding Intelligence to Testing

Intelligence is a big word. Is intelligence the same as wisdom, knowledge, and smartness? In general, when people talk about intelligence with respect to software testing, they automatically think of Artificial Intelligence (AI). In this talk, Phil examines where AI has applications in testing, what AI can do today and possibly tomorrow. He’ll discuss how AI is currently being used in prediction to optimize where testing can be done most effectively. He’ll also discuss current applications in automation including some current work his company is doing in mobile test automation. Most importantly, he’ll discuss where AI works well and where it doesn’t. In light of the trend towards autonomous vehicles, is a driver still needed for software testing?

Deriving Intelligence from RE for Better Testing

The noun intelligence is defined as “the ability to acquire and apply knowledge and skills” and the Latin root of the word means “to understand”. This brings us to the core of Requirements Engineering (RE) and how intelligence from RE helps the testing process.

Talking about intelligence also touches on information vs. knowledge management. Matthias will provide insights into these terms and how understanding the difference will help to better gather intelligence and testing based on requirements.

Matthias will give an overview of what deriving intelligence from RE means, what approach can help testers in acquiring knowledge and which RE artifacts involved in different software processes (traditional waterfall, iterative and agile processes).

The talk concludes with an outlook on current methods of integrating AI methods into RE.

How Worldwide (Software) Trends impact the Future of Software Testing

The changing global landscape creates opportunities and poses challenges for the world of technology and software testing. Together, we'll discuss emerging trends, make a few predictions, and talk about the future of software testing.

Test Modelling and Coverage - Exploration and Informal Models 

“ Testing is a process in which we create mental models of the environment, the program, human nature and the tests themselves.” Boris Beizer said that in 1990, but the idea that testers use models is much older than that. For almost everything humans do – that involves complexity – we create models to simplify, to scope, to mechanise, to understand.

Models are an essential part of being human. To take a few steps requires us to understand the configuration of all of the larger bones and joints of our bodies and the tensions in around 100 muscles. Our brains must understand all this, send an orchestrated set of nerve impulses to all these muscles to take a single step and calculate, calibrate and recalculate second by second – but much faster than that.

It takes huge processing power to control mechanical robots that simulate human movement. Humans simply don’t have that power, so we must simplify, through mental models. Modelling is essential, innate and human. Our brain is a superb modelling engine. As developers and testers, let’s use it to advantage.

In this session, Paul explores how we think as testers and how we use models to simplify, scope and explain what we do. Consider how our understanding of a problem is in effect, a model. Our approaches to testing are models. We explain what we do as testers as meaningful models – to stakeholders.

Part 1 – Exploration and Informal Models

·         • A New Model for Testing

·         • Exploring a System as a Source of Knowledge

·         • Heuristics, Cheat Sheets and Check Lists

Test Modelling and Coverage - Testing with Formal Models

“Testing is a process in which we create mental models of the environment, the program, human nature and the tests themselves.” Boris Beizer said that in 1990, but the idea that testers use models is much older than that. For almost everything humans do – that involves complexity – we create models to simplify, to scope, to mechanise, to understand.

Models are an essential part of being human. To take a few steps requires us to understand the configuration of all of the larger bones and joints of our bodies and the tensions in around 100 muscles. Our brains must understand all this, send an orchestrated set of nerve impulses to all these muscles to take a single step and calculate, calibrate and recalculate second by second – but much faster than that.

It takes huge processing power to control mechanical robots that simulate human movement. Humans simply don’t have that power, so we must simplify, through mental models. Modelling is essential, innate and human. Our brain is a superb modelling engine. As developers and testers, let’s use it to advantage.

In this session, Paul explores how we think as testers and how we use models to simplify, scope and explain what we do. Consider how our understanding of a problem is in effect, a model. Our approaches to testing are models. We explain what we do as testers as meaningful models – to stakeholders.

Part 1 – Testing with Formal Models

·         • Introducing Model-Based Testing 

·       • How do we Choose Our Models?

·        • Coverage

·       • Models in Use

Creative Intelligence for Today’s Test Team

Did the conference keynote topic of the morning sessions scare you? Good, but do NOT worry too much.

Robots, automation and even AI are good at some things, but testing will always need creativity, solid thought, experts, and inspiration. It will not happen overnight, but in the world of IT, change is a given constant. This workshop will explore these topics and help you to start to build skills in support of them.

Keys to support creative intelligent thinking include AI, dealing with big data, knowing the process rules (and how to break them), creative thinking with inside and outside of the box approaches, and finally staying flexible while embracing the changes as they happen.

Test team jobs and the supporting skills you will need include test: data analyst, architect-planner, modeller, designer, bug hunter, AI engineer, and manager. While some of these were mentioned in the keynote, in this workshop Jon Hagar will give attendees more hands-on exercises and practices to improve understanding of these topics.

Tester/QA Career Development for Today and Intelligent Tommorow

Have you ever wondered how to keep you career and jobs alive in this everchaning world of software and information tech (IT)? Have you heard testing/QA is dead? Well it is not.

Come to this fun and highly interactive workshop if you want to learn how to improve your testing/QA career. You will participate in exercises to define: tester/QA skill lists, what your skills are, where the test/QA industry jobs are and will be, build your own career plan, and how you can work on your career plan to have a happy and productivw test/QA career.

Jon Hagar will help you establish a path forward in this everchanging IT world. The path and exercise have worked for Jon during over 40 years of change, growth, and having fun in testing/QA. You too can have a great testre/QA career.

Attendance is limited.

Software Requirements in Agile

Regardless of whether you are working in a traditional or agile manner; all projects begin with requirements. Specifying software requirements not only provides a framework for the design as well as valuable documentation. In agile projects, well-written user stories promote collaboration and accuracy across all stakeholders including development, testing, and product management. Yet, many projects are derailed by poor specifications. Ambiguously written requirements, insufficient customer interviews, missing stakeholders, and communication failures can all lead to budget overruns, missed milestones, disappointed users and software that simply doesn’t do what it should. This workshop covers the basics your team will need to improve the requirements process: how to create unambiguous requirements, how to explore the solution space, how to discover stakeholders and users, and how to document functions, attributes, constraints, and expectations. With many hands-on exercises, you’ll go back to work with methods that you can use immediately to start improving the requirements process in your company.

How to Improve Mobile User Experience

UX has increased importance on the mobile platform, especially with subscription business models where users can switch applications in a heartbeat. So you've got only thirty seconds for your users to understand how to use your app and get value. How do you do that? A UX that works for and “talks” to the user, understanding what they want and when. To build a deeper understanding of how to improve your own app’s UX, Phil provides examples with key principles for design, test and evaluation of UX. Learn how to incorporate context and trust into your UX and keep your users coming back.

Improve Your Testing Process in Agile Projects

In this session, speaker delve into the scenario where testing fits into the agile project. Participants will learn about principal agile concepts and how these impact testing, understanding the tester's responsibilities and required skills set for testing in agile projects.

Speaker will also touch on how to approach test process improving in an agile project.

Improve Your Testing Process in Agile Projects

In this session, speaker delve into the scenario where testing fits into the agile project. Participants will learn about principal agile concepts and how these impact testing, understanding the tester's responsibilities and required skills set for testing in agile projects.

Speaker will also touch on how to approach test process improving in an agile project.

Building Blocks for Intelligence in Testing – a V&V Perspective

Verification & Validation (V&V) practices, regardless the degree of independence associated to it (i.e. regardless V&V or IVV ), involve multiple types of activities – reviews, testing, auditing etc. – and produce various reports and readings. Collectively, these artefacts contain substantial analytics (and intelligence) which contribute to improving your testing practice and make you a better tester.

In this session, Amir will share his vast experiences in leveraging such artefacts as building blocks for adding intelligence in testing.

Practical Application of Intelligence in Testing – a V&V Perspective

Intelligence derived from artefacts and analytics will only be beneficial if applied properly and appropriately. Unfortunately, there is no Hard and Fast Rules in ‘how’ and ‘where’ to apply such intelligence. This can only be learnt from experience.

In this session, Amir will share his experience in leveraging applying intelligence in testing.

12 Secrets of Effective Test Management - Strategy 

It is helpful to think of testing as less of a role and more of an activity that people undertake in their projects or organizations. Everyone tests, but some people specialiase and make career of it. In the same way, test management is an activity associated with testing. Whether you are the tester in a team or running all the testing in a 10,000 man-year programme, you have test management activities.

Many companies have decided that the role of test managers is no longer required. You might not have a dedicated test manager, but some critical test mangement activities need to be performed. Perhaps the tean jointly fulfil the role of a virtual test manager?

This tutorial sets out the key principles of these management activities and provide insights into how you can apply them in your own situation. Paul will sahre stories from his (and others') experience and identify the thinking processes you need to work through?

Part 1 will address the strategy portion:

  • Stakeholder Goals and Testing 

 • Defining Test Strategy

 • Modelling 

 • Risk 

 • Enough Testing 

 • Documentation 

12 Secrets of Effective Test Management  

It is hlepful to think of testing as less of a role and more of an activity that people undertake in their projects or organizations. Everyone tests, but some people specialiase and make career of it. In the same way, test management is an activity associated with testing. Whether you are the tester in a team or running all the testing in a 10,000 man-year programme, you have test management activities.

Many companies have decided that the role of test managers is no longer required. You might not have a dedicated test manager, but some critical test mangement activities need to be performed. Perhaps the tean jointly fulfil the role of a virtual test manager?

This tutorial sets out the key principles of these management activities and provide insights into how you can apply them in your own situation. Paul will sahre stories from his (and others') experience and identify the thinking processes you need to work through?

Part 2 will address the strategy portion: 

 • Planning

 • Execution

 • Testing as a Team

 • Service Testing

 • Infrastructure and Tools

 • Managing Your Career​

Test Intelligence and Data to Drive the Asian Mobile/IoT Future

How many of you have smart devices? How many computers do you use in a day? What are the concepts and skills we should be working on to improve test intelligence for such devices?

Software and computers are everywhere, being used by many people, and many times users are not even of aware they are using computers. We do however notice when these systems fail us or have bugs. These systems will always have bugs, even when we improve such systems with intelligence and drive them with data analytics.

So, working with computing systems and software testing are good career options. The new intelligent test jobs will come slowly but worse, the nature of these jobs will likely change over time. Additionally, there will be many jobs in the mobile, embedded and IoT system space. Jon Hagar will explore the basic engineering ideas of mobile/IoT systems with a tester focus. He will expand these basic ideas with hands-on practice in mobile/IoT devices using data analytics and intelligent test patterns.

ISO Standards, IV&V, and Test Planning for the AI Intelligent Future

Does AI in software mean that we will not need concepts such as standards, IV&V, and testing?

No. It is likely that many of the software systems we create will need these concepts and, in some cases, we may need new ideas. The hard part will be balancing machine learning with human thinking.

Jon Hagar will examine current “standards” for testing and IV&V. Then Jon will consider how these may need to change and be applied when AI and data analytics are incorporated into a project. To support these concepts, skilled and expert testers will be in demand worldwide

One definition of expert that Jon Hagar likes is someone that knows the rules but how to break the rules when they must be broken. Breaking of some rules will be needed for the AI-driven projects of the future. Jon will examine some of the rules of standards and then work exercises where we “break” things.

Risk Management in Agile

Software componentization has made software more unpredictable because unforeseen conditions can cause components to interact in ways we hadn’t imagined. Combined with greater complexity, increased user expectations and our desire to use Agile with ever increasing velocity, uncertainties and risks need to be actively managed. Classic risk management identifies risks and prioritizes them to determine impact to the project, but how does that differ in an agile project? Agile is designed to handle uncertainty in requirements as new features are requested and priorities shift. But what about the other uncertainties outside of requirements changes? Understanding those risks even before the project gets started, and those that can possibly derail the project after delivery are critical. Phil provides insight into the uncertainties and risks involved in agile software projects and supplements classic risk management approaches with how and when to apply within an agile process. Through a hands-on approach, Phil outlines risk management practices at each stage of the agile process. Take home pragmatic knowledge to identify and mitigate risks with the goal of focusing teams on the right things at the right time and producing higher quality software.

Software Quality Metrics the Agile Way (or Not)

When implementing software quality metrics, we need to first understand the purpose of the metric and who will be using it. Will the metric be used for measuring people, the process, illustrate the level of quality in software products, or drive towards a specific objective? QA managers typically want to deliver productivity metrics while management may want to see metrics that support customer or user satisfaction or cost related (ROI) initiatives. With agile development methods, we often lose sight that our primary objective is the same, quality. Many organizations are myopic on the primary objective of velocity. However, velocity means little without quality. Define quality for your organization with an agile looking glass with intermediate metrics that lead to both quality and velocity. Learn to develop and implement software quality metrics with actions toward improving on these two primary objectives.

Soft Skills for Testers and Test Managers

Soft skills are getting more and more important. It's an "old" subject but still very relevant. In this session speakers will cover issues of Appreciating the significance of soft skills in today's projects and principal techniques to apply. Not the least, speakers will also dive into the question: What can go wrong when soft skills are not applied?

Model-Based Testing in a Nutshell

In addition to going through the concept of MBT, the speaker will also elaborate on benefits of using MBT (adding "intelligence" to your testing approach), basics of modelling and how this is useful for testing. He will also address the subjects of tools support MBT the ‘Dos’ and ‘Don’ts in applying MBT

Intelligence-driven risk-based Testing

In risk-based testing, it is crucial to get the prioritization of test cases correct, A thorough risk assessment process or methodology is required. While key business drivers remain as the foundation for critical testing decisions, analytics derived from historical data have been widely used to enhance the risk assessments. With enriched analytics, intelligence gets into play and this is elevating testing to a higher level.

Learn more about it in this session.

Intelligence-driven Software Defect Prevention

As the age-old saying goes, “prevention is better than cure”. The same principle applies in good Software Quality Assurance (SQA) practice. Existing defect prevention strategies are largely based on historical data and lessons learnt from past projects. With enriched analytics, intelligence gets into play.

Coming from a TMMi Level 5-certified organisation where defect prevention is a key differentiator, Amir elaborate more on how intelligence can drive software defect prevention.

SCM and Intelligent Software Engineering

Can you name the critical skills now required for a degree in software engineering? Here are a few found in a quick Google search for software engineering: AI, machine learning, digital systems design, robotics, factory automation, pattern recognition, numerical methods, computer graphics, and database management just to name a few. While these skills will definitely help in the architecture and design of systems, they won’t assist in the process of creating credible software.

One of the critical skills in establishing the credibility of software is understanding and putting into practice SCM—no matter the type of development project or products or role on project. Laura will cover basic SCM skills; measures to ensure software credibility including standards; types of data to demonstrate software validity; collaborations on the Agile/Dev-Ops team with each other, their customers or management. Laura will also offer a discussion on how AI and tools could possibly change how SCM is practiced in the future.

SCM – Defining a Strategy for Product Integrity to Support Intelligence in Testing

How do you know that you can trust any software product you come in contact with (cars, trucks, airplanes, medical devices, entertainment systems, home appliances, etc.)? Does anyone on your team really know what the software is designed to do or how security for the product works? How can your customer trust that your team has produced a valid or reliable product in a known configuration for them?

Laura will provide training on 7 basic SCM concepts and how those SCM concepts support intelligence in testing and product integrity to include a discussion of requirements traceability, software product configuration control, data analysis, and test traceability. Participants should be prepared to ask and answer tough questions about software products and the many implications of poorly designed, developed, or tested software, and how SCM and intelligence in testing can assist in product integrity.

Behavior-driven Testing Technique (using the testing tool Cucumber)

Speaker will provide overview on behavior driven design/ development/ testing and how this technique is used in agile development, how this technique is related to requirements engineering. Specifically, the speaker will be using the Cucumber testing tool as part of the hands-on experience during the session.

Notes:

• No programming language background needed but participants need to bring a laptop (Windows/ Linux/ Mac).

• Cucumber installation will be provided as a Virtual Machine image. Participants need to install the free tool Virtual Box on their Laptop

Usability Testing – a Practical Perspective

This is an introductory workshop on Usability Testing. The session covers offers theories aspects of Usability Testing and hands-on application of key elements of the theories.

Notes:

• Participants would need a smart phone or tablet

• Participants would need to install a simple app used for the usability test

• No knowledge in UI/UX design or testing needed

Intelligent Test list Generator for IoT Systems based on Artificial Intelligence Techniques

Speaker will provide overview on how to create test list based on the combination between all IoT Systems inputs using Artificial Intelligence techniques. Specifically, the speaker will introduce some existing online combinatorial tools as well as give a softcopy of their developed tools. A tutorial of how to use the tool to generate the test list with minimum size that cover the most important combinations in IoT Systems.

Notes:

• No programming language background needed but participants need to bring a laptop (Windows/ Linux/ Mac).

• The tool will be provided as a JAR file. Participants need to install the free tool on their Laptops. JDK is recommended to be installed in the participants laptops to be easy to use the existing online tools.

Mining Intelligence from QS Certification Report

MSTB`s Quality Software Product Certification Scheme (QS) provides an independent, third-party certification of software products based against international standards. The reports that came along with the QS certification contains “intelligence” and additional insights on how to, other than improve the software product in question; improve quality assurance & testing process and even future development project(s).

We will show how you can maximize the benefits of going on the QS certification journey with us, apart from just getting the QS certificate itself!

First Step Of Test Architecture Design (Viewpoint Models)

For testing large-scale and/or mission-critical systems it is necessary to design test architecture though concepts of test architecture are not yet well established.

In this workshop, Nishi shares the fundamental concepts, principles and heuristics of test architecture design, while you can try to make and review parts of test architecture design in group works.

In this morning session, participants will try to make test viewpoint models, which contain what to test. Since a test viewpoint is abstract or simple high-level test case, the test viewpoint model enabled test engineers to stretch their imagination for test design.

First Step Of Test Architecture Design (Test Container Models)

For testing large-scale and/or mission-critical systems it is necessary to design test architecture though concepts of test architecture are not yet well established. 

In this workshop, Nishi shares the fundamental concepts, principles and heuristics of test architecture design, while you can try to make and review parts of test architecture design in group works.

In the afternoon session, participants will try to make test container models, which describe overview of test design from various test levels and test types. SInce the test container model can visualize relationships among various test levels and test types, it helps test engineers to make proper test plans and strategies.

What is Functional Size Measurement (FSM) how can it help you be a better tester?

This is an introduction to FSM (FUnctional Size Measurement, also known as Function Points) for testers. Among others, Carol will cover: 

 • What is FSM 

 • How can it be used (estimating, communication, unit pricing, project management, test coverage, requirements management) 

 • What gets measured in FSM and what DOES not 

 • Why testers need to know about FSM

Just Enough Function Points

An introduction that covers how and what to count with Function Points.

Among others, Carol will cover:

 • Basic terminology

 • What counts

 • How to count - easy examples

 • Short case study

 • Enough Testing