Test Strategies, Analytical Testing & Beyond

Software will never be defect free. At the same time the importance of software to the society, and it’s complexity are growing at a high rate. Testing is and will always remain an essential part of software development, but finds it hard to keep up with these ever growing challenges. Just applying traditional testing practices in many cases in not enough anymore. This keynote will discuss the various test strategies that can be applied, and how they can assists the test professional. Special focus will be given to analytical testing. What is analytical testing and how can it be applied in today’s challenging projects? How does risk-based testing relate to analytical testing? These and many more questions will be dealt with during this opening keynote setting the scene for SOFTECAsia 2018.

From Testing to Cognitive QA

The digital shift is disrupting our current IT landscapes as well as our business models. As a result, we need to rewire our IT systems, redesign our process and rethink how we approach our customers and ensure satisfied customers. Applying Analytics and AI in software development is booming. Decades of research in analytics, visualizations, statistics, artificial intelligence, etc. has generated a large number of powerful methods for learning from data. This presentation addresses the challenges and constraints imposed by the digital shift as well as the opportunities it brings for testing and Quality Assurance especially considering Analytics. It presents the concept of “Cognitive QA” - an intelligent approach leveraging self-learning and analytical technologies to allow Predictive QA Dashboards, Smart QA Analytics, Intelligent QA Automation and Cognitive QA Platforms. The presentation will demonstrate how to apply this concept and will investigate what this means for us as testers and quality professionals.

How Analytics Can Drive Software Test Architectures and Advanced Support Environments

Computers and software drive much of the world now, and testing has become critical to the success of any product. However, too often we forget the test environments and architecture to adequately ensure the quality of the software. The scope and size of some test architectures and associated support environments imposes problems for testers. In fact, many software systems are only tested in isolation and not assessed within a broader system architecture environment.

This presentation defines current and future cyber systems, the associated challenges/risks, the impact from analytics testing, how to assess the analytics information and then the implications to testing. After providing definitions and context, Jon presents examples of useful existing large-scale test architectures and environments that can be driven by data analytics including simple analysis methods and more complex concepts such as computer learning. Finally, Jon considers possible future solutions for IoT cyber-physical test architectures and environments with implications for the future of IoT testing.

Positioning Malaysian IVV Industry in the Global Map - Public Sector Spearheading the Way

The track records for ICT projects in the Malaysian Public Sector has not been flattering. There had been cases about wastages and leakages, variation orders, extension of time, et cetera. Often, agencies issuing tenders without quantifying exactly what they want while bidders make winning offers without first understanding what exactly wanted by the tenderer. Time and budget overruns are not uncommon with some projects degraded into Projek Sakit (ailing projects), if not outright failure.

The scenario is changing. MAMPU, the central agency which evaluates proposals and monitor implementation of ICT projects in the Malaysian public sector, is charging ahead with the institutionalisation of Software Quality Assurance (SQA) and software testing for the sector.

The IV&V Handbook for the Malaysian Public Sector, first released in 2016, has provided detailed guidelines for quality-centric management of ICT project procurement and implementation. It has since been revised and enhanced.

Through Independent Verification and Validation (IV&V) approach, MAMPU is instilling mechanisms to ensure quality of software/systems acquired/developed for the public sector by systematically measuring key elements of a projects such as schedules, deliverables, resources and quality of services.

Speaker will share pertinent point of this initiative that is envisioned to propel Malaysian public sector to higher levels of quality delivery of services to the public, enabled by quality of software and systems deployed, and at the same time drive the growth of Malaysia’s IVV industry.

Testing is Constant Learning

This keynote will focus on Tom's career in software development, simulation, testing, automation and management. From methods in the 1970's to today's topics of data analytics, machine learning and artificial intelligence. Tom will include an overview of his career and life lessons that helped him be a better tester, help enhance government acquisition policy and achieve success.

Combining Agile Principles with Data Analysis for Test Teams

Are you working on an agile team? Is your project really agile? Are you on a team that wants to become agile or more agile? Is agile a "right fit" for your project?

Your data--the data used every day on project, should illustrate the levels of dedication to agile principles. Laura will walk participants through the Agile Manifesto and the Agile Principles as they relate to testing, quality assurance and data analysis, highlight the skills necessary to be a successful agile tester, and then examine the data used on an agile project. This session will offer case studies in agile environments.

Taking the Pulse of Testing World - Highlights of World Quality Report

Capgemini’s World Quality Report (WQR) is a global report analysing application quality and testing trends across multiple industries in 32 countries. This ninth edition of the report illustrates the impact of trends like Internet of Things, where organizations seek to disrupt with digital at an ever-faster pace when it comes to Quality Assurance (QA) and Testing function, which is increasingly required to transform into a business enabler. This year’s research was conducted among 1,600 CIOs and IT and testing leaders across the globe. It reveals a shifting set of primary objectives for the QA and Testing function compared with past years, but also reveals a return to the main objective of QA & Testing—to improve the overall quality of software, products, and processes. This presentation presents some of the key findings and recommendations and addresses the role of Analytics in Testing in relation to the theme of the conference.

What is Analytics Testing Anyways?

A quick search of the term “analytics testing” yields many links on an internet search engine. A review of some these links detail that the phrase is over loaded and address many areas including hardware, chemical, web, and deep learning systems. Jon Hagar explores the phrase, its history, the different usages, provides some software definitions which might lead to an international standard, and then thinks out side of the “box” to the other kinds of systems that might use and benefit from Analytics Testing. Jon considers different phases of testing that might benefit from Analytics including development, production, and operations. Further, since much of the literature seems to focus on Web and Transaction Systems that use Analytics, Jon presents concrete case studies of other software environments where Analytics Testing should be considered. Attendees come away with a better understanding of Analytics testing, how to do it, and where it can be of benefit to companies, ops teams, and governments.

TMMI in context - ISO 29119 & Agile​

The Test Maturity Model integration (TMMi) has a rapidly growing uptake and is now the world leading model for test process improvement. It’s growing popularity is amongst other based upon it being the only independent test process improvement model, and the simple presentation of maturity levels that it provides. But how does the TMMi stand up to the ISO 29119 testing standard? The TMMi and the ISO 29119 will be compared based on their respective content, and also the assessment processes and possible certification for both will dealt with as part of this comparison. The mistaken belief is that the TMMi and Agile approaches are at odds. Agile approaches and TMMi can not only coexist, but successfully integrate to bring substantial benefits to both Agile and traditional development and testing organizations. This session will show with examples that TMMi and Agile methods can effectively work together.

Leveraging Data Analytics for Financial Business Intelligence System​

Jean-Paul will share experience and lessons-learnt from his years of practicing and leading software testing projects in banking/financial services environment.

Quantifying Confidence Using Metrics – an IV&V Perspective

Independent Verification and Validation (IV&V) practices involve multiple types of activities – review, testing, auditing etc – and produce various reports and readings.

• How do we review/analyse these reports and readings leveraging measurements and metrics?

• What are the tell-tale signs that the product/project is not in the pink of health?

• What does it take to ensure stakeholders have high confidence in software products and services offered?

• What are the contributing elements in evaluation of the level of confidence of project execution, and product to be deployed and subsequently maintained?
In this session, Amir will share his vast experiences accumulated over years of leading and consulting on IV&V and SQA projects

Practical Risk-Based Testing

Although most projects implicitly use some kind of risk-based approach for prioritizing testing activities, critical testing decisions should be based on a thorough product risk assessment process with key business drivers as the foundation. PRISMA (PRoduct RISk MAnagement), a practical method for performing systematic product risk assessments is presented. It is explained how to carry out risk identification and analyzes, and how to use the outcome to select the best test approach. Learn how to use PRISMA in both traditional and agile projects, including creating an agile sprint test plan. Practical experiences are shared, problems overcome and results observed employing product risk assessments. Learn how to optimize your test effort by including product risk assessment in your testing practices. Develop a rigorous approach to inform project stakeholders about important testing decisions and provide clear visibility on product risk status.

Requirements Engineering for Testers

Testers use requirements as the basis of test cases, review them for testability, and often participate in general requirements reviews or inspections. Unfortunately, many testers have little knowledge or skills in requirements engineering. What level of quality and detail is realistic to expect in requirements documents? What does testability really mean? How can testers help improve requirements? These questions and more will be answered while helping the attendee to develop skills in requirements engineering. Requirements issues and solutions are illustrated with practical case studies, and hands-on classroom exercises in finding, specifying and evaluating requirements are conducted. Walk through the requirements process from a tester’s viewpoint to learn what you can should contribute to requirements quality. At the end attendees will collaboratively create a set of “Golden rules” that every tester needs to successfully participate in the requirements engineering process.

Applying a Test Strategy

As a test practitioner you need to identify the main challenges of the test project and find out how to deal with these. This is the essence of a test strategy that will lay the foundation for your test planning, architecture and budget. This tutorial explains the role and use of test strategies. It demonstrates how to construct a pragmatic strategy that successfully embodies the goal of the project, the priorities and the associated risks. The difficult part in many projects resides in putting the strategy to practice. What happens when reality deviates from our carefully thought-out plan? In a perfect world you need well-defined, unambiguous and testable requirements before you start testing. You also need adequate resources and sufficient time to complete all your planned tests. In reality you often have to get the job done with whatever means you’ve got: poor or missing requirements, scarce recourses and very tight schedules. Another challenge is how to approach test prioritization and decision making. There is always a trade-off in relation to scope, time and cost during the test process. In this workshop, you'll explore some of the typical challenges and discuss what "tactics" to apply to resolve them following a risk-based approach.
Key points

• Elements of a test strategy - how to construct a pragmatic and adaptive strategy.

• Dealing with requirements and identifying system quality attributes.

• How to apply a risk-based testing approach

Pragmatic Test Metrics

Successful test management implies making complex decisions. These decisions need to be based on solid, quantitative data and well-calculated risk. This information can be supported by measurements of products, processes and resources. This workshop addresses metrics and how to establish a set of metrics to support various aspects of development and testing such as estimating, planning, predicting and monitoring. Learn how to collect, analyse, present and communicate metrics effectively. Common metrics are described in the field of product quality, software complexity test progress and defect trending.
Using practical examples and exercises, the workshop illustrates how the metrics can be utilised for:

• Assessing the state and quality of the system under test and determine when you can stop testing

• Establishing a “test dashboard” to manage the test effort and focus on areas that deserve special attention. Use indicators giving early warnings of problems.

• Metrics and analytics – how to make our metrics smarter using analytics

• Improving the process.
The tutorial will also address metrics in the context of agile methodologies, which metrics are “agile” and which are problematic when shifting from traditional to agile testing and will enlighten on how to draw on Analytics to improve our data basis and sharpen our metrics.

Using Data Analytics to Drive Testing Critical Mobile, Embedded and IoT Devices

The world is rapidly moving to large-scale deployment of real-time cyber-physical systems made up of the Internet of Things (IoT) and embedded software often supported by mobile devices. Cyber-physical systems merge software, hardware, and operations to interact with the real physical world. These systems require specialised testing using traditional test patterns as well as modelling approaches that are best driven by data analytics. The reasons that specialised testing is needed include limited human user interface(s) to support testing, short performance time cycles, unique hardware interfaces, sophisticated communications, short lifecycles, system-of-systems integration, and numerous functional or non-functional requirements.

The workshop will begin by examining how to use data analytics based on error profiles to determine software risk areas and test patterns for testing Mobile/Embedded/IoT systems. Jon examines how teams can leverage data analytics to improve testing, thus speeding high-quality products to market. Jon explains how to mine and organise the data using concepts such as mind maps, taxonomies, and analytics testing using deep learning mathematical filtering. Students will get to practice analysing and organising data, so they can continue their analysis with data from their projects when they return to their work environments. Jon then examines real-world test patterns including UML models that are based on the data analytics testing to impact product quality.

Test patterns are in the tradition of James Whittaker’s book series “How to Break Software” which uses attacks to support test exploration. Jon extends the testing “attack” and “tour” patterns to Mobile/Embedded/IoT software systems. Test attack and tour patterns are similar to software design patterns but aimed at providing information about the software under test. Jon applies patterns with analytics testing to help business deliver quality in the complex space of mobile-embedded-IoT systems. Students learn how the emerging concepts of patterns, analytics, and cyber physical systems pose opportunities to testers and their organization.

Using Data Analytics to Drive Testing Critical Mobile, Embedded and IoT Devices

The world is rapidly moving to large-scale deployment of real-time cyber-physical systems made up of the Internet of Things (IoT) and embedded software often supported by mobile devices. Cyber-physical systems merge software, hardware, and operations to interact with the real physical world. These systems require specialised testing using traditional test patterns as well as modelling approaches that are best driven by data analytics. The reasons that specialised testing is needed include limited human user interface(s) to support testing, short performance time cycles, unique hardware interfaces, sophisticated communications, short lifecycles, system-of-systems integration, and numerous functional or non-functional requirements.

The workshop will begin by examining how to use data analytics based on error profiles to determine software risk areas and test patterns for testing Mobile/Embedded/IoT systems. Jon examines how teams can leverage data analytics to improve testing, thus speeding high-quality products to market. Jon explains how to mine and organise the data using concepts such as mind maps, taxonomies, and analytics testing using deep learning mathematical filtering. Students will get to practice analysing and organising data, so they can continue their analysis with data from their projects when they return to their work environments. Jon then examines real-world test patterns including UML models that are based on the data analytics testing to impact product quality.

Test patterns are in the tradition of James Whittaker’s book series “How to Break Software” which uses attacks to support test exploration. Jon extends the testing “attack” and “tour” patterns to Mobile/Embedded/IoT software systems. Test attack and tour patterns are similar to software design patterns but aimed at providing information about the software under test. Jon applies patterns with analytics testing to help business deliver quality in the complex space of mobile-embedded-IoT systems. Students learn how the emerging concepts of patterns, analytics, and cyber physical systems pose opportunities to testers and their organization.

How to Fit into Agile Projects as Tester or Requirements Engineer/Business Analyst

Agile software development especially Scrum as an agile framework becomes more and more popular in Malaysia to organize software projects. How does the traditional role of a tester and requirements engineer/business analyst fit into agile projects?

The tutorial will answer this question and will provide methods and techniques for testers and requirements engineers/business analysts to be part of an agile cross-functional team.

Agile beginners are very welcome. The tutorial will give an overview on the Agile framework Scrum and Kanbaan and the mindset behind agile software development to bring everyone on the same page. We will look into different team setups for agile project, see which ‘traditional’ methods benefit an agile project, and which agile methods are important to add to your personal toolbox. The tutorial will also include exercises to experience some of the methods hands-on.

Qualitative & Quantitative User and Usability Testing – a Career Path from Testing to UX

User Experience (UX) design has gained a lot of prominence over the last years not only in companies like Google or Apple. It led to the new role of UX Engineer with responsibilities far beyond UI design. An import part in UX is user and usability testing both using qualitative and quantitative methods. These test methods allow testing an application already in early prototype state which not only improves the user experience but also reduces the overall project budget.

Coming from the traditional role of a tester, user and usability testing offer a potential new career path into UX design. It is not necessary to have any knowledge about UX. The tutorial will provide an overview of the field of UX design and its methods and techniques with a focus on user and usability testing. Hands-on exercises during the tutorial will offer a direct experience of some of the methods. A list of available online resources, training courses, and possible UX certifications will also be discussed.

Round Table Workshop: Improving SQA, SCM and Testing Processes Using Data Analytics

Software Quality Assurance (SQA), Configuration Management (SCM) and Testing are critical software development and Information Technology (IT) support functions. Test teams often establish one or more of these activities while gathering quantitative data from these functions, but then do not use analysis to improve process activities. A fundamental concept in ideals like total quality management (TQM) and statistical process control (SPC) improvement is understanding the data and then, how to modify activities or functions to become more agile or lean. This session will begin with a short examination of differing types of data and respective analysis, then move on to a discussion of how to use data, SPC, and analytics, to work on real-world problems and class exercises. Participants are requested to bring 1 typed sheet describing their project and their data to be used in conjunction with small team exercises. The focus will be on helping attendees move forward in the real world by identifying data and analytics that can be used the next day at work. Sharing of ideas and concepts will be encouraged using:

• Analysis of data to aide decision making

• Organized brainstorming to help understand the data and what changes could/might be expected based on the analysis

• Selling the ideas with data to the team which could imply a certification, a compliance, following a specific method, etc.

• Data-driven quality and SCM to identify a "real story" or trend

Test Automation: Stories of the Good, the Bad and the Lessons Learned

This tutorial is intended to provide test automation lessons learned from the past and present that will help attendees make information-based decisions about how and when to automation testing.

The tutorial will start out with some history and move into a discussion of attendee’s experiences and desires for test automation. Attendees will learn some practical approaches to test automation methodologies that have proven to be very successful.

Lastly, will be a review how data analytics from Software Development, Software Quality Assurance (SQA), Software Configuration Management (SCM) and Testing processes and methods can be used to improve test automation.

DevOps and Software Testing Metrics in Financial Service Sector

Jean-Paul will share experience and lessons-learnt from his years of practicing and leading software testing projects in banking/financial services environment.

DevOps and Software Testing Metrics in Financial Service Sector

Jean-Paul will share experience and lessons-learnt from his years of practicing and leading software testing projects in banking/financial services environment.