Talk at DesignOps Global Conference 2019 , Manchester, England, 30-31st May 2019

Topic: [TBD – On  DesOps /DesignOps ]
Date: 30/31 May 2019
Venue:  Manchester, England

Super excited to be part of DesignOps Global Conference (www.designops-conference.com) to share ideas with some great minds on hashtagdesignops hashtagdesops hashtagdesign and the future of hashtagdesign from the lens of hashtagopenorg culture and hashtagdesignthinking being applied as the way of life to improve and optimize the operations in the organizations to deliver the “wow” experience, through best utilizing the hashtagprocess and hashtagecosystems and hashtagtechnology in context. Thanks, Peter Fossick. Looking forward to a series of engaging sessions for every hashtagdesigner / hashtagleader.

 

About the Conference:

The DesignOps Global Conference is for design leaders, developers, practitioners, product managers, service innovators and business leaders that are defining the way we design and develop new products and services. Join us Manchester for the DesignOps Global Conference 2019 Agile and Beyond – New Frontiers in Design
30 + 31 May, 2019

The themes for this year’s DesignOps Global Conference are:

  • DAY 1
    • Theme 1: DesignOps and the impact of design
    • Theme 2: Collaborating at speed and scale
  • DAY 2
    • Theme 3: Developing new cultures – developing new organisations
    • Theme 4: DesignOps in the era of AI and cognitive computing

More details of the event and to get the pass visit https://designops-conference.com/

 

 

 

 

 

 

 

 

[Workshop Deck] PART-2: Applying DesOps in Your Enterprise

[This is the SECOND part of the half-day workshop titled “Applying DesOps in Your Enterprise” I conducted at the “UX India Conference 2018” on 4th October, 2018, Bengaluru. This part focused on ‘Design System’ aspect of DesOps, and tried to address the challenges that designer face when they collaborate using around either two or more design systems. Neither of the design system as of today talk to each other in the same way, nor is the fact that that none of the design systems organize and define their patterns in the same structure.

This part of the work shop attempts to solve that issue through introducing the concept of “CDN based Open Design Eco-System” (* CDN – Content Delivery Network) that is modelled on a Nuclear Design Model which can be scaled up and be made semantic, so that the automation of design process can be made possible. In the workshop we also see an implementation or a proof of the concept for the web-design and development domain using HTML, CSS and Javascript. This uses CSS3 variables to implement an extensible framework based on the Nuclear Design Model.

The code can be downloaded from https://github.com/OpenDesignSystem/ODS-CDN .

You can explore more at

http://desops.io and http://opendesignsystem.org/portal/nuclear-design/

Workshop at UX India International Conference 2018 : Applying DesOps in Your Enterprise

]

 

 

You can explore more here:

https://www.linkedin.com/pulse/open-design-system-adding-semantics-part-3-heredity-samir-dash/

‘Semantic Design System : Redefining Design Systems for DesOps’ https://www.slideshare.net/MobileWish/semantic-design-system-redefining-design-systems-for-desops-v10-1sep-2018

Nuclear Design: https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-2-nuclear-model-samir-dash/

Open Design System Ontology: https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-3-open-ontology-samir-dash/?

 

 

(c) Samir Dash, 2018. All rights reserved. This content including the images are licensed under a Creative Commons Attribution 4.0 International license

 

 

Open Design System – Adding Semantics to Design System (Part-3): Heredity and Inheritance of Attributes

The word “evolution” was first found to be used around 1647 which was more popularised by the Naturalists towards the end of 18th century who upheld the age old pre-socratic Greek philosopher Anaximander’s view that humans must have evolved from an animal and this evolution must have sprung from sea. From Darwin’s theory of evolution, we see two important characteristics :

  1. In the eco-system the individuals of the same species are likely to differ in their measurable characteristics
  2. Such kind of variations are inherited (heredity).

For any complex and growing (evolving ) echo-system when we try to find the model to explain the behaviour of evolution, these two characteristics helps build a model that can provide meaningful reasoning behind the complex variations that exist.

Darwin’s The Desent of Man was more focused on hypothesis that variations are transmitted from parent to the off-spring, though it was a few years after established.

During mid 19th century, Gregor Mendel’s extensive pea-plant breeding explained and came up with a meaningful model.

This model was able to interpret and explain Darwin’s hypothesis around hereditywhere it toys around the questions such as —

“is it possible that an animal having, for instance, the structure and habits of a bat, could have been formed by the modification of some animal with wholly different habits? Can we believe that natural selection could produce, on the one hand, organs of trifling importance, such as the tail of a giraffe, which serves as a fly-flapper, and, on the other hand, organs of such wonderful structure, as the eye, of which we hardly as yet fully understand the inimitable perfection?”

The natural life-forms and their associated complexity of variations, basically, was well explained by the model of heredity. The moment we envision a design-system that can explain the variations in life-forms, it would be no surprise to be in similar lines of what Darwin had deduced. In the similar lens, when we take the case of any design-system, we see similar attributes. The variations of any entity can be explained with inheritance of attributes from parents.

Irrespective of the typical design system maturity level, in order to make a design system scalable, it should have this very characteristic of having the model to define variations and inheritance.

In typical sense of “design system”, as we refer in our day jobs in techno-design industry revolving around software application and related eco-systems, we face the use-cases that are the tip of the iceberg of the operational problems we face in a design process. The activities like, the categorization of UI patterns and mapping them to the user interface needs, manual export of the semantics while different teams work on different tool-chains especially in open and collaborative design scenarios along with long hours of religious debates” of the utilitarian values of widgets proposed to be used against the variants that are in the implementations, are some of such examples, which has become the part of the job, where unaccountable time, effort and of course organization money become under-utilised.

Design System help generalize the the diversified aspect of patterns we use in design. But, the offshoots of the design systems across the industry, to cater to the need of the individual organisation, who are developing them, are not solving the issue of bringing a normalization, rather they are making it more distributed. There are no two design systems that are equal in structure or in the approach they follow to define the patterns in them.

So the “design system”s which are supposed to be a “common-language” for their respective design process that every team member can adhere to in order to produce a consistent design, do not have a “common language” among themselves.

Along with that, none of the design systems, as of today has a model that can help explain any other design system. Even, the basic smallest pattern or the building button, they define, does not explain the variations we see across cross domain, organization and technological context.

A simple case of a “button” pattern, does not seem anymore simple, where we try to connect the dots between the variation we see regarding that.

For example how is a button different from that of a toggle button, or a text-link or a simple radio-button? Even a simple hyperlink with some background color and cursor properties can cloak itself as a mental model of the user defining the concept of a ‘button’. Also the technological dimensions to variation can span beyond the technology used, platform it is rendering and the framework or language in play.

This is a perfect example even in such a narrow definition of a design system (i.e. a pattern library for UI or a basic brand-system or style guide), how it is fit case for the need for a model to define these variations and inherent relationships among the entities to explain in a logical way how these variations are linked to each other.

(Fig. Ref: https://www.linkedin.com/pulse/desops-prepare-today-future-design-samir-dash/)

When I think about the Design System, it’s not limited to the UI or brand library typically that we refer and try to define, rather, being a part of DesOps mindset, where every organization is a Design Organization (as every organization is in employing creativity in solving the problems for its customers through its products, services etc. ), it spans across and touches different and diversified domains, technological dimensions and information spaces.

(Fig. Ref: https://www.linkedin.com/pulse/desops-prepare-today-future-design-samir-dash/)

For example, a design system can be a knowledge system for system design meant for developers, technical architects, data scientists. Similarly, a design system can define the elementary components and practices for process design – e.g. the elementary blocks of a Design Thinking framework like that of IBM Design Thinking framework can be explained and built using such approach. Or the design system can also cater to the need of the intersection of specific industry need and technological domain at the micro level — e.g. a design system that can define the model for a color system needed for specific type LED monitor manufacturing industry. In all these scenarios, the task becomes critical to define a model that can stand for this “universe” what we refer as a “design system”, where we can branch out or fork and build the shared meaning that can span across all diverse disciplines, domains, technologies and methodologies.

(To be Continued to the next post)

(c) Samir Dash, 2018. All rights reserved. This content including the images are licensed under a Creative Commons Attribution 4.0 International license

Previous post of this series:

https://www.linkedin.com/pulse/open-design-system-adding-semantics-part-1-background-samir-dash/

https://www.linkedin.com/pulse/open-design-system-adding-semantics-part-2-roles-pattern-samir-dash/

You can also explore the following links:

‘Semantic Design System : Redefining Design Systems for DesOps’ https://www.slideshare.net/MobileWish/semantic-design-system-redefining-design-systems-for-desops-v10-1sep-2018

Nuclear Design: https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-2-nuclear-model-samir-dash/

Open Design System Ontology: https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-3-open-ontology-samir-dash/?

Open Design System – Adding Semantics to Design System (Part-2): Roles of Pattern Library in Design Automation

In search of a tool that can minimize the translations of values, in context to UI design space, I was toying with an idea of Dittoa simplified version of the tool that can look familiar to the different roles involved in the process and at the same time it will align with a process that is about having the same source files at any of the stages of the process and can align with any design system with easy configuration mechanism.

This approach of Ditto was mostly about the conversion to application code via using a controlled configurable UI pattern library serving as the base element for a dynamic IDE that would help to reduce the gaps between the roles.

The idea involved a ‘single source’ based process that can work with standard pattern-libraries/design-systems.

Earlier to this, back around 2013-14, I had experimented with conversion of design files (PSD/AI/PDF etc.) to style guide through prototypes like Specstra as an early attempt to bring automation into design process to reduce the manual intervention around drafting style guides.

This was another different kind of approach, similar to which there are many solutions are available including Adobe Extract and Avocado etc.

Sometime back, while conceptualising the BetaStudio that focused on envisioning the automation aspects to include the usability dimensions of application testing as a part of an automated eco-system more aligned to DesOps mindset, I was playing around OpenCV and its ports in JS that can have a different perspective than my earlier approach to define a set of design benchmarks (the 1st circle in the stages defined in below image) to compare against the models/metrics towards the later part of the stages.

But this time I wanted to explore apart from the two above said approaches, there can be any other practical approaches, which can help building a UI design-automation solution component for the flow. For this I came up with a conceptual workflow “Pattern-AI” that involved some image recognition through a live camera and matching that for potential UI patterns by processing the camera input using computer-vision technologies, as shown in the flow below:

Based on this my set up was very simple, good enough for a hello world type validation of the concept, having a webcam output, being processes through a Kinetic based web application running in Chrome, which tried to identify the matching pattern (a simple image, icon and button ) drawn on the paper to that with a predefined pattern array with those object names. Though I never went beyond just matching the name, it was obvious that as I was able to match a pattern, generating a predefined HTML-CSS code was a piece of cake post that.

This experimentation helped me to later see that possible variation on this line can also be a part of the high level vision of bringing design automation especially in UI design space, like a missing puzzle piece.

However, this also made it clear that whatever, the potential approach might be, finally what we need is a “consistent and scalable design system model” that is essential to ensure whatever is getting translated from the different types of source through any of these various approaches, as that is the point where all these information is getting mapped and the meaning of each pattern is formed.

The basic questions that arise from these were…

When the system gets a clue from the input/source design (or hand-drawn wireframe) as a dropdown, can it consistently map it to a target design system or a pattern-library or even a brand-system based widget-library consistently?

Also even assuming that it does able to identify the pattern, for example as a button , how it can select or define the target implementation code a in a consistent manner? In this example case, a button can be defined as a <input type=’button’ /> or <div id=”button”></div>or even <button></button> etc. ?

This was an eye-opener, as it was clear from this that, irrespective of the technology used, and irrespective of the methodology is used to generate the required information for design benchmark, without the essential “common language” among the design-systems in use, the translation of those information/artefacts might not make sense to the systems running the automation. And this is where the whole purpose of DesOps mindset will fail.

This actually hints on a problem statement about the gap that currently our design systems have — i.e. a consistent way to define the patterns (and note this is not just limited to UI focused design-system) that can be scalable, and coming up with some approach to bring “semantics” to the design-systems in focus.

We will dive more into it in next posts. Meanwhile you can explore these links:

Previous post of this series: https://www.linkedin.com/pulse/open-design-system-adding-semantics-part-1-background-samir-dash/

Nuclear Design: https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-2-nuclear-model-samir-dash/

Open Design System Ontology: https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-3-open-ontology-samir-dash/?

(c) Samir Dash, 2018. All rights reserved. This content including the images are licensed under a Creative Commons Attribution 4.0 International license

#nucleardesign #design #desops #designops #designsystem #opendesignsystem #odso

Open Design System – Adding Semantics to Design System (Part-1): Background

“Design System” plays a critical role in bringing a common language and consistency in experience across different products and brands of the organisation, along with fuelling a collaborative approach towards design, making it easier for different team members to contribute.

With observation of design systems, we can notice, almost all of them are having different structure and approach to define these, and almost none of them can be directly used in automation of design as they are not semantic in nature. Therefore most of these current approaches to design systems are not future-proof for tomorrow’s design operations (DesOps).

Take an example of a representation of information in a tabular form. In HTML, the desired table element does the basic thing work. But most of the real-life sites, utilize the responsive frameworks like Bootstrap, and on the course, they mostly use the framework versioned table component. For advance controlled display, some may use extended components based on the framework, e.g. Bootstrap Datagrid. In some cases due to the need for the two-way data binding and similar needs application might be running Angular JS driven grids. Some other might prefer using some niche components like Data Tables. In many cases, there are hybrid of niche components married to specific frameworks, for example, the Data Tables with Bootstrap 4.

This actually shows how diversified is the form and the usage of components or the UI patterns. When such things happen, the amalgamation of associated design systems takes place. The target design system attributes, directions are used to customize and in the process the guidelines based on which the external components, interactions, patterns and style are changed.

The power of design systems should lie in the fact that they can establish a common, publisher neutral platform integrating distributed computing as well as design eco-systems.

One of the barrier to wider adoption of SWS technology is the lack of tools for creating specifications that can help consumption as well as comprehension of the design systems at large by the systems without much direct human intervention.

The one of the major way out for such a future is using RDF or micro-data formats to define the ontologies through formats like OWL, RDFa etc.

The success of the Design Systems lies in their abilities, in becoming a part of the distributed brand as well as design system, where any of the components or aspects can be extensible. Also the sustainability of the designSystems lies in the fact that they need to be migrated to and converge with any other design system. In the future of design operations, its highly possible in the world of CI&CD, the design systems should be able to be translated from one brand system to another effortlessly without the single human-intervention.

So what’s the way to achieve it?

The solution lies in Semantic Design System. Semantic Design System, is the future. Like the Semantic web, the goal of the Design Systems is to translate from one form to the other and ensure the machine readability and comprehensibility drives the structure and how the design system is fleshed out.

This is also, critical for open design systems, as the typical effort behind such systems is to make them available as more common set of patterns that can be adopted, used and extended by any other brand system. But as we diving towards the world of hybrid design systems, where the aesthetic as well as interaction enablement through CSS, JS frameworks are founding pillars, it is essential to provide a commonly used language that can define the basic relationships among different entities of a design system. The way forward is a design system defined in terms of a shared language or an ontology.

An “Open Design System”, as I envision, is the one that addresses this issue, and focuses on building a design system, that is equally human-readable for collaboration and at the same time understood by the machine by being a “Semantic Design System” for the next generation of design operation with automation, machine learning and artificial intelligence. The talk also introduces, what I believe as “Nuclear Design”, a modelling approach (I will expand on this one in coming part of this series ) that helps to lay out a framework that is the foundation for building design systems with semantics.

The semantics of the ontology can be used by the machine learning or AI systems in a similar manner how it currently uses the data modelling using the graph databases. This opens up a door to the future for the design that can support DesOps (aka. DesignOps) in organisations.

The Graph Data Model

The semantic web uses the graph data model to store data, RDF is the format in which it is written.

(Fig. Source: http://www.linkeddatatools.com/introducing-rdf )

In traditional data bases there are some kind of important elements against which the relationship is defined. In traditional data bases there are some kind of important elements against which the relationship is defined. uses graph structures for semantic queries with nodes, edges and properties to represent and store data.

Welcome to Open Design System Ontology (ODSO)!

(Continued to next post. )

Meanwhile, you can explore the following links:

Nuclear Designhttps://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-2-nuclear-model-samir-dash/

Open Design System Ontology: https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-3-open-ontology-samir-dash/?

(c) Samir Dash, 2018. All rights reserved. This content including the images are licensed under a Creative Commons Attribution 4.0 International license

#nucleardesign #design #desops #designops #designsystem #opendesignsystem #odso

Semantic Design System (Part -3): The Open Design System Ontology

[NOTE: Check the part 1 at : https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-1-gaps-todays-samir-dash/ and part 2 at https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-2-nuclear-model-samir-dash/]

This originally part of a talk I gave at UX-India Pre-Conference Meetup at Symbiosis College, Bengaluru on 1st September 2018, titled: Semantic Design System — Redefining Design Systems for DesOps . The talk explored the existing implementations of design-systems and will be making an attempt to provide high-level approach to define design systems for the next-gen enterprises.

“Design System” plays a critical role in bringing a common language and consistency in experience across different products and brands of the organisation, along with fuelling a collaborative approach towards design, making it easier for different team members to contribute.

(Continued to next post. )

#nucleardesign #design #desops #designops #designsystem #opendesignsystem #odso

(c) Samir Dash, 2018. All rights reserved. This content including the images are licensed under a Creative Commons Attribution 4.0 International license

Semantic Design System (Part -2) : The Nuclear Design Model

[NOTE: Check the part 1 at : https://www.linkedin.com/pulse/photo-essay-semantic-design-system-part-1-gaps-todays-samir-dash/]

This originally part of a talk I gave at UX-India Pre-Conference Meetup at Symbiosis College, Bengaluru on 1st September 2018, titled: Semantic Design System — Redefining Design Systems for DesOps . The talk explored the existing implementations of design-systems and will be making an attempt to provide high-level approach to define design systems for the next-gen enterprises.

“Design System” plays a critical role in bringing a common language and consistency in experience across different products and brands of the organisation, along with fuelling a collaborative approach towards design, making it easier for different team members to contribute.

(Continued to next post. )

#nucleardesign #design #desops #designops #designsystem

(c) Samir Dash, 2018. All rights reserved. This content including the images are licensed under a Creative Commons Attribution 4.0 International license

Semantic Design System (Part -1) : The Gaps in Today’s Design Systems

This originally part of a talk I gave at UX-India Pre-Conference Meetup at Symbiosis College, Bengaluru on 1st September 2018, titled: Semantic Design System — Redefining Design Systems for DesOps . The talk explored the existing implementations of design-systems and will be making an attempt to provide high-level approach to define design systems for the next-gen enterprises.

“Design System” plays a critical role in bringing a common language and consistency in experience across different products and brands of the organisation, along with fuelling a collaborative approach towards design, making it easier for different team members to contribute.

With observation of design systems, we can notice, almost all of them are having different structure and approach to define these, and almost none of them can be directly used in automation of design as they are not semantic in nature. Therefore most of these current approaches to design systems are not future-proof for tomorrow’s design operations (DesOps). Here I coin the “Semantic Design System”, that address this issue, and focuses building a design system, that is equally human-readable and at the same time understood by the machine for the next generation of design operation with automation, machine learning and artificial intelligence. The talk also introduces, what Samir believes as “Nuclear Design”, an modelling approach that helps to lay out a framework that is the foundation for building design systems with semantics.

(Continued to next post. )

(c) Samir Dash, 2018. All rights reserved. This content including the images are licensed under a Creative Commons Attribution 4.0 International license

Download the Paper – “In Search of Truth: At the Crossroad of Critical Theory and Technology in DesOps World”

It was a great pleasure to be at UGC seminar in Department of English B.B.Autonomous Mahavidyalaya, Chandikhole on the topic ” In search of truth: At the crossroad of critical theory and technology in DesOps world” It was a very engaging session presided by principal Dr. Kedarnath Das, HOD English, Prof. Rajendra Padhi, Senior lecturers Jachindra Rout, Bimal Ch. Mallick, Dilagovinda Pratap, Nabajyoti Biswal. Dr. B.K.Nayak and the students from the department of English, Maths and Science.

Resource person Prof. Samir Dash, Bangalore. Sanjeev Behera, Psychologist, Presided by principal Dr. Kedarnath Das, HOD English, Prof. Rajendra Padhi, Senior lecturers Jachindra Rout, Bimal Ch. Mallick, Dilagovinda Pratap, Nabajyoti Biswal. Dr. B.K.Nayak.Image may contain: one or more people and text

 

Topic: In Search of Truth: At the Crossroad of Critical Theory and Technology in DesOps world
Date: 16 July 2018
Venue: Department of English, Baba Bhairabananda Autonomous Mahavidyalaya, Jajpur, Odisha (http://bbmchandikhole.org/)

 

Read it here: 
https://www.linkedin.com/pulse/search-truth-crossroad-critical-theory-technology-desops-samir-dash/

Downloadable Paper/Slide: 

https://www.slideshare.net/MobileWish/in-search-of-truth-at-the-crossroad-of-critical-theory-and-technology-in-desops-world-16-july-2018-bbautonomus-college-chandikhol

 

Abstract:
This UGC seminar session was an attempt to understand, from a non-traditional lens, the relevance of critical theory in context to today’s ever-changing technology space that is moving towards the Automation, Artificial Intelligence, Machine Learning and Distributed Computing, become much more important in history than ever, as it deals with softer aspects of human identity and socio-cultural dimension through communication or human expression.

This interactive session will have two major focus:

1. A brief overview of the critical theories from a diachronic lens that will be helping the students in grasping the fundamentals in a socio-cultural context.

2. A cross-discipline comparison with the modern design-driven practices in the software industry that would help the students understand the potential and opportunities in the real world scenario where these theories would help.

#DesOps #DevOps #design #designthinking #communication #criticaltheory #literature #englishliterature #englishlanguage

     

 

 

Beta Testing in the Ever Changing World of Automation

(Apart from some details about my prototype in Beta Testing, this also explores the interesting definitions from ISO regarding quality that brings “reviews in context” that fundamentally lauds usability philosophies. (Originally first published at RedHat Blog in January 2018)

Beta testing is fundamentally all about the testing of a product performed by real users in a real environment. There are many tags we use to refer to the testing of similar characteristics, such as User Acceptance Testing (UAT), Customer Acceptance Testing (CAT), Customer Validation, and Field Testing (more popular in Europe). Whichever tag we use for these testing cases, the basic components are more or less the same. To discover and fix potential issues, this involves the user and front-end user interface (UI) testing, as well as the user experience (UX) related testing. This always happens in the iteration of the software development lifecycle (SDLC) where the idea has transformed into design and has passed the development phases, while the unit and integration testing has already been completed.

The beta stage in the product lifecycle management (PLM) is the ideal opportunity to hear from the target market and to plan for the road ahead.

As we zoom into this phase of testing it has a board spectrum of ranges. On one side is the Front-End or UI related testing involving UI functionality (Cosmetic, UI level Interaction and Visual look and feel). At the other end is the User Experience (UX), including user testing involving more from A/B (split) testing, hypothesis, user behavior tracking and analysis, heat maps, user flows and segment preference study or exploratory testing and feedback loops.

Beta testing relies on the popular belief that goodness will prevail, which defines the typical tools that carry out such tests. Examples are the shortening of beta cycles, reducing the time investment, increasing tester participation, improving the feedback loop and visibility, etc.

The Importance of Beta Testing

If we dive deeper into the factors behind the existence of tools from different angles, we find two major reasons that advocate the importance of beta testing:

1. Left-right brain analogy, which points to the overlap between humans and technology.

The typical belief is that the left-hand side of the brain mostly processes the logical thinking, while the right-hand side is more about the emotional thinking. Based on this popular analogy, when we map the different aspects involved in different stages of SDLC for a digital product across a straight line from left to right (refer to the diagram below), we will notice the logical and more human-centered aspects are divided by an imaginary line from the center. We will also notice the gradual progression of the emotional index for the components from left to right.

When we map these to the beta testing phase, we notice these right-hand components are predominant. As users of the products (like humans), we are more emotionally connected to such aspects of the product, which are validated or verified in a beta test. This makes beta testing one of the most important testing phases in any SDLC.

Another interesting point to note is that when we look from the traditional software approach to define “criticality”, the areas that are tested during user acceptance testing (UAT / Beta), mostly fall into Class 3 and 4 type of criticality. But because these touch the core human aspects, they become more important. This Youtube video (Boy Receives Enchroma Glasses From Father) helps to illustrate the importance of technology touching the human emotions. It was posted by a popular brand of glasses to demonstrate how they can correct colorblindness in real time for the end user.

Interestingly, this is an aspect of “accessibility”, which is typically covered during a beta test. Considering this video, the aspect of accessibility naturally raises the question about what we can do for this father and son as a tester, as a developer, or as a designer? And when we look at the stats, we find that the number of people the accessibility impacts is huge. One in every five people is challenged by some kind of disability. And unfortunately, some reports indicate that a majority of websites do not conform to the World Wide Web contortion’s (W3C) accessibility guidelines (in 2011 almost 98% of websites failed the W3C’s accessibility guideline).

This helps to demonstrate the human angle, which advocates why beta testing is important to ensure these aspects are validated and verified to ensure the target user needs are completely met.

2. A second reason for advocating the importance of beta testing is evaluating it from the international standards perspective (ISO/IEC 9126-4). This helps to define the difference between usability and quality.

The International Standards Organization (ISO) has been focused on the standards around quality versus usability over time. In 1998 ISO identified efficiency, effectiveness and satisfaction as major attributes of usability. In 1999 a quality model was proposed, involving an approach to measure quality in terms of software quality and external factors. In 2001 the ISO/IEC 9126-4 standard suggested that the difference between usability and the quality in use is a matter of context of use. ISO/IEC 9126-4 also distinguished external quality versus internal quality and defined related metrics. Metrics for external quality can be obtained only by executing the software product in the system environment for which the product is intended.

This shows that without usability/human computer interaction (HCI) in the right context, the
quality process is incomplete. The context referred to here is fundamental to a beta test where you have real users in a real environment, thereby making the case of the beta test stronger.

Beta Testing Challenges

Now that we know why beta testing is so very critical, let’s explore the challenges that are involved with a beta stage.

Any time standards are included, including ISO/IEC 9126, most of these models are static and none of them accurately describe the relationship between phases in the product development cycle and appropriate usability measures at specific project milestones. Any standard also provides relatively few guidelines about how to interpret scores from specific usability metrics. And specific to usability as a quality factor, it is worth noting that usability is that aspect of quality where the metrics have to be interpreted.

When we look at popular beta-testing tools of today, the top beta testing tools leave the interpretation to the customer or to the end-user’s discretion. This highlights our number one challenge in beta testing, which is how to filter out pure perception from the actual and valid issues and concerns brought forward.

As most of the issues are related to user testing (split testing and front-end testing), there is no optimized single window solution that is smart enough to handle this in an effective manner. Real users in the real environment are handicapped to comprehend all the aspects of beta testing and properly react. It’s all a matter of perspective and all of them cannot be validated with real data from benchmarks or standards.

The World Quality Report in 2015-16 Edition indicated that expectations from Beta testing is changing dramatically. It hinted that the customers are looking for more product insights through a reliable way to test quality and functionality, along with the regular usability and user testing in real customer facing environment.

It’s not only the Beta Testing, but more user-demand is also impacted by the rising complexities and challenges due to accelerated changes in the technology, development, and delivery mechanisms. The 2017-18 World Quality Report states that the test environment and test data continue to be an achilles heel for QA and Testing.   The challenges with testing in agile development are also increasing. There is now a demand for automation, mobility, and ubiquity, along with smartness to be implemented in the software quality testing. Many believe that the analytics-based automation solutions would be the first step in transforming to smarter QA and smarter test automation. While this true for overall QA and testing, this is also true for Beta Testing, which deals with the functional aspect of the product.

Let’s see where we stand today. If we explore popular beta testing solutions, we will get a big vacuum in the area where we try to benchmark the user’s need for more functional aspects, along with the usability and user testing aspects. Also, you can notice in the diagram below that there is ample space to play around with the smart testing scenario with the use of cognitive, automation, and machine learning.

(Note: Above figure shows my subjective analysis of the competitive scenario.)

Basically, we lack “smartness” and proper “automation” in Beta Testing Solutions.

Apart from all these, there are some more challenges that we can notice if we start evaluating the user needs from the corresponding persona’s viewpoint. For example, even when assuming that the functional aspect is to be validated, the end-user or the customer may have an inability to recognize it. The product owner, customer, or end-user are part of the user segment who may not be aware of the nuts and bolts of the technology involved in the development of the product they are testing to sign it off. It’s like the classic example of a customer who is about to buy a second-hand car and inspects the vehicle before making the payment. In most of the cases, he is paying the money without being able to recognize “What’s inside the bonnet!”. This is the ultimate use-case that advocates to “empower the customer”.

So, how do we empower the end user or the customer? The tools should support that in a way so that the user can have his own peace of mind while validating the product. Unfortunately, many small tools which try to solve some of those little issues to empower the user are scattered (example: Google Chrome extension that helps to analyze CSS and creates the report or an on-screen ruler that the user can use to check alignment, etc.). The ground reality is that there is no single-window extension/widgets based solution available. Also, not all widgets are connected. And those which are available, not all are comprehensible to the customer/end-user as almost all of them are either developer or tester centric. They are not meant for the customer without any special skills!

When we look at the automation solutions in testing as part of much Continous Integration (CI) and Continuous Delivery (CD), they are engaged and effective in mostly “pre-beta” stage of SDLC. Also, they require specific skills to run them. With the focus on DevOps, in many cases, the CI-CD solutions are getting developed and integrated with the new age solutions looking at the rising complexities of technology stacks, languages, frameworks, etc.. But most of them are for the skilled dev or test specialists to use and execute them. And this is something that does not translate well when it comes to Beta testing where the end-user or the customer or the “real user in real environment” are involved.

Assuming we can have all these automation features enabled in BETA, it still points to another limitation in the existing solutions. It’s because the employment of automation brings in its own challenge of “information explosion”, where the end user needs to deal with the higher volume of data from automation. With so much information, the user will struggle to get a consolidated and meaningful insight of the specific product context.   So what do we need to solve these challenges? Here is my view — we need smart, connected, single window beta testing solutions with automation that can be comprehensible to the end-users in a real environment without the help of the geeks.

For sometime since a last few years, I have been exploring these aspects for the ideal beta testing solution and was working on the model and a proof of concept called “Beta Studio”, representing the ideal beta testing solution, which should have all these —  Beta-Testing that utilizes data from all stages of SDLC and PLM along with the standards + specs and user testing data to provide more meaningful insights to the customer.  Test real applications in real environments by the real users. Customer as well as end-user centric. Test soft aspects of the application — Usability, Accessibility, Cosmetic, etc..  Be smart enough to compare and analyze these soft aspects of the application against functional testing data.

Use machine-learning & cognitive to make the more meaningful recommendation and not just dump info about bugs and potential issues. Here is an indicative vision of Beta Studio:

Mostly this vision of the ideal beta testing solution touches upon all the aspects we just discussed. It also touches upon all the interaction points of the different personas; e.g. customer, end-user, developer, tester, product owner, project manager, support engineer, etc.. It should cover the entire Product Life Cycle and utilize automation along with the machine learning components such as Computer Vision (CV) and Natural Language Processing (NLP). Then gathering this information to be processed by the cognitive aspect to generate the desired insights about the product and recommendations. During this process, the system will involve data from standards and specs along with the design benchmark generated from the inputs at the design phase of the SDLC, so that meaningful insights can be generated.

In order to see this vision translated into reality, what do we need? The following diagram hints about the six steps we need to take.

  1. First we should create the design benchmark from the information at the design stage that can be used in comparing the product features against metrics based on this design benchmark.
  2. Then automate and perform manual tracking of the product during runtime in real time and then categorize and collate this data.
  3. This involves creating features to support the user feedback cycle and user testing aspects (exploratory, split testing capabilities).
  4. Collect all standards and specifications on different aspects — e.g. Web Content Accessibility Guideline (WCAG) Section 508, Web Accessibility Initiative Specs ARIA, Design Principles, W3C Compliance, JS Standards, CSS Standards & Grids, Code Optimization Metrics Error codes & Specs, Device Specific Guidelines (e.g. Apple Human Interface Guideline) etc.
  5. Build the critical components such as Computer Vision and Natural Language Processing units which would process all the data collected in all these stages and generate the desired metrics and inferences.
  6. Build the unit to generate the model to map the data and compare against the metrics.
  7. The final step is to build the cognitive unit that can compare the data and apply the correct models and metrics to carry out the filtering of the data and generate the insights which can be shared as actionable output to the end-user/customer.

While experimenting for BetaStudio, I have explored different aspects and built some bare bone POCs. For example, Specstra is a component that can help create Design benchmark from design files.

With Specstra I was trying to address the issues related to the cosmetic aspect or visual look and feel. Typically when it comes to cosmetic issues, more than 30% are non-functional and mostly cosmetic. There is no reliable solution that helps in benchmarking these kind of issues against specific standards. One third of the issues found during the beta / UAT stages of testing are mostly cosmetic and alignment issues, including category 3 and 4 types. And these happen mostly because the two persona’s involved (developer and designer) have their own boundaries defined by a mythical fine line.

When a developer is in focus, roughly 45% of them are not aware of all the design principles employed or the heuristics of UX to be employed. Similarly, half of the designers are not aware of more than half of the evolving technological solutions about design.

And in 70% of the of the projects, we do not get detailed design specs to benchmark with. Detailing out a spec for design comes with a cost and required skills. In more than two-thirds of the cases of development there is the absence of detailed design with specs. Many of the designs are not standardized and most of them do not have clear and detailed specs. Design is carried out by different tools so it is not always easy to have a centralized place where all the designs info is available for benchmarking.

Specstra comes in handy to solve this. It is an Automation POC that is more like a cloud-based Visual Design Style Guide Generator from the third party design source files. This is a case where the user would like to continue using his existing Design tools like Photoshop/Sketch or Illustrator, PDF etc..

You can view the video of the demo here:

View the demo of the BetaStudio POC here:

I understand that reaching the goal of an ideal beta testing solution might require effort and will likely evolve over time. Rest assured, the journey has started for all of us to connect and explore how to make it a reality.

Originally published at the RedHat Blog at https://developers.redhat.com/blog/2018/01/05/beta-testing-automation/