The “When/If – Then” Tool to Capture Hypothesis


(This article was originally published onApril 29, 2018 Linkedin Blog at https://www.linkedin.com/pulse/desops-next-wave-design-part-10-whenif-tool-capture-samir-dash/


As I mentioned in the previous article, the assumptions or the hypothesis can be made as a formalized statement of “if …. then” or “when … then” kind of format. To use ensure that this is captured correctly during collaborative sessions and within the design team, I have been using the same and some variation of the same format as a tool which I call “When/If – Then” Tool. This is a simple tool that can be very effective during collaborative sessions, to capture the assumptions that form the basis of why we believe that how the certain aspect of user pain-points and user needs should be translated into solutions using specific aspects.

Basically, this tool was helpful than the traditional elaborative templates to capture assumptions, as it was straightforward and can be used with a minimal explanation to a cross-functional crowd, a large portion of such population may not be accustomed towards the use the typical Tools and methodologies. The “When/If – Then” Tool is synonymous with the typical cause-and-effect kind of flow that is understood by and large and required very minimal explanation, and can be like any other UCD tools, can be constructed using simple post-its or printed formats.

Here is a post-it version of a barebone “When/If – Then” Tool:

Following is the basic format of this tool that has primarily 3 boxes — the left hand spanning column represents the persona or the target user (even segment given a larger scope from the market perspective) whereas the right-hand side boxes represent “When” or “If” part of the assumption and the lower box “Then” forms the outcome aspect of the assumption or hypothesis.

You can download a printable PDF of the template at http://desops.io/2018/05/07/when-if-then-tool-printable-version-for-a4-size-paper/

Basically, the above template helps you in completing an activity to frame your Hypothesis statement that are key to the the whole hypothesis-driven design (HDD) methodology. Now it is important that this tool can be applied to two-part blocks that can be used seeing as what you assumed versus what outcome have you got throughout experimentation.

A friend of mine asked me a valid question that it is fair to assume that HDD based approach as informs the assumption as a part of the course correction, whether it would replace the research from the process? My view on this is that the research (in the traditional definition like user and market research etc. ) has a key role in the process, as it is a mechanism to inform the stakeholder to create a vision. Technically a vision, itself is a set of hypotheses formulated by the stakeholder or the product manager. So it has a unique role in the process at the beginning of each cycle of the iterative process (let’s say to contribute to new requirements that flows down from the vision) and will stay. However, the hypothesis-driven design (HDD) helps in making a formalized approach and adds a mechanism to ensure the feedback-loop that informs these research or chunks-of-research that happens along the way in a structured and non-ambiguous way that enriches the research involving the additional validation process with evidence. Which acts as a catalyst to improve the efficiency of the research in a positive way. So essentially research is happening organically throughout the process in an on-going manner.

In the next article of the series, I will continue the exploration on the same topics. Keep in touch.

(c) 2018, Samir Dash. All rights reserved.

Lean UX : Another Agile UX?

Agile is the most popular and successful SDLC model as of today as it allows better scope in providing continuous and iterative refinement to the product. Historically when developers out of their frustrations with waterfall model turned to the growing Agile Movement to regain their control over the process, they found that “like its ancestors, Agile also didn’t take UX into account. Several of the Agile methods, such as Scrum and XP, recommended users sitting with the team during the development process, but that isn’t the same as design. Everyone who figured out how to get what they wanted from plugging UX into a phased waterfall approach was now struggling to work inside the Agile methods. The Agile principles, that focus more on communication and less on contracts, didn’t fit the status quo UX processes”.So efforts were made again to implement UX into Agile methods just like the way it was implemented into waterfall model . But it was not easy as , in waterfall model there are 2 things which helped implemented UX :

  1. The objectives of the project stays same from kickoff to the point where the finished product is launched.
  2. The designers created the set of design specifications as a contract which the developers had to implement into the final product.

And above two cannot be expected from Agile model as it is based on iterations and gradual exploration of what is best fit for the final product . On ejust simply cannot predict the final design from the start of the project. So many attempts were made to get the best agile SDLC practices that can incorporate the UX , before “Lean UX” was born.

As the above figure shows the documentation and guidelines are stripped to their bare minimum components, providing the minimum amount of information necessary to get started on implementation. Also Long detailed design cycles are discarded in favor of very short, iterative, low-fidelity cycles, with feedback coming from all members of the implementation team early and often.

Challenges with Agile model of SDLC to implement UX

There are several challenges in implementing UX in Agile model effectively and these challenges include:

  • Different approaches. Usability methodologies are centered on the user and holistic view of the user needs whereas agile methodologies take a broader view and focus on the stakeholder. Agile methods primarily focus on delivering working software early.
  • Different goals: Software engineers focus on the technical design, implementation the system where as UX practitioners focus on “developing systems so end-users can use them effectively”.
  • Organizational challenges: The agile methodologies focus on strategy where teams are self-organizing whereas UX focuses on a centralized UX groups within some organizations so that the needed practices, tools, and standards can be provided.
  • UX practitioners struggle to be heard: Many UX practitioners often complain that the results of their work are not considered in the design decisions and even if it is heard, there seems to be focus more on engineering decisions over the usability decisions.

Lean UX and Agile Model

Many of UX practitioners see “Lean UX” as the answers to the challenge we see in implementing UX into an Agile SDLC where it uses “taking the best parts of our current UX practices and redesigning them specifically for use in an agile world”. Lean UX, is about reducing waste in a process by removing it from the value chain of the usability process..

The proactive measures for border engagement in Agile model has paved path to a new and more practical implementation of UX discipline and methods called “Lean UX” Lean UX once blended with any exiting Agile SDLC, helps to “create a more productive team and a more collaborative process” .

The basic principles Lean UX uses to provide positive refinements to SDLC are through the following 3 foundation stones for it:

  1. Design Thinking:This foundation upholds the concept that “every aspect of a business can be approached with design methods” and gives “designers permission and precedent to work beyond their typical boundaries”.
  2. Agile Software Development: Core values of Agile are the key to Lean UX.
  3. Lean Startup method:Lean Startup uses a feedback loop called “build-measure-learn” to minimize project risk and gets teams building quickly and learning quickly

Lean UX and UX in Agile SDLC

“Lean UX” is seen as the answers to the challenge we see in implementing UX into an Agile SDLC. As discussed in the earlier chapters, Lean UX principles use “taking the best parts of our current UX practices and redesigning them specifically for use in an agile world”.

Lean UX solved many issues were even there with the practices and usability process used in waterfall and different derivative iterative models. In Lean UX practices, there is no more requirements for the “objectives will stay the same and that you can design for a single solution throughout the project”. Also there no more need for the designers to create bulky guidelines and documentations for the developers to be used as a contract. Rather the Lean UX practice upholding agile approach, only focuses on each independent design iteration as a “hypothesis” which has to be validated from a “customer perspective” and from a “business perspective” . The beauty here is because of the Agile process being followed, fast iterations can happen to quickly test hypotheses and get to a great design in the end of the project.

In context of a real world situation the Lean UX is something like:

“The traditional paper work is discarded, while the focus is turned to making sketches of the idea. Then the sketch is presented and discussed with the team. The initial prototype effort is very small comparing to detailed documents, so it’s easy to make changes. After it’s agreed internally, rough prototype is made and tested with users. The learning from users help refine the idea and iteration starts over again”

And this makes case for “collaboration with the entire team” as it becomes critical to the success of the product and the whole project.

The Beauty of Lean UX: Everything is familiar

No practice used in Lean UX is something new. Rather it is “built from well-understood UX practices”. Many of the techniques used over the time in various UX process and have the practical usability even today, have been packaged properly in Lean UX.

Lean UX is not Same as Agile UX

Lean UX is a totally different term than Agile UX.Lean UX details methods and their practical application in dynamic environment of a Lean StartupIt is the converging point for product development and business, through constant measurement and so called “learning loops” (build – measure – learn).

Agile UX defines update of Agile Software Methodology with UX Design methods. It aims to unify developers and designers in the Agile process of product development.

However Lean UX uses Agile UX methodologies, tools to coordinate their software development.

Foundation Stones of Lean UX:

The key ingredients of Lean UX which act as foundation stones for it are:

  1. Design Thinking:
    This foundation upholds the concept that “every aspect of a business can be approached with design methods” and gives “designers permission and precedent to work beyond their typical boundaries”.
  2. Agile Software Development:
    Core values of Agile are the key to Lean UX. It forces on 4 major principles of Agile development to product design:

    • Individuals and interactions over processes and tools: This principle louds the concept of exchange of ideas freely and frequently in a team. “The constraints of current processes and production tools are eschewed in favor of conversation with colleagues”
    • Working Software over Documentation:This focuses on bringing out solution quickly so that it can be “accessed for market and viability”.
    • Customer Collaboration over Contract Negotiation:Collaboration with teammates and customers builds consensus behind decisions which results in faster iterations and lessens heavy documentations.
    • Responding to change over following a plan: “The assumption in Lean UX is that the initial product designs will be wrong, so the goal should be to find out what’s wrong with them as soon as possible” and this helps in finding the right direction quickly.
  3. Lean Startup method:
    Lean Startup uses a feedback loop called “build-measure-learn” to minimize project risk and gets teams building quickly and learning quickly. Typically startups are “free to build products in a manner which differentiate on quality”. Also startups can focus on “intrinsic value and usefulness of the product, rather than on a long list of mostly irrelevant features”. Startups have a distinguishable feature of reducing long product cycles into smaller, shorter chunks and validating these iterations with people that will use the products, actually opens the gate for important information needed to avoid expensive development cycles that come withsome kind of risk. ”The secret sauce of lean startup people is that they advocate for user experience research and design as one of the primary solutions to their business problems, and they do it using plain language”.

Lean Startup method: The concept of “Build-Measure-Learn”

The fundamental activity of a startup is to “turn ideas into products” at the first step. Next it has the aim to measure how customers respond and then to learn whether to pivot or persevere. So basically a startup’s success depends on this feedback loop. To be successful a startup has to accelerate that feedback loop. The feedback loop being employed here includes three primary activities: build the product, measure data and learn new ideas.

However the Lean Startup method employed in Lean UX , is slightly different – it’s basically about “Think-Make-Check”. The difference lies in the fact that in latter case the “feedback loop incorporates your own thoughts as a designer, not just ideas learned through measurement”.

Minimum Viable Product (MVP) – Prototyping at it’s best in Lean Startup Method

Minimum viable product is a version of the product that enables a full turn of the Build-Measure-Learn loop with a minimum amount of effort and the least amount of development time. The MVP is, in fact, an early prototype that serves as a tool to learn and test the team’s assumptions.

Lean UX is where prototyping is best promoted, focusing the prototype on major components of the experience. Once created, it will be immediately testable by any and all users to start the feedback loop. In most of the case the fidelity of the prototype is not a road block, rather the only mission is to get a quick prototype that can be tested quickly. However where the need for better visualization has priority, the best practices and tools are used to develop high fidelity prototypes in shortest time possible.

Principles of Lean UX

There are some guiding principles behind Lean UX which can be used to make sure the methodologies used in a Lean process is on track.

  1. Cross-Functional Teams: Specialists from various disciplines come together to form a cross functional team to create the product. Such a team typically consists of Software engineering, product management, interaction design, visual design, content strategy, marketing, and quality assurance (QA).
  2. Small, Dedicated, Collocated: Keep your teams small—no more than 10 total core people as keeping small team has the benefit of small teams comes down to three words: communication,focus, and camaraderie. It is easier to manage smaller team as keeping track of status report , change management and learning.
  3. Progress = Outcomes, Not Output: The focus should be on business goals which are typically are the “outcomes”, rather than the output product/system or service.
  4. Problem-Focused Teams:“A problem-focused team is one that has been assigned a business problem to solve, as opposed to a set of features to implement”.
  5. Removing Waste: This is one of the key ingredients of Lean UX which is focused on “removal of anything that doesn’t lead to the ultimate goal” so that the team resource can be utilized properly.
  6. Small Batch Size: Lean UX focuses on “notion to keep inventory low and quality high”.
  7. Continuous Discovery: “Regular customer conversations provide frequent opportunities for validating new product ideas”
  8. GOOB: The New User-Centricity: GOOB stands for “getting out of the building” — meeting-room debates about user needs won’t be settled conclusively within your office. Instead, the answers lie out in the marketplace, outside of your building.
  9. Shared Understanding: The more a team collectively understands what it’s doing and why, the less it has to depend on secondhand reports and detailed documents to continue its work.
  10. Anti-Pattern: Rock-stars, Gurus, and Ninjas:Team cohesion breaks down when you add individuals with large egos who are determined to stand out and be stars. So more efforts should on team collaboration.
  11. Externalizing Your Work:“Externalizing gets ideas out of teammates’ heads and on to the wall, allowing everyone to see where the team stands”.
  12. Making over Analysis: “There is more value in creating the first version of an idea than spending half a day debating its merits in a conference room”.
  13. Learning over Growth: “Lean UX favors a focus on learning first and scaling second”.
  14. Permission to Fail: “Lean UX teams need to experiment with ideas. Most of these ideas will fail.The team must be safe to fail if they are to be successful”.
  15. Getting Out of the Deliverables Business: “The team’s focus should be on learning which features have the biggest impact on the their customers. The artifacts the team uses to gain that knowledge are irrelevant.”

(c)2013-14, Samir Dash

This article a part of the title UX Simplified: Models & Methodologies by Samir Dash (ISBN: 978-1-3115-9110-4 / ASIN: B00LPQ22O0 )

“When/If – Then” Tool : Printable Version ( A4 Size Paper)

This “When/If – Then” tool helps to formalize hypothesis or assumption for any Hypothesis -Driven Design or Development.

This tool is helpful than the traditional elaborative templates to capture assumptions, as it was straightforward and can be used with a minimal explanation to a cross-functional crowd, a large portion of such population may not be accustomed towards the use the typical Tools and methodologies. The “When/If – Then” Tool is synonymous with the typical cause-and-effect kind of flow that is understood by and large and required very minimal explanation, and can be like any other UCD tools, can be constructed using simple post-its or printed formats.

Feel free to download and use.

Licence:

(c) 2018, “When/If – Then” Tool by Samir Dash

Creative Commons Attribution-Share Alike 4.0 International License

Lean Methodologies & Hypothesis-Driven Design (HDD)

The second life-line as pointed out in the dimension of ‘Culture’ is “Cultural Shift towards Lean Philosophies” (Refer the part 5 of the current series here https://www.linkedin.com/pulse/desops-next-wave-design-part-5-definition-3-dimensions-samir-dash/ ). In this part of the series, we will be deep diving for the same.

Typically any lean methodologies are based two basic goals:

  1. Helping the organization understanding customer value
  2. Using the key processes of the organization to continuously increase it and work towards delivering a perfect value to the customer through a perfect value creation process with zero waste.

In the context of Design and User Experience(UX), in the seminal book Lean UX by Jeff Gothelf, provides some lean methodologies break the stalemate between the speed of Agile and the need for design in product development lifecycle” [Jeff Gothelf and Josh Seidenjeff, The Lean UX]. If we observe closely, the basic foundations of Lean UX prescribed are common to the core Lean philosophies, namely:

  1. Remove Wastage from Design Process — Moving away from heavy documentation to a process that creates artefacts which can be directly used in the design and development lifecycle itself.
  2. Cross-functional Collaboration — All the stakeholders from any parts of the product lifecycle including the designer, developers, quality experts, analysts, marketers and end-users all collaborate in a transparent and productive way.
  3. Experimentation Based Iterative Execution — So fundamentally it alludes to the UCD core principles, that assumes that the designer (or the team) learns from the execution of the prototypes in an iterative cycle that starts early in the product lifecycle, from which the learning is used as an input to improve the product along the way. If you notice this is the very basis of what we talked about the continuous and integrated feedback loop.

The use of Agile and Design Thinking practices can be seen alluding to the philosophies above. However, regarding the third philosophy — to ensure that the designer has the right approach to conduct the experimentation so that the feedback can be used in a productive way to take informative decision making along the way across the design process requires some non-ambiguous methodologies that can help the designer making some assumption and validate that through experimentation. This is the basis of one of the core foundations of Lean UX and related lean methodologies, which is popularly known as Hypothesis-Driven Design (HDD). As any design that we produce is based on certain assumption and HDD provides the non-ambiguous approach to the same, that follows the third philosophy i.e. Experimentation Based Iterative Execution.

“Declaring assumptions is the first step, in Lean UX process” [Jeff Gothelf and Josh Seidenjeff, The Lean UX] . This statement reflects how the assumptions or the hypothesis is the core the Lean UX principles. Basically, the four steps in the Lean UX approach is the same blocks that appear while in HDD, and can be mapped to the following blocks:

  1. Making Assumption / Formation of Hypothesis
  2. Build a Prototype / Minimum Viable Product (MVP)
  3. Execute / Run the Prototype
  4. Get Feedback / Observation

Now step 4 is the feedback loop that connects to step 1 making a cycle, thereby using the feedback from current iteration as the input to form the assumption for the next iteration.

Interestingly this means the HDD process always depends on outcomes rather than traditional outputs of any traditional process. The outcomes are kind of hints from the segment that can give direction to the design along the way and help in getting incremental validation of the vision by acting as “evidence” or the and more precisely they are the factors that help in “course correction” for the design. As the outcomes of the assumption act as pieces of evidence, it helps in making data-driven decisions.

You may ask, does that mean the design is reduced to quantitative factors which will act as evidence to support some of the designs? Now, here is the interesting aspect that helps to understand the approach — the feedback or the evidence can be either quantitative or qualitative in nature. And to avoid the risk of being limited to measurement-driven design, the qualitative perspective is provided equal footing in HDD approach. This makes a fit case for DesOps that can help in converging the technology with this philosophy to form required service design solution powered by machine learning, artificial intelligence and automation. The aspect of the actual implementation using technology is what we will be exploring in coming articles of this series, mostly while discussing the third dimension i.e. technology.

The very first step in HDD, i.e. “Making Assumption / Formation of Hypothesis” involves articulation of the assumption or the hypothesis. And this is the most important aspect of HDD, as the other steps depend on this. Typical hypothesis statement can be framed as per the following simplest form —

I think doing [this] will result in [that ].

There multiple variations to this that many professionals use in their area for HDD based research and design /development activities. Some add the factors such as the segment or the persona or the end user of the product, something like the following —

For [user / him/her ] I think doing [this] will result in [that ].

If you see the above keeps segment in a context that helps it keep focused on to specific persona, which thereby can lead to the user-centred design process. This is essentially labelled by many as a design hypothesis.

Some use additional factors of forming negation of the hypothesis by adding an attribute like whether this hypothesis is believed by him as true or false. To formulate a null hypothesis and thereby move on to define an alternate hypothesis (which is essentially a negation to the null hypothesis one craves to validate, but due to nature and structure or any metrics associated with that, it may not practically be possible to achieve that. So in such kind of cases the implicitly the state of belief is presented as another factor to take part in the validation process —

Because I think [this] is true, I think doing [this] will result in [that ].

I believe that any hypothesis statement can be cultivated from the bare-bones model of cause and effect being rendered as a formalized statement of “if …. then” or “when … then” kind of format. At least this gives a more practical approach to build a real-world HDD based system which is technically feasible and is the essence of any DesOps system.

The next important aspect in this regard is how to ensure that we are considering the qualitative aspect of the evidence, as discussed above. If we look at irrespective of the fact that there is a crowd of applications out there which claims to support the business through a data-informed approached, they are primarily a set of some variation between the extremes like A/B testing tools and data-driven analytics and data modelling based prediction systems. But if we notice with care at the core of HDD we found that the essence of HDD lies in making sense of the data and treat it as evidence so as to inject it to a UCD based process to fuel design or business decisions. Though coming up with a real-world HDD system would require similar tools and technologies of data-driven systems, but they will be supporting components to HDD based system. In this regard, the real HDD based full fledged system is literally non-existent as of today, while I am writing this article. In regards to DesOps, this makes a lot of difference, as most of the HDD frameworks out there are not 100% autonomous. Most of the implementation of HDD is mostly based on the work practices and methodologies, which is implemented depending on the need and feasibility. However, these are good enough when we use these, in context to the tools we gain from UCD, Lean models and the Design Thinking principles. But, it is interesting that the philosophies of DesOps, take it to the next level, making it autonomous and integrated with the over-arching feedback loop we have been referring many times.

To be continued… Keep tuned in.

 

(c) 2018, Samir Dash. All rights reserved.

Beta Testing in the Ever Changing World of Automation

(Apart from some details about my prototype in Beta Testing, this also explores the interesting definitions from ISO regarding quality that brings “reviews in context” that fundamentally lauds usability philosophies. (Originally first published at RedHat Blog in January 2018)

Beta testing is fundamentally all about the testing of a product performed by real users in a real environment. There are many tags we use to refer to the testing of similar characteristics, such as User Acceptance Testing (UAT), Customer Acceptance Testing (CAT), Customer Validation, and Field Testing (more popular in Europe). Whichever tag we use for these testing cases, the basic components are more or less the same. To discover and fix potential issues, this involves the user and front-end user interface (UI) testing, as well as the user experience (UX) related testing. This always happens in the iteration of the software development lifecycle (SDLC) where the idea has transformed into design and has passed the development phases, while the unit and integration testing has already been completed.

The beta stage in the product lifecycle management (PLM) is the ideal opportunity to hear from the target market and to plan for the road ahead.

As we zoom into this phase of testing it has a board spectrum of ranges. On one side is the Front-End or UI related testing involving UI functionality (Cosmetic, UI level Interaction and Visual look and feel). At the other end is the User Experience (UX), including user testing involving more from A/B (split) testing, hypothesis, user behavior tracking and analysis, heat maps, user flows and segment preference study or exploratory testing and feedback loops.

Beta testing relies on the popular belief that goodness will prevail, which defines the typical tools that carry out such tests. Examples are the shortening of beta cycles, reducing the time investment, increasing tester participation, improving the feedback loop and visibility, etc.

The Importance of Beta Testing

If we dive deeper into the factors behind the existence of tools from different angles, we find two major reasons that advocate the importance of beta testing:

1. Left-right brain analogy, which points to the overlap between humans and technology.

The typical belief is that the left-hand side of the brain mostly processes the logical thinking, while the right-hand side is more about the emotional thinking. Based on this popular analogy, when we map the different aspects involved in different stages of SDLC for a digital product across a straight line from left to right (refer to the diagram below), we will notice the logical and more human-centered aspects are divided by an imaginary line from the center. We will also notice the gradual progression of the emotional index for the components from left to right.

When we map these to the beta testing phase, we notice these right-hand components are predominant. As users of the products (like humans), we are more emotionally connected to such aspects of the product, which are validated or verified in a beta test. This makes beta testing one of the most important testing phases in any SDLC.

Another interesting point to note is that when we look from the traditional software approach to define “criticality”, the areas that are tested during user acceptance testing (UAT / Beta), mostly fall into Class 3 and 4 type of criticality. But because these touch the core human aspects, they become more important. This Youtube video (Boy Receives Enchroma Glasses From Father) helps to illustrate the importance of technology touching the human emotions. It was posted by a popular brand of glasses to demonstrate how they can correct colorblindness in real time for the end user.

Interestingly, this is an aspect of “accessibility”, which is typically covered during a beta test. Considering this video, the aspect of accessibility naturally raises the question about what we can do for this father and son as a tester, as a developer, or as a designer? And when we look at the stats, we find that the number of people the accessibility impacts is huge. One in every five people is challenged by some kind of disability. And unfortunately, some reports indicate that a majority of websites do not conform to the World Wide Web contortion’s (W3C) accessibility guidelines (in 2011 almost 98% of websites failed the W3C’s accessibility guideline).

This helps to demonstrate the human angle, which advocates why beta testing is important to ensure these aspects are validated and verified to ensure the target user needs are completely met.

2. A second reason for advocating the importance of beta testing is evaluating it from the international standards perspective (ISO/IEC 9126-4). This helps to define the difference between usability and quality.

The International Standards Organization (ISO) has been focused on the standards around quality versus usability over time. In 1998 ISO identified efficiency, effectiveness and satisfaction as major attributes of usability. In 1999 a quality model was proposed, involving an approach to measure quality in terms of software quality and external factors. In 2001 the ISO/IEC 9126-4 standard suggested that the difference between usability and the quality in use is a matter of context of use. ISO/IEC 9126-4 also distinguished external quality versus internal quality and defined related metrics. Metrics for external quality can be obtained only by executing the software product in the system environment for which the product is intended.

This shows that without usability/human computer interaction (HCI) in the right context, the
quality process is incomplete. The context referred to here is fundamental to a beta test where you have real users in a real environment, thereby making the case of the beta test stronger.

Beta Testing Challenges

Now that we know why beta testing is so very critical, let’s explore the challenges that are involved with a beta stage.

Any time standards are included, including ISO/IEC 9126, most of these models are static and none of them accurately describe the relationship between phases in the product development cycle and appropriate usability measures at specific project milestones. Any standard also provides relatively few guidelines about how to interpret scores from specific usability metrics. And specific to usability as a quality factor, it is worth noting that usability is that aspect of quality where the metrics have to be interpreted.

When we look at popular beta-testing tools of today, the top beta testing tools leave the interpretation to the customer or to the end-user’s discretion. This highlights our number one challenge in beta testing, which is how to filter out pure perception from the actual and valid issues and concerns brought forward.

As most of the issues are related to user testing (split testing and front-end testing), there is no optimized single window solution that is smart enough to handle this in an effective manner. Real users in the real environment are handicapped to comprehend all the aspects of beta testing and properly react. It’s all a matter of perspective and all of them cannot be validated with real data from benchmarks or standards.

The World Quality Report in 2015-16 Edition indicated that expectations from Beta testing is changing dramatically. It hinted that the customers are looking for more product insights through a reliable way to test quality and functionality, along with the regular usability and user testing in real customer facing environment.

It’s not only the Beta Testing, but more user-demand is also impacted by the rising complexities and challenges due to accelerated changes in the technology, development, and delivery mechanisms. The 2017-18 World Quality Report states that the test environment and test data continue to be an achilles heel for QA and Testing.   The challenges with testing in agile development are also increasing. There is now a demand for automation, mobility, and ubiquity, along with smartness to be implemented in the software quality testing. Many believe that the analytics-based automation solutions would be the first step in transforming to smarter QA and smarter test automation. While this true for overall QA and testing, this is also true for Beta Testing, which deals with the functional aspect of the product.

Let’s see where we stand today. If we explore popular beta testing solutions, we will get a big vacuum in the area where we try to benchmark the user’s need for more functional aspects, along with the usability and user testing aspects. Also, you can notice in the diagram below that there is ample space to play around with the smart testing scenario with the use of cognitive, automation, and machine learning.

(Note: Above figure shows my subjective analysis of the competitive scenario.)

Basically, we lack “smartness” and proper “automation” in Beta Testing Solutions.

Apart from all these, there are some more challenges that we can notice if we start evaluating the user needs from the corresponding persona’s viewpoint. For example, even when assuming that the functional aspect is to be validated, the end-user or the customer may have an inability to recognize it. The product owner, customer, or end-user are part of the user segment who may not be aware of the nuts and bolts of the technology involved in the development of the product they are testing to sign it off. It’s like the classic example of a customer who is about to buy a second-hand car and inspects the vehicle before making the payment. In most of the cases, he is paying the money without being able to recognize “What’s inside the bonnet!”. This is the ultimate use-case that advocates to “empower the customer”.

So, how do we empower the end user or the customer? The tools should support that in a way so that the user can have his own peace of mind while validating the product. Unfortunately, many small tools which try to solve some of those little issues to empower the user are scattered (example: Google Chrome extension that helps to analyze CSS and creates the report or an on-screen ruler that the user can use to check alignment, etc.). The ground reality is that there is no single-window extension/widgets based solution available. Also, not all widgets are connected. And those which are available, not all are comprehensible to the customer/end-user as almost all of them are either developer or tester centric. They are not meant for the customer without any special skills!

When we look at the automation solutions in testing as part of much Continous Integration (CI) and Continuous Delivery (CD), they are engaged and effective in mostly “pre-beta” stage of SDLC. Also, they require specific skills to run them. With the focus on DevOps, in many cases, the CI-CD solutions are getting developed and integrated with the new age solutions looking at the rising complexities of technology stacks, languages, frameworks, etc.. But most of them are for the skilled dev or test specialists to use and execute them. And this is something that does not translate well when it comes to Beta testing where the end-user or the customer or the “real user in real environment” are involved.

Assuming we can have all these automation features enabled in BETA, it still points to another limitation in the existing solutions. It’s because the employment of automation brings in its own challenge of “information explosion”, where the end user needs to deal with the higher volume of data from automation. With so much information, the user will struggle to get a consolidated and meaningful insight of the specific product context.   So what do we need to solve these challenges? Here is my view — we need smart, connected, single window beta testing solutions with automation that can be comprehensible to the end-users in a real environment without the help of the geeks.

For sometime since a last few years, I have been exploring these aspects for the ideal beta testing solution and was working on the model and a proof of concept called “Beta Studio”, representing the ideal beta testing solution, which should have all these —  Beta-Testing that utilizes data from all stages of SDLC and PLM along with the standards + specs and user testing data to provide more meaningful insights to the customer.  Test real applications in real environments by the real users. Customer as well as end-user centric. Test soft aspects of the application — Usability, Accessibility, Cosmetic, etc..  Be smart enough to compare and analyze these soft aspects of the application against functional testing data.

Use machine-learning & cognitive to make the more meaningful recommendation and not just dump info about bugs and potential issues. Here is an indicative vision of Beta Studio:

Mostly this vision of the ideal beta testing solution touches upon all the aspects we just discussed. It also touches upon all the interaction points of the different personas; e.g. customer, end-user, developer, tester, product owner, project manager, support engineer, etc.. It should cover the entire Product Life Cycle and utilize automation along with the machine learning components such as Computer Vision (CV) and Natural Language Processing (NLP). Then gathering this information to be processed by the cognitive aspect to generate the desired insights about the product and recommendations. During this process, the system will involve data from standards and specs along with the design benchmark generated from the inputs at the design phase of the SDLC, so that meaningful insights can be generated.

In order to see this vision translated into reality, what do we need? The following diagram hints about the six steps we need to take.

  1. First we should create the design benchmark from the information at the design stage that can be used in comparing the product features against metrics based on this design benchmark.
  2. Then automate and perform manual tracking of the product during runtime in real time and then categorize and collate this data.
  3. This involves creating features to support the user feedback cycle and user testing aspects (exploratory, split testing capabilities).
  4. Collect all standards and specifications on different aspects — e.g. Web Content Accessibility Guideline (WCAG) Section 508, Web Accessibility Initiative Specs ARIA, Design Principles, W3C Compliance, JS Standards, CSS Standards & Grids, Code Optimization Metrics Error codes & Specs, Device Specific Guidelines (e.g. Apple Human Interface Guideline) etc.
  5. Build the critical components such as Computer Vision and Natural Language Processing units which would process all the data collected in all these stages and generate the desired metrics and inferences.
  6. Build the unit to generate the model to map the data and compare against the metrics.
  7. The final step is to build the cognitive unit that can compare the data and apply the correct models and metrics to carry out the filtering of the data and generate the insights which can be shared as actionable output to the end-user/customer.

While experimenting for BetaStudio, I have explored different aspects and built some bare bone POCs. For example, Specstra is a component that can help create Design benchmark from design files.

With Specstra I was trying to address the issues related to the cosmetic aspect or visual look and feel. Typically when it comes to cosmetic issues, more than 30% are non-functional and mostly cosmetic. There is no reliable solution that helps in benchmarking these kind of issues against specific standards. One third of the issues found during the beta / UAT stages of testing are mostly cosmetic and alignment issues, including category 3 and 4 types. And these happen mostly because the two persona’s involved (developer and designer) have their own boundaries defined by a mythical fine line.

When a developer is in focus, roughly 45% of them are not aware of all the design principles employed or the heuristics of UX to be employed. Similarly, half of the designers are not aware of more than half of the evolving technological solutions about design.

And in 70% of the of the projects, we do not get detailed design specs to benchmark with. Detailing out a spec for design comes with a cost and required skills. In more than two-thirds of the cases of development there is the absence of detailed design with specs. Many of the designs are not standardized and most of them do not have clear and detailed specs. Design is carried out by different tools so it is not always easy to have a centralized place where all the designs info is available for benchmarking.

Specstra comes in handy to solve this. It is an Automation POC that is more like a cloud-based Visual Design Style Guide Generator from the third party design source files. This is a case where the user would like to continue using his existing Design tools like Photoshop/Sketch or Illustrator, PDF etc..

You can view the video of the demo here:

View the demo of the BetaStudio POC here:

I understand that reaching the goal of an ideal beta testing solution might require effort and will likely evolve over time. Rest assured, the journey has started for all of us to connect and explore how to make it a reality.

Originally published at the RedHat Blog at https://developers.redhat.com/blog/2018/01/05/beta-testing-automation/

 

 

 

 

DesOps: The Business Value Proposition

Each design is a proposed business solution which is essentially is a hypothesis. Any design process — as strives to get the answer or solution to a fundamental problem — essentially starts with the problem in mind and some assumptions in mind, which is mostly a hypothesis. And to solve the problem, with the assumed hypothesis or the business value in mind, the designer iterates and if he uses User-Centered Development (UCD) approaches, he would go into the cycle of Think – Make – Test cycle. And this implicit way of solving the issue creatively uses references to different aspects of business, namely :

  • The complete business offering
  • Customer orientation and service innovation for customer relationship
  • Business Infrastructure
  • Revenue Streams
  • Productivity and Cost control structures

As DesOps principles and practices have Hypothesis Driven Design & Development (HDD) and UCD process as its core components, it also refers to the same business value propositions. Implementing DesOps actually takes these into consideration and tries to use technology to improve the process around these.

If we refer to any standard business model frameworks such as Business Model Canvas, a template that is popularly used for developing and investigating every important aspect of the organization. The framework in such a template outlines investigations for areas such as key partners, key activities, key resources, value propositions, customer relationship, channels, customer services, cost structure and revenue streams, which always helps to understand and identify the core goals, strengths and priorities of the business that provides the context in which the UX has to be seen. This can be seen in the following equation: “Customer needs + Business context +Technological feasibility = Successful UX making the successful product”.

In a business model, we refer to UX when we plan a strategy for “Value Proposition”. In the typical Business model Canvas, value proposition involves areas like the following, which can be seen traced back to DesOps principles & practices.

  • Newness: In DesOps, this is associated with “Continous Discovery” and “Design Thinking” practices.
  • Performance: Improving performance by optimizing the process blocks through Agile & Lean methodologies through the implementation of automation through service design approach.
  • CustomizationDesOps implementation focuses on customizing the process blocks through service design approach and defining business process redesign/engineering.
  • Getting the Job Done”: This is fundamentally the result oriented approaches taken in DesOps which touches upon different aspects like roles, cohesive team play, removing wastage through Lean methodologies and the similar.
  • Design: This is core to DesOps, and is seen as the creative problem-solving.
  • Brand/ Status: Brand and Status is well associated with the feedback loops that include the real users in context and feeding the design process with continuous feedback including brand perception and related mental model of the user of the target segment.
  • Price: Though purely a market associated component, the price can be dramatically reduced through implementation of DesOps, as it focuses on reducing waste and improving efficiency through automation and process improvements
  • Cost Reduction: It is one of the fundamental components of ROI on DesOps in an organization. DesOps helps reducing cost through service design approach to optimize and redefine business process.
  • Risk Reduction: DesOps helps reducing ambiguity by the implementation of optimized process and automation powered by the feedback loop that touches all the roles and the aspects be it human or machine in context. This improves reliability and thereby reduces risk.
  • Accessibility: Through the implementation of Design Thinking, UCD model like contextual and participatory designs and continuous feedback loop, DesOps helps to ensure the accessibility factors are in consideration while iterating over a design.
  • ConvenienceUsability: Through its advocacy of UCD models and Design Thinking and integrated feedback loop, DesOps help in ensuring usability aspects in all design delivered.

From a product manager’s standpoint, the successful UX meant for a business must balance between the needs of the users and the feasibility of implementation of the UX solution within the business context, and all the practices and principles of DesOps converge towards this.

 

(c) 2018, Samir Dash. All rights reserved.

The 10 Practices of Design Operations (DesOps)

A method, procedure, process, or rule used in a particular field is typically defined as practice for that field. In last article, we discovered about the 10 guiding principles that drive the DesOps. No wonder the practices involved in DesOps, loud the same principles to the core. Note that we are still discussing the culture driven by DesOps / DesignOps, that is typically fuelled by these practices.

Here are the broad practices that drive the DesOps philosophies:

 

Download Poster: http://desops.io/2018/05/09/infographic-the-ten-practices-of-desops-aka-designops/

 

1. Design Thinking

This practice ensures that we employ the creative problem solving, the typical methodologies and tools of Design thinking. Here are some quick notes and list of methodologies and tools used in IBM version of it which anyway follows the fundamental principles of Design Thinking https://medium.com/eunoia-i-o/quick-guide-notes-on-the-ibm-design-thinking-78490d7433dd.

The typical tools of Design Thinking, such as Stakeholders map, Experience maps (As Isand To Be), Personae, Roadmap, MVP, Kano Modelling, Story Boarding, Priority Grid etc. are coupled with continuous practices defined in the following to implement the Continuous Loop of the Design Thinking practice that holds the DesOps philosophies.

2. User-centred Design (UCD) and Usability Design

Users (both the typical user / persona from UX angle and the segment from marketing/business context ) are at the core of DesOps. Any design solution generated fundamentally is an advocacy of the user needs and tries to direct the business goals to build upon this. The business goals are also in such cases are market specific and are based on the pulse of the segments driven by the user needs. You can have a quick note on UCD and usability-design here https://www.linkedin.com/pulse/20140702070557-9377042-usability-design-and-user-centered-design-ucd/. As UCD or Usability design focuses on the Iterative Design approach of User Centered System Design (UCSD) process this fundamentally contributes to DesOps goals.

As UCD supports growing product through iterative design that is also fuelled by all 3 models of design which contribute to typical Design Thinking as well as Lean UX models, namely:

  • Cooperative Design: This involves designers and users on an equal footing.
  • Participatory Design (PD): Inspired by Cooperative Design, focusing on the participation of users
  • Contextual Design: “Customer-Centered Design” in the actual context, including some ideas from Participatory Design.

And here are the steps we use while implementing UCD , irrespective of the above models we follow: All these UCD models involve more or less a set of activities grouped into the following steps mentioned below:

  • STEP 1 – Planning: in this stage the UCD process is planned and if needed customized. It involves understanding the business needs and setting up the goals and objectives of the UX activities. Also forming the right team for the UX needs and if needed hiring specialties fall into this step.
  • STEP 2 – User data collection and analysis: This step involves data collection through different applicable methodologies such as user interviews, developing personas, conducting scenarios, use-cases and user stories analysis, setting up measurable goals.
  • STEP 3 – Designing and Prototyping: This involves activities like card sorting, conducting IA, wireframing and developing prototyping.
  • STEP 4 – Content writing: this involves content refinement and writing for web and similar activities.
  • STEP 5 – Usability testing: This involves is a set of activities of conducting tests and heuristic evaluations and reporting to allow refinement of the product. However, Usability Testing can have its set of steps involving similar activities such as planning , Team forming, testing, review and data analysis and reporting.

And you will see all these naturally fall into the places while any DesOps is implemented.

3. Hypothesis-Driven Design/Development (HDD) & Data-driven Decisions making

The DesOps story remains incomplete without referring to one of its key practices that are Hypothesis-driven – development (HDD), which certainly contributes to the service design like DesOps, that brings possibilities of changes to inculcate design thinking, innovation and organizational changes. It also promotes the lifecycle methods and adjusts them to ensure that integrated work-flow and work-culture is established that can make the best use of data-driven decision making by running multiple early-stage experiments (synonymous with what we are trying to achieve through continuous feedback loop and prototyping) and gathering insights from their outcomes (and not just output!). Another interesting thing is that this advocates the use of UCD approaches as it focuses on making an assumption, running experimenting and validating them with measurable data, and thereby taking some action based on the same.

Will elaborate on HDD driven methodologies in context to DesOps as we move in this series of articles.

4. Agile / Iterative Life Cycle

There are several challenges in integrating UX design and related activities into a typical agile software development lifecycle process. The most common problem is typically “ finding a balance between up-front interaction design and integrating interaction design with iterative coding with the aim of delivering working software instead of early design concepts”. This happens mostly because typical pure SDLC approaches primarily aim at “efficient coding tactics together with project management and team organization instead of usability engineering”.As Agile is more “a way of thinking about creating software products’ rather than being a specific process or methodology hints at the challenges of UX integration into it as integrating user research and UX design with agile, is itself an “agile antipattern”.The very idea of SDLC is a process for developing software, traditionally never kept the “user” into a focus, or event kept any scope for methodologies that try to bring any component that is not considered as a native ingredient of the process of creating a software product. The focus was always the “cost, scope, and schedule” that drive any traditional SDLC models including Agile. And sure enough, this typically gives rise to the challenges for UX integration into any SDLC as project managers never try to upset the balance of these three by reducing costs, tightening deadlines, and adding features in the specification. To know more about the typical challenges we face while implementing design / UX into Agile SDLC read my earlier article here: https://www.linkedin.com/pulse/20140706143027-9377042-challenges-in-ux-integration-with-different-sdlc-models/

However, there ways to mend this gaps between process driven life cycle models such as Agile model — one of such is to implement usability design model (we discussed that as a practice of DesOps above). Usability process supplements to any software development life cycle at various stages, as is not a complete product development process as it does not output the final product at the end of the process cycle. One such solution is reflected in below diagram :

And it is interesting to see that each cycle in such solution is actually contributing to a continuous cycle of Conceptualize – Design – Build – Test that aligns nicely with DesOpsprinciples and other practices.

5. Lean UX Approach

The Lean UX practice focuses on the Lean philosophies that focus primarily on reducing waste from the process and provide ways to simplify and expedite the delivery keeping the quality intact or enhanced. Interestingly the Lean UX is based on the 3 foundations which are also part of the DesOps practices list we are discussing:

  • Design Thinking: This foundation upholds the concept that “every aspect of a business can be approached with design methods” and gives “designers permission and precedent to work beyond their typical boundaries”.
  • Agile Software Development: Core values of Agile are the key to Lean UX.
  • Lean Startup method: Lean Startup uses a feedback loop called “build-measure-learn” to minimize project risk and gets teams building quickly and learning quickly

No practice used in Lean UX is something new. Rather it is “built from well-understood UX practices”. Many of the techniques used over the time in various UX process and have the practical usability even today have been packaged properly in Lean UX. So the following foundation pillars of this also supports DesOps as inherited from this practice:

  • Cross-Functional Teams: Specialists from various disciplines come together to form a cross-functional team to create the product. Such a team typically consists of Software engineering, product management, interaction design, visual design, content strategy, marketing, and quality assurance (QA).
  • Small, Dedicated, Collocated: Keep your teams small—no more than 10 total core people as keeping small team has the benefit of small teams comes down to three words: communication, focus, and camaraderie. It is easier to manage the smaller team as keeping track of status report, change management and learning.
  • Progress = Outcomes, Not Output: The focus should be on business goals which are typically are the “outcomes”, rather than the output product/system or service.
  • Problem-Focused Teams:“A problem-focused team is one that has been assigned a business problem to solve, as opposed to a set of features to implement”.
  • Removing Waste: This is one of the key ingredients of Lean UX which is focused on “removal of anything that doesn’t lead to the ultimate goal” so that the team resource can be utilized properly.
  • Small Batch Size: Lean UX focuses on “notion to keep inventory low and quality high”.
  • Continuous Discovery: “Regular customer conversations provide frequent opportunities for validating new product ideas”
  • GOOB: The New User-Centricity: GOOB stands for “getting out of the building” — meeting-room debates about user needs won’t be settled conclusively within your office. Instead, the answers lie out in the marketplace, outside of your building.
  • Shared Understanding: The more a team collectively understands what it’s doing and why, the less it has to depend on secondhand reports and detailed documents to continue its work.
  • Anti-Pattern: Rock-stars, Gurus, and Ninjas: Team cohesion breaks down when you add individuals with large egos who are determined to stand out and be stars. So more efforts should on team collaboration.
  • Externalizing Your Work: “Externalizing gets ideas out of teammates’ heads and on to the wall, allowing everyone to see where the team stands”.
  • Making over Analysis: “There is more value in creating the first version of an idea than spending half a day debating its merits in a conference room”.
  • Learning over Growth: “Lean UX favours a focus on learning first and scaling second”.
  • Permission to Fail: “Lean UX teams need to experiment with ideas. Most of these ideas will fail.The team must be safe to fail if they are to be successful”.
  • Getting Out of the Deliverables Business: “The team’s focus should be on learning which features have the biggest impact on the their customers. The artefacts the team uses to gain that knowledge are irrelevant.”

You can read more in one of my articles here:https://www.linkedin.com/pulse/20140710010240-9377042-lean-ux-another-agile-ux/?

6. Fail-Fast through Prototyping

Typically a Fail-Fast is about immediately reporting any condition that is likely to indicate a failure. Also, Fail-Fast allows gathering early stage feedback that serves as an input for the continuous UCD model which helps bring up solutions to the design issues using such input and thereby minimizes the risk of product failure in the hand of users or in the market. This is also a philosophy that aligns to the Lean Startup methodology and accelerates innovation as it encourages taking early stage risk. Typically the startup cultures undertake bold experiments to determine the long-term viability of a product or strategy, rather than proceeding cautiously and investing years in a doomed approach. In service design, this helps to improve the processes to make use of systems that support Lean methodologies and model. The great part is that this in DesOps while getting combined with UCD processes, provides options to run short and quick UCD iterative cycles of Think – Make – Break kind of model.

Prototype plays a crucial role in UCD models to achieve Fail-Fast and thereby ensuring early feedback on the design is received that can contribute to the evolution of the product. Different fidelity of prototypes is used in order to ensure that the target experience can be tested.

7. Continuous Discovery

Continuous Discovery is primaily involved with the conceptualization stage of the product lifecycle. This practice mostly is driven factors like

  • User focus: The goals of the activity, the work domain or context of use, the users’ goals, tasks and needs should control the development.
  • Active user involvement: Representative users should actively participate, early and continuously throughout the entire development process and throughout the system life cycle.
  • Simple design representations: The design must be represented in such ways that it can be easily understood by users and all other stakeholders.
  • Explicit and conscious design activities: The development process should contain dedicated design activities.

This practice, however, is not just limited to conceptualization stage, but organically is part of the evaluation and build stages as the outcomes from such stages of life cycle, it gets the input of feedbacks and evaluation results which aid in the discovery of the solution through the design process.

8. Continuous builds and delivery

This practice focuses on continuous design delivery that ensures that the DesOps sustains the lifecycle and supports iterative UCD cycles. This involves the process that supports the design of the solution which thereby contributing to the system development that is iterative and incremental. The early part of life cycles involving such practice typically gains life from prototyping which is used to visualize and evaluate ideas and design solutions in cooperation with the users.

So the factors in this practices are :

  • Evolutionary systems development: The systems development should be both iterative and incremental.
  • Prototyping: Early and continuously, prototypes should be used to visualize and evaluate ideas and design solutions in cooperation with the users.

9. Integrated & incremental testing

Evaluation and getting the feedback from all stages of the lifecycle is key to any DesOpsimplementation, therefore integrated testing (including usability testing) in an incremental fashion is what that plays a stronger role among all the practices. This actually draws from the UCD models running a User Centered System Design (UCSD) approach. As UCD experts help in benchmarking usability tests popularly known as “summative evaluations” that evaluates the performance of the system /product developed on several grounds. The metrics of this test is typically based on the “error rate for users as they use the system”, the “time it takes to attain proficiency performing a task”, and the “time it takes to perform a task once proficiency has been attained”. So the factor that drives the practice is —

  1. Evaluate use in context: Baseline usability goals and design criteria should control the development.

Note that the key here is that all the testing should support evaluations in context. In the real context of use, getting the data is what makes this effective and thereby making DesOpsmore fruitful.

(Fig – Source: Re-imagining Beta Testing in the Ever-Changing World of Automationhttps://medium.com/eunoia-i-o/re-imagining-beta-testing-in-the-ever-changing-world-of-automation-3579ac418007 )

The ISO standard also defines Quality process where context plays the major role. And interestingly Usability testing and HCI aspects are all driven by context. Read one of of my articles on how context plays a critical role in testing and usability, which also narrates an experiment named BetaStudio here – https://medium.com/eunoia-i-o/re-imagining-beta-testing-in-the-ever-changing-world-of-automation-3579ac418007 .

10. Service Design on an Integrated Feedback-Loop Model

The integrated feedback loop is actually more than getting testing reports. This practice ensures that the feedback flows from any point to any point as needed, may it be from stakeholder to Designers, or End-Users to Developer, Testers to Designers or in any path that flows from one persona to the other. Also, this includes the service design that helps to implement the DesOps which ensures the information, as well as the feedback, is flowing seamlessly even including from and to the systems and different roles. This certainly uses service design employing recent technologies like Automation, Machine Learning and Artificial Intelligence etc.

Hope this article was helpful. Keep tuning in for the next parts of this article series. Before moving on to next points on Culture of DesOps, we will be looking into a Business Model Canvas and try to see how DesOps fits in. Do share the words.

 

 

(c) 2018, Samir Dash. All rights reserved.

The 10 Principles Driving DesOps Culture


(Originally first published on April 17, 2018 at  https://www.linkedin.com/pulse/desops-next-wave-design-part-6-10-principles-driving-its-samir-dash/  )


As pointed out in the last article, the dimension ‘Culture’ has at the hindsight 3 life-lines, namely

i. Principles & Practices

ii. Cultural Shift towards Lean Philosophies 

iii. The way the team works (Mostly Design team but not excluding the intra-team work-culture)

In the current article, we will be focusing on the Principles that drive the DesOps philosophies. Broadly, the DesOps can be seen based on 10 commandments or the guiding principles which help set the goals —

 

Download Poster: http://desops.io/2018/05/08/infographic-the-ten-commandments-of-desops/

 

1. Implementing DesOps is to follow service design methodologies.

In reality, DesOps is the most evolved service design methodology, that touches upon all the different roles involved in a product lifecycle starting from conceptualization till the point product being used by the consumers. Therefore, DesOps implementation should establish the best practices for designing services according to both the needs of customers and the competencies and capabilities of service providers.

2. The feedback loop should cut across the product lifecycle

Ensure the requirements are fed into the DevOps cycle through feedback loop seamlessly so that the adjustment of the requirements can be done based on the feedback loop in the design stage that is also iterative and in the agile sprint loops. The requirements formulation is one of the core activities of design.And the requirements should be able to adjust based on the feedback that typically comes from means that cut across design as well as the development -test-deployment-production stages. So it involves the feedback from the large part of the delivery-deployment track, which is fundamentally DevOps impacted process areas. And when continuous integration and delivery (CI & CD) are part of the DevOps implementation, the continuous feedbacks that pours from various stages of DevOps should be addressed in an agile as well as iterative design stage, where these feedback fuel the design decisions to adjust requirements ignorer to meet the user as well as market needs. It is therefore critical that the DesOps implementation must follow the principle of completing the full circle of feedback loop that would cut across both designs as well as the dev-deploy-production stages – i.e. the complete SDLC and use it as an input for Design.

3. Empower stakeholders for better decision making — Hypothesis and Data-driven decision making for design and development

Many decisions are taken for a product based on the experience and the gut-feel of the stakeholders that impacts the product. The design – the creative problem solving – gets impacted by this as in real world, design decisions are always biased by the stakeholder’s view of the product and the market. A true DesOps should enable the stakeholder to take decisions based on a more data-driven way. Typical A/B testing, Hypothesis-Driven Development approach and analytics-driven decision making always help the stakeholder to evaluate the options and it guides him to take the more pragmatic decisions by reducing his baseness from his pre-conceptualised version of the product, user, and market. True DesOps, aspires to a world where decisions are taken by validating the data against the postulates that impacts the design as well as the success of the product itself in the hands of users and market.

4. Enable Design Thinking in the team

When we broadly speak about design, we do mean the core essence that is about creative problem solving which transcends beyond the typical professional design practices such as social context and business. Because of this, the Design Thinking strategies as solution-focused thinking fuels innovation and in today’s world, it is a popular approach that is employed in communities and organizations.

Many organizations, like IBM, KPMG etc. Have been improving the Design frameworks around the core concept of design thinking to come up with design strategies that would work with product life cycles and project management processes supporting Agile and Iterative approaches. For example, IBM Design Thinking (here is a quick guide https://medium.com/eunoia-i-o/quick-guide-notes-on-the-ibm-design-thinking-78490d7433dd  )

However, in many angles, all these versions of Design Thinking fail flat to impact the final deliverable products in a quick paced agile environment powered by continuous integration and delivery track. The workshops of such Design Thinking sessions do provide innovative outputs, but when it comes to implementations, the sync between these and the final MVP that goes out of the delivery track, is lost and in most of the cases the stakeholders feel that despite the investment into the design thinking approaches, the expected level of outcome in delivery is not met and in many cases changes in requirement in the middle, too many iterations leading to over budgeting, mismatch of skill set and resources requirements to the initial planning work and the feasibility aspects were blamed for the inferior quality of delivery in contrast to the grand outcome from the Design Thinking.

Some years back, I was part of a large UX team of a Software Conglomerate, where I practiced their version of the Design Thinking framework. And during many workshops with major customers, and also from casual discussions with some of the stakeholders and Design Thinking practicenors, I realised that though many of our design thinking workshops were successful and were received by the clients with excitement and sense of achievement, this in many cases died down after a few months of implementation happened over fast paced agile delivery modes, as the translation of the great solutions that came out from the design thinking sessions, didn’t actually live up to the expectations due to too many changes in requirements happened because of feasibility, resource management, technology and similar issues.

For me, the takeaway from there was that in fast-paced delivery track, the feedback loop must be connected across the design stages to be reviewed at the same pace to ensure that the overall speed of the delivery with quality remains intact. In many cases, because of the fact the issues that popped up during the implementation stages, were never efficiently part of the Design Thinking sessions of the next iteration or design cycle that runs, it was not effective in coming out more practical and pragmatic design solutions and make the necessary adjustments to the requirements. And in such cases the DevOps even when efficiently implemented, the lagging and relatively slow churning-outs from the Design Thinkings, the overall delivery gets impacted thereby resulting into inferior quality MVP than that of the envisioned one. As the Design and Development are two wheels of the bicycle that the product is riding on, even if the Dev wheel is made efficient by implementing the best possible DevOps, the overall pace and efficiency of the cycle is reduced and is equal to the other lagging wheel i.e. Design.

To overcome this, one of the core principles of DesOps is to empower Design Thinking to make it more efficient in a fast-paced DevOps enabled delivery process, thereby making it the best fit case for the organizations to adopt.

5. Advocate Lean Methodologies and Agile Philosophies

All “Lean” models typically focus on reducing waste in a process by removing it from the value chain of the usability process. One of the key goals of DesOps is about taking proactive measures in reducing waste and fundamentally support tries to bring the practical implementations of it. Therefore, DesOps, at the core advocates, Lean philosophies like that we see in Lean UX models. Along with the fact that Lean UX itself is based on 3 basic principles like “Design Thinking”, “Agile Software Development” and “Lean Startup methods” like “build – measure – learn”, it is an organic part of in DesOps philosophies.

6. Translate User-Centered Design into an actual process that can be used on the ground

DesOps supports User-Centered Design (UCD) process in which “the needs, wants and limitations of end-users of a product are given extensive attention at each stage”. Basically, in DesOps, the focus is same as on any optimized UCD process, where most of the priority is on understanding the behavioral aspect of the user interacting so that the user’s learning curve in using the system can be evaluated in order to optimize and reduce it. As DesOps, emphasizes on optimizing the product around “how users can, want or need to use the product, rather than forcing the users to change the behavior to accommodate the product”, it loads the philosophies of typical UCD model. So DesOps inherits the following factors from UCD, namely —

  1. Needs of Users
  2. Limitations of Users
  3. Preference of Users
  4. Business objectives of the Product. 

7. Cohesive play in the cross-functional team— designer, stakeholders and developers into the team play

As DesOps advocates LeanUX, it inherits one key attributes from Lean UX, which is about cross-functional team-play. Typically the Lean UX advocates that specialists from various disciplines should come together to co-create the product. Such team traditionally comprises of roles in software engineering, product management, interaction design, visual design, content strategy, marketing and quality assurance or testing disciplines. DesOps fundamentally enables this in bring these diverse roles to the same page and improves productivity through improving feedback loop and communication and providing a train of outcomes of the product development track that helps in “continuous discovery” and validation of product requirements and ideas.

8. Technology decisions should be guided by lowering the boundaries between roles and automation to reduce waste and repetitive jobs that work for the product and project.

Perhaps the biggest visible attribute of DesOps is the technology aspect, and the basic principle around technology is about the technology decision making. This is one of the most critical and impacting principles, as the right decision making for choosing the correct technology that will enable or help implement the most effective DesOps for the target organization/team. As DesOps involves the workflows, that touches the different aspects of Software Development Life Cycle and as the goal of DesOps is to achieve scale, the tracking, and traceability of workflows and responsibilities involved from different team members, technology, and tools employed to make it possible are carefully selected to meet the specific design culture that was redesigned with the re-engineered processes and workflows. Employment of automation, machine learning, and artificial intelligence plays a key role in the context of the technology selection to aid the DesOps implementation as well as improving the case for it to play a complementary role for DevOps to make the full-circle. We will elaborate on this aspect in details in coming article of the series.

Back in 2014-15, in one of my old exploration into visual design prototype called Specstra,I was exploring the possibilities to remove repetitive manual work of draftsmanship in the creation of visual design specs or style guide (part of static design systems) involving automation the process to reduce the efforts from days to just 2-3 minutes. It used standard design file formats including properitry ones like Photoshop (.PSD), Illustrator (.AI) and PDF, as input to generate completely detailed pixel perfect style guide with annotations and target resolution and device screen density specific specifications that can be used by the development team to create screens. You can read about this story at IBM Design Blog at https://medium.com/design-ibm/specstra-experimenting-with-automation-in-design-industry-4641c0b4244d or watch the video below :

(Video – In my experimental prototype Specstra the attempt was made to bring automation to visual design process to reduce repetitive manual tasks. )

9. Redesigning and re-engineering the Processes

Optimised and re-designed processes are key to reduce waste and remove repetition, At the same time lower boundaries between roles. Broadly the processes that are redesigned in a DesOps implementation include strategy and workflows for reducing wastage. Any repetitive blocks are removed or replaced with the ones that provide quick turnaround time for design deliverables as well as the access to all-around feedback -loops across the lifecycle including development, deployment areas. The goals of improvement of processes also include lowering boundary among different roles to enable a cross-functional team to quickly iterate and produce design outputs which can be fed into DevOps thread in the cycle. Also, process improvements include automation focus along with workflows for traceability and tracking and preparing benchmark data and validating the feedbacks and product performance against them in a seamless way. Also, alongside the focus of the process redesign as part of DesOps include the areas to customize and complement workflows to the existing SDLC if required, that is in the process where possible e.g. Agile, Iterative and Lean models.

10. Enable reviews based on data-driven benchmarks.

A DesOps implementation should support benchmarking and validating against those benchmarks for design as well as implementation aspects. You might have noticed, that the feedback-loop is fundamental to the DesOps philosophies. And you might have noticed that in the previous principles, most of them directly or indirectly pave the way for integrated feedback-loop that helps to enable faster delivery track for an efficient and productive product delivery with the tint of innovation. So, it is critical for the DesOps as a service design must make ways for benchmarking the various attributes of the product and the delivery method itself that contributes to the speed and agility so that the feedback loop can make good use of these to validate and support a faster and informed decision making. In one of my recent explorations into beta-testing for RedHat’s QE CampX and Idea Incubation prog, I had focused on this aspect and tried to come up with a prototype called BetaStudio,that tried to address how to create benchmark for attributes which are more on the creative side and fall in the design stages of a product’s lifecycle, and thereby using some kind of mechanism to validate based on such benchmarks. Read about Beta Studio at https://developers.redhat.com/blog/2018/01/05/beta-testing-automation/

The above diagram shows the high-level vision of BetaStudio that used benchmarking of design aspects which were used at a later stage (mostly the track covered by DevOps) to validate the feedback most of which were generated by automation. In later articles, I will be elaborating on this aspect in details.

(Video – In my experimental prototype BetaStudio the benchmark based review and feedback loop was explored )

While implementing DesOps in the organization if we contradict any of these principles, it would result in a half-baked design operation that would not help in achieving the goal. In one of the organizations I was part of in recent past, some attempts were made to improve the design process. And in the effort to make the design team’s activities streamlined, certain design tools licenses were bought and the team members are asked to follow certain fixed process and semi-automation plugins etc. Along with this an earlier design system was revamped to provide commonly used patterns and widgets to be used. In the name of improving the process Agile model was cut-pasted and was followed. But over the period it was noticed that the overall efficiency dropped in the team. The interesting part was even though the steps to improve the design work culture implemented some of the key principles we discussed, and missed many, which led to a failed process re-engineering. The whole agile process implemented was designed to make the design teamwork as a separate entity to match the organization structure of the team as a separate business unit in order to ensure easy budgeting and determine responsibilities of the team across multiple projects running. If we review this case, we can find that the putting the design team’s own process delivery cycle didn’t contribute to the overall product delivery cycle. It conflicted with the DesOps principles like Cohesive Team-play and supporting Lean model which advocated the arching DesOps should encompass and cut-across all the teams responsible for the product and not just the design team. Also making design team as a single entity, put it into a silo because of which the feedback loop was not effective, thereby it impacted the translation of UCD philosophies.

Well, we will elaborate on similar cases in upcoming articles of this series sometime later. Now as we covered principles, let’s see in the next article, what are the practices that contribute to an idea DesOps. Hope you enjoyed the DesOps journey!

 

 

(c) 2018, Samir Dash. All rights reserved.

The 3 Dimensions of DesOps

In recent times many have attempted to define the DesOps in easier words to the community. But many of those attempts never went beyond the explorations beyond the efforts put to define a Design System. In many cases in many UX communities and teams, the definitions of DesOps is limited within the scope of Design System, which in turn essentially is a meta-product i.e. a product that enables other products. Then what is the scope of DesOps?

In my view, DesOps is not only about tools or technology employed which is generally referred to the discourses about binging automation and improved process re-engineering in the engineering sense. Rather DesOps is about defining a culture of design through improved work practices and communication among the design teams with the inter-cum-intra project/product teams and stakeholders and aiding these practices with the help of technology to bring communications among the involved tools and the arching eco-systems.

So DesOps is rather a combination of —

Designing of Design Culture and Communications + Work practices EcoSystem of Tools & Technologies 

In the Designing of Design Culture basically the design process is involved to come up with a solution that takes into account the existing gaps and areas of improvement which is needed for a specific organisation where communities or a teams involved in product life cycles, (in case of Software industry, it’s more influenced from the Software Development Life Cycle ) to improve productivity, remove wastage (in terms of effort, delivery timeline, man-power etc. ) and in many cases establish a lean process and methodologies. As DesOpsis associated with these aspects, and especially the Lean Process and methodologies, it intern impacts the culture and how the team collaborate, work and communicate.

DesOps, in other words, empowers the philosophies of the Lean UX and Fail-Fast models of start-up organisations in a better way. So the process that involves the work-culture, team – communication, product feedback loops etc. are re-designed as a part of DesOpsimplementation and this is referred as Work practices.

To aid the above two, in DesOps implementation, the focus is more on automation involving certain workflow engineering to support the re-design process (referred in the paragraph above). Basically, this 3rd aspect is more about the translation of the about two aspects with the help of technologies and building and putting in places the tools to define the required eco-system that brings seamless delivery track and complement it with DevOps through continuous and integrated delivery approach with the arching feedback loop. The use of automation in this aspect improves efficiency and scales the whole process across the whole product delivery cycle and reduces wastage.

So here is a crude way to group the different dimensions of DesOps, which come into play while modelling DesOps solution for a target organisation or team. Let’s not take them literally as many of these dimensions have overlaps among them, for example, the way we work might involve components from the cultural shift dimension.

1. Culture – The social behavior and norms that It is internally affected by both forces encouraging change and forces resisting change — which are related to structures and events within the DesOps system of humans as well as machines, and are involved in the perpetuation of cultural ideas and practices within current structures, which themselves are subject to change.

i. Principles & Practices

ii. Cultural Shift towards Lean Philosophies

iii. The way the team works (Mostly Design team but not excluding the intra-team work-culture)

2. Process – Process, primarily, a sequence of interdependent and linked procedures, is an important aspect that is made up of the workflows and the over-arching feedback-loop that acts as the spine of DesOps. Being organically associated with different functions, it involves the actions on different spheres of the operations running within the organization as well as the systems involved.

i. Work-flows

ii. Feedback- loop

3. Eco-System – Eco-System being is a regularly interacting or interdependent group of entities forming an integrated whole, deals with the technology, tools and the design systems powered with automation, that makes DesOps real on the ground. Eco-system provide the required support to the culture as well as the processes to function as well as transform.

i. Technologies

ii. Tools

iii. Design Systems

iv. Automation architectures and approaches

Stay tuned and be part of the DesOps journey with me.

 

 

(c) 2018, Samir Dash. All rights reserved.

The Maturity of Design Systems

To understand where a DesignSystem  of an organisation stands in context of implementing a DesOps, first step is to evaluate the existing DesignSystem that is in place and contributes to the organisation’s design process. (We will explore the process aspect in details, in later articles in this series.) To evaluate any DesignSystems  in a broadway we can easily form a metrics that takes care of the following two perspectives on the system.

Designer System Types

Typically the Design Systems can be broadly categorised into 3 types, namely: StaticDynamic and Generative :

Static: Most of the attributes and elements that make this system is mostly static in nature. For example in a Static type Design System, the style guide may be pre-defined print ready reference, defining basic color standards and typography etc. The user has to read through and manually refer it to decide those related attributes in his work. This kind of System mainly prescribes guidelines, rules, principles which does not automatically change or created in a dynamic way either in the stages of creation or implementation by the developers etc. Typical organisational style guides or UI pattern documentations where the system describes how and where to use the patterns with some sample code to refer are falling under this kind of categories.

Dynamic: This kind of Design Ssystem  have the content as well as the principles of implementations are designed and developed in a way that can be directly used in the code. The creation and implementation of the content are dynamic and mostly geared towards the actual elements that can directly be used in the code. This kind of Design System  is more than a reference system for the developer, rather as a part of the actual build of the actual products developed as a part of it. Most easily noticeable traits of this kind of design system  is that some special purpose frameworks, code libraries are part of it, which integrate into the actual builds of the products.

Generative: Generative Design System  are the ones, which generate the actual build ready outputs that can directly go into the build of the product. For example, instead of a static style guide, a generative Design System  can have the tool that will generate dev-ready HTML, CSS and JavaScript based output from the designer/developer inputs. The output of such system, take care of the context for which the design outputs is needed. Let’s say if the developer needs to build a cross-platform hybrid app, hen such system can generate the code that will take care of the scenarios for the interaction and behaviour for all target device resolutions, screen density as well as the behaviour for native wrappers as well as in-browser functionalities and restrictions. We will again journey into the details of the Generative Design Systems shortly.

Designer System Maturity 

The other angle to look at the Design Systems is to scale the maturity to measure how much the system has evolved. One of most important aspect of any Design System is to understand the maturity of it, as this helps to understand where it is in the overall DesOpsroadmap. Irrespective of the varied and complex categorisation of the same, we can still name the maturity as Low , Medium and High to get a quick and easy comprehension. And when we try to map the maturity, it takes care of the categorisation aspect.

Low Maturity: When the Design System has a low maturity, it mostly depends on the static attributes that we discussed above. The creation and maintenance of different attributes are mostly the result of manual effort and the most interesting point about this is that the designer and developers have the cognitive load to refer and comprehend and take decisions on what to use and not use in specific context of their work or product. It is also important, there may be some attributes which might have dynamic attributes , but most of them are out of the transition that the design system is having due to its evolution,

Medium Maturity: In the Design Systems with medium maturity, the most elements and attributes are mainly dynamic in nature. These systems mostly depend on the frameworks, libraries etc. There may be some overlaps in static and well as generative attributes.

High Maturity: Similarly in Higher maturity, apart from the fact that it mostly contains generative attributes, it involves the aspects of automation, computer-vision and may deploy artificial intelligence (AI) to provide continuous pipelines that aspires to remove the human intervention form the process blocks. On ground reality it might require the human intervention to feed in the creative juices or decision power that impacts critical human needs or contexts.

When we map these 2 perspectives horizontally and vertically, we get the the right insight into the DesignSystem’s position in the graph that allows us to clearly understand where the gaps are for the DesignSystem to evolve on which dimensions.

Note that the metrics that govern the success of a DesOps implementation is almost synonymous to this metrics we explored about Design Systems. The factors that adds to this metric on Design Systems,  includes aspect where we measure how impactful the Design System is in touching the different design process lifecycle blocks where each role like an Information Architect or an Interaction Designer , Visual Designer or even the Developer are attached to, in the delivery track. This aspect is more figuratively termed as a Living Design System. 

The Living Design System

The scaling of design is a classic issue. Moreover in recent times the explosive growth of technology across different devices, platforms and ecosystems, it became an ever-growing monster that every designer faces sooner or later. Native (Windows, Android, iOS, Linux etc. ) Web (HTML, HTML5, CSS, CSS3, JavaScript and frameworks etc. ) Along with the combination – the Hybrid – make the scaling of the design language an unending challenge.

The Salesforce design team tried to solve the challenges of applying similar designs across cross-platform product families by introducing a dynamically configurable design asset system which viewed the individual entities of any design system as design tokens.

Technically it was a single JSON file that was the “Single Source of Truth” that contained a set of name-value pairs that defined the properties and their relationships under different categories like text colours, backgrounds, border sizes, font sizes, etc. This JSON was consumed by the framework (i.e. The Lightning Designing System link: https://www.lightningdesignsystem.com/downloads/) developed and templatized for different target platforms, devices, Operating Systems etc. The Lightning Designing System framework generated different formatted outputs for CSS via common CSS preprocessors like Sass, Less and Stylus. Also there was an output in XML format that is supported in Android and JSON for iOS specific development. The Salesforce Design Tokens open-sourced at https://github.com/salesforce-ux/design-system.

The second interesting aspect of this was the use of GitHub to host the design system. Unlike the design system of traditional organisation, where the design system was hosted as downloadable form (even in the cases where the version control like Git is used) these have to be either translated into desired formats for the target platform or hosted especially along with the code. But here the design tokens representing the styles definition and the properties, as hosted on GitHub, were directly integrated into the build process contributing to the Continuous Integration and Development approach of development. In this sense, it was more as a Living System acting a single source of truth, from which the required branch is pulled and be made as a part of the build.

Many other pattern library and/design systems like RedHat’s PatternFlyhttp://www.patternfly.org is also available in similar approach at GitHub (i.e. at https://github.com/patternfly )as that of this second aspect we discussed now. But the idea of making the style guide being available as a single source of truth in a combination of this second aspect is what makes the case of the Salesforce design system unique among similar attempts for an approach to delivering a consistent design across different platforms.

 

(c) 2018, Samir Dash. All rights reserved.