WorkflowSprints.md

title: "Understanding and Implementing Workflow-first Design (Workflow Sprints)"
subtitle: "Why you should ditch the technology-first lens and think about workflow."
author: Adam Hyde 
date: "2022-09-27"
category: "article"

Technology! Workflow!

In this article I’ll be looking at a critical and under-examined step required to build publishing technology - optimising workflows.

I am in the business of building publishing technology. However, I actually believe the core of what I do is designing and optimising publishing workflows. Technology is just an (optional) outcome of that process.

To get to good technology, it is very important to start the process by not thinking about technology at all. If you frame your thinking through the lens of technology, you will more than likely end up with an expensive failure. What we are really interested in are good workflows and, later, designing technology to encapsulate and enable good workflow.

To help us understand this problem we need to define workflow as it means many different things to different people - by workflow, I mean the complete set and order of actions an organisation (usually a publisher) undertakes to prepare an object (book, journal article, preprint review, question set etc) for publishing or sharing.

For example, workflow is everything that is done from start to finish that falls within a publisher's control, to prepare a book or journal article for publication. Or, every action required by a community to prepare and share reviews on preprints.

Note: I use the term ‘publisher’ very liberally (see definitions at bottom).

What publishers want is to improve their publishing processes, but they tend to think purely in terms of technology as both the problem and the solution. As a result of this ‘technology first’ thinking the process for many publishers looks like this:

1. Choose a new technology
2. Use the technology

It is worth noting that there are some good motivations for wanting to change technology which do not originate in trying to improve workflow. Liberating an organisation from expensive licensing of proprietary technology, for example, is an increasingly common driver.

Replacing unstable legacy technology is also a good motivation.

However, regardless of the motivation, if you are determined to change technology, you should also see this as an opportunity to improve how you work - to improve your workflow.

As mentioned above, the “technology first” approach can be problematic. Three common outcomes of technology-first thinking are:

1. You can make an expensive technology change without improving the publishing process.
2. You may create new problems and poorer outcomes for the publishing process.
3. A ‘rush to adopt’ can come with unforeseen tradeoffs that later may prove very costly.

These undesirable outcomes are better avoided by adopting a workflow-first framework.

For new entrants and incumbents alike, a workflow-first process looks like this:

1. Understand the current (or speculative) workflow
2. Optimise the workflow
3. Design technology to enable the workflow
4. Build (or choose) the technology (optional)

In these two articles we will look at steps 1 and 2 only. Steps 3 and 4 might require a thesis.

Step 1: Understand the current workflow - is necessary to generate an understanding of what you are actually doing right now. This step needs to drill down into the nitty gritty of the daily workflow so you can get a complete picture of what actually happens ‘on the floor’. High level executive summaries are not what we are after. We need to know what actually happens.

Without understanding exactly what you are doing now, improving what you do will be based on false assumptions. As poet Suzy Kassem has said:

"Assumptions are quick exits for lazy minds that like to graze out in the fields without bother."

When examining workflow we need to bother people. To ask hard questions and get to the bottom of things. I’ve often heard folks say they know their workflow, or someone within their organisation knows the workflow. But after asking a lot of exploratory questions I inevitably find that no one person understands exactly what happens at all stages of the workflow. Consequently, understanding your current workflow, to the granularity required to move to the next steps, requires a process in itself. But we will discuss this in more detail in the next article.

However Step 1, while important (and important to get right), is just prep for the most critical step of the entire process:

Step 2: Optimise the workflow Optimising the workflow is what every publishing technology project should aim to do. If you don’t optimise the workflow, if you haven’t improved what you do, then what have you achieved? It would make sense then, to prioritise thinking about improving workflow before you start to think about technology.

Optimising the workflow is, in itself, a design problem and requires dedicated effort. Unfortunately this step is often skipped or paid only a passing tribute. Conflating steps 2 and 3, for example, is typical of the popular publishing RFP (Request for Proposals) process (see https://upstream.force11.org/can-we-do-away-with-rfps/) :

1. Understand the current (or speculative) workflow
2. Design technology to enable the workflow
3. Build (or choose) a new technology (optional)

An RFP process may be motivated by a few big ticket items that may need improving (or licensing issues as per above), but often lacks a comprehensive design process aimed at optimising the workflow. Instead, technology is chosen because it fulfills the requirements of the current workflow audit (Step 1) with a few possible fixes - this is because most RFP processes focus on updating technology rather than improving publishing processes. Consequently there are minimal, if any, workflow gains and a failed opportunity to identify many optimisations that can deeply affect a good technology choice and improve how you publish.

To avoid these problems and to improve how you publish, it is really necessary to focus dedicated design effort to optimising workflow.

The kind of process I’m talking about was once brilliantly described to me by a friend Martin Thompson (Australian Network for Art and Technology) a long time ago. Martin was explaining to me the principles of video compression for streaming media, but the principles can be applied to workflow optimisation. Imagine a plastic bag full of air with an object in it (let's say a book). The air represents all the time and effort required to produce that book. Now suck out the air, and we see the bag and book remain the same but the volume of air has reduced. We have reduced the effort and time to produce the book, while the book remains unchanged.

This is a simple but clear metaphor for what we are trying to achieve. In summary, optimising workflow is about saving time and effort while not compromising the end product.

Workflow optimisation requires an examination of every stage of the process (as discovered in step one of our workflow-first process) and asking if that stage can be improved or eliminated. We are also asking some other questions such as -

• Can anything be improved to help later stages be more efficient?
• Are there ‘eddies’ in a workflow that can be fixed?
• Can operations occur concurrently rather than sequentially?

When asking these questions we will discover many efficiencies and we can slowly start drawing out a complete improved workflow. Some steps will disappear, some will be improved by mildly altering the order of operations, others will be improved by small tweaks. Some changes are more significant and may require greater change.

Once we have been through every stage looking for efficiencies, once we have our complete and optimised workflow, we are ready to start thinking about what we need to enable these changes. Technology may be part of the answer, but it is never the only answer, and in some cases it is not the answer at all. If we do decide new technology is required then only now should we start on the path to designing or choosing the tools to support our optimised workflow.

Next we will look at what a methodology for understanding and optimising workflows can look like - Workflow Sprints.

What is a Workflow Sprint?

A Workflow Sprint is a fast and efficient methodology for helping organisations understand and optimise their publishing workflows before making technology choices.

I designed the methodology after witnessing many publishers making these common mistakes:

1. Investing in slow and costly ‘requirements research’
2. Looking at the problem exclusively through a ‘technology lens’

Understanding and optimising workflow need not be a slow or expensive process. Too often, these processes are lengthy research projects that interview individual team members and produce immense amounts of narrative and flow diagrams. Such outputs are seldom clarifying. On the other hand, Workflow Sprints produce crisp clarity in a process that can be counted in days, not months or years.

A Workflow Sprint first investigates and describes the current workflow and then produces an optimised workflow. From these outputs, your organisation will be in a good position to consider technology choices.

Principles of Workflow Sprints

Workflow Sprints are a series of facilitated sessions that can occur remotely or in real space. There are some fundamental principles of Workflow Sprints:

Principle 1: Operational Stakeholders are best placed to describe and optimise workflow
The best people for describing an existing (or speculative) workflow and optimising that workflow are the people actually doing the work now - your team (operational stakeholders - more on this below).

Principle 2: Group discussion simultaneously expedites the process and adds clarity
Group discussion involving the operational stakeholders is the most effective way to understand and optimise workflow. Having these experts in the same room eliminates guesswork and team members can drill down into an issue together in quick time to tease out the fidelity needed.

Principle 3: Facilitation is key
Experienced facilitation conducted by someone external to your organisation is critical for keeping the momentum going and managing any potential and problematic team dynamics.

Principle 4: Keep the documentation light
If the documentation of the workflow is not something decision makers can easily grasp, then you are doing it wrong. For this reason, I have designed Workflow MarkUp (WFMU) to aid in expediting the documentation process while producing clear, human-readable documents.

Now that we have the principles covered let’s take a brief look at what a Workflow Sprint actually looks like.

What does a Workflow Sprint look like?

Photos included are from actual Workflow Sprints (arXiv, Open Education Network, Ravenspace, Érudit, Organisation for Human Brain Mapping, Wormbase/Micropublications.org).

A Workflow Sprint is a facilitated, structured conversation between the organisation’s primary stakeholders. The stakeholder group should comprise those people that are part of the current workflow (constituting all parts of the workflow) and decision makers.

Operational stakeholders are the people who do the work. These are the most important people to have involved because they are the ones that understand what is done, on the floor, at this time. Sometimes, if automation is involved, a trusted technician that knows the limits and possibilities of publishing technology might be useful. No Workflow Sprint can occur without operational stakeholders.

Strategic stakeholders are the organisation's decision makers. The decision makers may, or may not be part of the process, however, they will certainly utilise the outcomes to guide them in making decisions going forward. It can be a good idea to include strategic stakeholders to build internal buy-in.

The Workflow Sprint draws these two types of stakeholders together in a series of remote or realspace sessions. If held in realspace the event can occur over one day. For remote events (which we have found to be more fatiguing), the Sprint should be broken down into multiple shorter sessions over several days.

The facilitator guides the group through a process designed specifically for your organisation. The steps usually include (discussed in more detail below):

1. Introductions
2. High-level description of the goal
3. High-level discussion
4. Documentation of the existing workflow
5. Open discussion
6. Designing an optimal workflow
7. Report (optional)

Each phase of the above process should be fun and characterised by a high level of engagement from the team.

Simple tools such as whiteboards (or ‘virtual whiteboards’ if remote), large sheets of paper, and markers are often used to draw out existing workflow processes or future workflow ideas.

Workflow Sprints are fun, efficient, cost-effective, and fast. The process is a good way to generate additional internal buy-in for a project and to quickly help everyone understand the broader goals and challenges.

We have found that Workflow Sprints are also extremely effective team-building events. If held in realspace, we advocate good food and coffee! If done remotely, we sometimes include some less formal moments to help people relax and get into the flow of the event.

Now let's look at the process in more detail. The following is not comprehensive, but it should give you a good starting point. If you want to drill down further into the process, feel free to contact me.

Speculative vs Extractive Workflow Sprints

Before we get started, a quick word about two types of Workflow Sprints that I have already hinted at throughout this document. The two types are:

Speculative - Speculative Workflow Sprints imagine new workflows. Typically new entrants or start ups require speculative sprints. Inevitably the speculative variety can reveal a lot of contradictory requirements and ‘blurry vision’ that need to be discussed and examined in detail. Sometimes, the outcomes may also lead to compromises the stakeholders may find difficult to accept. Consequently, Speculative Workflow Sprints often take longer and require a higher degree of expert facilitation.

Extractive - Extractive Workflow Sprints start by examining a known (existing) workflow. Generally speaking, these descriptive processes are easier to facilitate and quicker to document.

Realspace vs Remote