John Valentine
Modular workflow
First publication22 May 2024
Edits23 July 2024

Note: I'm continuously improving this article, and intend to add diagrams and working examples.

If you're managing a practice, a manufacturing process, or a business function, you want them to perform the best they can be.

To do that, responsible people need to be confident in their work, and be agents of their own success to know when they can improve the usual way of doing things. Successful leaders guide their practice members to take action to improve.

Here, I'll advocate for continuous improvement, collaborative trust and agency for responsible colleagues, and servant leadership to guide practice. I'll also describe systems and abstract methods that support principles of good practice, and create foundations for successful digital transformation.

Contents

Approach

Aim for quality

If your objective is to produce something more than once, you'll want to achieve a minimum standard every time. This is the core of quality. Everything you do should serve good output consistently. Know when you don't, and manage failure.

Lead with specifications and standards

Standards are a statement of what you want to achieve with every item of work. If you're starting out or working towards better ways of working, standards can be aspirational, or if you're a mature practice, they're a specification for delivery.

Begin with the biggest picture in your area of responsibility. If that serves a customer need, focus on how you achieve that.

Your minimum viable product ("MVP") is what you need to output, to fulfil your customer's needs. When you list out the characteristics of everything you need to deliver, you have a standard. It's also an output specification.

Everything you do should serve delivering the output specification. To do that well every time, you'll need other people to do some things before you can begin work to deliver that. This is your input specification. For example, you might need a purchase order, a design brief, a delivery date, and some data about the customer.

[diagram]

For your area of responsibility, you need to transform inputs into outputs, using your process. Your process has "good quality" when you can validate your work, verify it against expectations, and deliver it to match the output specification.

Good quality means happy customers, provided you've set customer expectations well and defined adequate standards for delivery.

Principles of practice

Good quality isn't only about standards and workflows. It includes ways of working and values:

  • Maintain specifications and standards documentation, and adapt them as needs change.
  • A process is a documented way you reliably deliver specifications that serve customer needs.
  • Trust responsible operatives to own their process, and advocate to negotiate the contract with their suppliers to give them what they need to deliver their best work.
  • Build your information systems to support both success and failure.
  • Create a culture of transparency and honesty, where it's good to challenge current ways of working to justify exceptions, and where you can analyse system failures without blaming individuals.
  • Focus discussion to convert problems into actionable policy decisions.
  • Split standards for different specialisms. For example, in technical writing, split some aspects of style guides by audience.
  • Promote successful process within your practice, and evangelize your needs around the business, describing how it can help other business functions deliver and serve customer needs.
  • Implement change management, to involve stakeholders and communicate changes to those who need to know.
  • Optimize and automate repetitive tasks, and understand when you need to invest in tools and workflow.

Modular workflow

The module is a powerful abstraction. It provides:

  • Input specification
  • Process
  • Output specification
Process module Input Output

We can call this block a module. For workflows, you can link modules to create supply chains, or you can examine a process as a chain of sub-modules. Treat sub-modules like modules.

Chains

Create chains where the output serves the input of the next step. Output should exactly match the next input.

Process module 1 Input Output = Process module 2 Input Output

If you have mismatches between an output and its next input, then the person responsible for the second module has to do extra work to make the input conform to their needs. This causes inefficiency, and can add costs and delays.

You can merge matching input and output interfaces, to become a specification. The output of Process 1 conforms to the specification, and Process 2 needs input that complies with the specification.

Process module 1 Input Process module 2 Output Specification

Merge workflows

Where a module needs to wait for two supply chains, you can compose specifications into the input.

Process module 1 Input Process module 2 Input Process module 3 Output Specification Specification

Subflows

You can expand a module into a chain of its own, where the input of the first submodule matches the input of the process, and correspondingly, output of the last submodule matches the output of the process.

Process Input Output Sub-process 1 Input Sub-process 2 Specification Sub-process 3 Output Specification = =

Level of detail

You can fall into a trap, of creating subworkflows to a fine level of detail. Sometimes that's not necessary. Where it might be necessary:

  • You can match modules with operators who are qualified to perform a task.
  • Work passes from one person to another between modules, with seemless handover. You need accurate and accessible data about all relevant objects and state. State is most well-defined when the output specification of a module is fulfilled, so these are ideal status points in digital workflows.
  • You have quality problems, and want to check that operators perform modules of work to specification.
  • You need accountability and visibility of quality.

Workflow diagrams

This article has mostly covered workflow architecture. Users might better understand a simplified workflow diagram or flowchart.

You can show workflows by privileging status or by privileging process. I prefer the convention where I represent:

  • Process modules as boxes
  • Actions as labels on directed lines between process boxes
  • Branching decisions as boxes like modules, but different shape, with labelled outcomes on the directed lines (like the flowchart convention)

I arrange the workflow in a grid with dimensions:

  • Horizontal: status, showing progress from left to right. You can also use group spans to identify broader status groups and simplify the layout.
  • Vertical: the role within the organization, of a person responsible for completing a module.

[diagram]

The vertical crossings between bands identify points where critical communication keeps the work flowing.

For a complete workflow, add:

  • Exceptional subworkflows that fix failure or infrequent events.
  • External dependencies like communications and approvals.

Abstractions and diagrams

Directed graphs

For digital transformation, you need data architecture that mimics the state of the workflow. In trivial systems, where the workflow is linear and without branches, you can represent state as a single value.

For more complicated systems, with more than one concurrent workflow path, a single status doesn't give you the full picture, unless you define milestones on the main path that the other paths converge upon. For detailed progress reports, you need to store completion state for each module, and report on the progress of each possible path.

Directed graphs let you navigate a workflow. For each module, you maintain a list of dependent modules. You can have more than one dependent module for any given module, and one module can enable many others to start.

Relational database

That's a many-to-many relationship, which you can represent with a linking table.

ModuleMust-do moduleAsynchronous*
31false
32false

*optional

To implement sub-workflows with asynchronous inputs, you can add a Boolean asynchronous property to the above table to indicate that an input is not required yet, and allow work to start when all other asynchronous = false dependent modules are complete. If you're implementing service level agreements (SLAs), you might want a module for resolving this external input, as a start trigger for SLA events.

Object models

Store the above must-do structure as an array of records in the workflow scope. Alternatively, with more traversal work but better object containment, you can link between modules with linked list patterns, where you store an array of foreign module identifiers in each module.

Templates and instantiation

So far, we've described a workflow as an directed set of instructions and intermediate targets, to achieve a result at the end. This is like a template. When it comes to doing the work, we'll need a fresh copy of the template, so we can fill in details as we go. The instantiated copy is a stateful record of work, and we store it for future analysis.

Mathematics and category theory

Process modules are:

  • functors that mutate the state of the reflected system. Formally, they're not contravariant (reversible), so reversion to a former state needs an explicit functor (process).
  • associative and composable in chains with other modules, where the output specification is isomorphic to the next functor's import specification. Constrain them with a directed graph.

Other definitions

  • Objects are the workflow specifications (parameterisations of inputs and outputs), or resting points between process modules.
  • The collection of all objects in the workflow is a set.
  • Morphisms are modules. We don't use identity morphisms.
  • A single workflow state cursor is possible only for trivial linear workflows. For asynchronous and concurrent workflows, the cursor paradigm needs more than one cursor. To query progress, it's better to navigate the graph for the local properties in each object.
  • A functor only generates the next workflow state when business logic constraints are met.