Category Archives: Methodology

Modelling HPM as a PSM, using HPM

I conclude my book on Problem Structuring with some comments on a processual turn in Operational Research (Yearworth, 2025) and specifically comment

“In modelling the world processually we can also model our interventions within the same model … Or put more simply, problematic situations are processes as are the means of intervention.” (p. 270). 

Modelling our interventions specifically requires the possibility of representing our use of a problem structuring method as a model. Checkland and Poulter described the process of using Soft Systems Methodology (SSM) in the activity system ‘language’ of SSM itself (Checkland & Poulter, 2006, p. 194; Yearworth, 2025, p. 78). Checkland and Scholes went further and modelled the system to use SSM in the same purposeful activity system language i.e., the ongoing reflective practice of using SSM in client engagements (Checkland & Scholes, 1990, p. 294).  

The same approach has been used for describing the use of Hierarchical Process Models (HPM) for problem structuring. This was first manifest in the STEEP Project as means of self-evaluation of how well the methodology was performing, making use of the Italian Flag as a means of capturing judgement of process performance (Yearworth et al., 2015, p. 9). In the Healthy Resilient Cities project (Yearworth, 2015), we started to model the process of using the PSM within the model of the problematic situation itself (Yearworth, 2025, p. 173) i.e., the process <Improving the resilience of healthcare provisioning in Bristol…> contained within it the process <Using problem structuring>. Further work on exploiting the Italian Flag for capturing judgements of process performance in the use of a PSM was explored in depth by Lowe, Espinosa and Yearworth (2020).

The development of HPM as a PSM is described fully in Chapters 9 and 10 of my book. However, the work of modelling HPM as a PSM using HPM itself that was started in the STEEP project is still ongoing. The following shows its current incarnation in Strategyfinder.

HPM of using HPM as a PSM

Note that methodological learning, an essential element of using a PSM, is reflected in the model at Process #26 <Evaluating the engagement and improving our understanding …> and the processes it contains. The model also references other models that could be incorporated as enhancements, for example using the framework for improving facilitation developed by Ackermann (1996, p. 95), which has been interpreted in my book as another process model (Yearworth, 2025, p. 108). This illustrates the property that all HPM are composable according to their necessity and sufficiency for the success of the process that acts as the anchor for incorporation.

Ackermann, F. (1996). Participants’ perceptions on the role of facilitators using group decision support systems. Group Decision and Negotiation, 5(1), 93-112. https://doi.org/10.1007/BF02404178

Checkland, P., & Poulter, J. (2006). Learning for action : a short definitive account of soft systems methodology, and its use for practitioner, teachers and students. John Wiley & Sons: Chichester. 

Checkland, P., & Scholes, J. (1990). Soft systems methodology in action. John Wiley & Sons: Chichester. 

Lowe, D., Espinosa, A., & Yearworth, M. (2020). Constitutive rules for guiding the use of the viable system model: Reflections on practice. European Journal of Operational Research, 287(3), 1014-1035. https://doi.org/10.1016/j.ejor.2020.05.030

Yearworth, M. (2015). Healthy Resilient Cities: Building a Business Case for Adaption (NERC NE/N007638/1)[Grant]. Bristol. http://gotw.nerc.ac.uk/list_full.asp?pcode=NE%2FN007638%2F1

Yearworth, M. (2025). Problem Structuring: Methodology in Practice (1st ed.). John Wiley & Sons, Inc.: Hoboken. https://doi.org/10.1002/9781119744856

Yearworth, M., Schien, D., Burger, K., Shabajee, P., & Freeman, R. (2015). STEEP Project Deliverable D2.1(R2) – Energy Master Plan Process Modelling. STEEP PROJECT (314277) – Systems Thinking for Comprehensive City Efficient Energy Planning, pp78. Retrieved 26th January 2023, from https://www.grounded.systems/wp-content/uploads/2023/01/01_STEEP_D2.1_Energy_Master_Plan_process_model_update_M24_DEF_sent.pdf

Augmented Qualitative Analysis (AQA) and Large Language Models (LLMs)

Conducting experiments in the automated labelling of topics generated using the Augmented Qualitative Analysis (AQA) process outlined in an earlier post has resulted in some observations that have some bearing on the use of Large Language Models (LLMs) in Soft OR/PSM practice.

The starting point for AQA was the partial automation of some of the elements of qualitative analysis, which has resulted in the use of probabilistic topic models to code text in a corpus at the paragraph level and to produce maps of the interrelationship of concepts (qua topics). These maps of interrelationships can only be put forward for consideration as potentially causal links after the topics have been labelled. A map of topic X being linked to topic Y n times is only of statistical interest. We need meaning to be attached to the topics – ideally a process of consideration by a group that ‘owns’ the raw data – before we can produce a putative causal loop diagram (CLD).

To ground this technique in traditions of qualitative analysis would require the labelling of topics to proceed through an inductive process of inspecting the term lists and inspecting the text coded by the topics to build-up an understanding of what the topic means to the stakeholders (e.g., see Croidieu and Kim (2017)). This is a back-and-forth process that continues until all the topics have been labelled. The fact that the map can be updated with these topic labels in parallel provides an additional perspective on the understanding of the meaning of the topics. 

With the advent of LLMs it is possible to feed the term lists and the example text coded by a topic – and even the map of interrelationships – into a tool like ChatGPT with the purpose of generating topic labels. However, experiments in doing this have produced disappointing results, despite extensive efforts in refining prompts. From the perspective of a qualitative researcher, the coding seems to be too much in the text, too in-vivo. Despite attempts to get the LLM to draw on the breadth of its training data there seemed little evidence of the sort of theorising from the data that is a key feature of qualitative analysis (Hannigan et al., 2019).

This is clearly a new area and other researchers have conducted experiments on precisely this point of prompt engineering e.g., see Barua, Widmer and Hitzler (2024). However, there is still the sense that a LLM is operating as nothing more than a ‘stochastic parrot’ (Bender et al., 2021). Further, coupling the outputs from a probabilistic topic model to a LLM are unlikely to generate the sort of management insight that is discussed by Hannigan et al. (2019); although the putative causal maps are likely to make sense to the participants in a group, and are statistically justified. Ultimately, the use of LLMs in a process of problem structuring is only ever going to be limited. Problematising is a human activity, it requires a felt-sense of a situation being problematic for an intent to intervene to emerge. Asking a LLM to feel something is a wayward expectation.

The recommendation here, for any group working with a large and potentially growing corpus of documents and in need of a technique that supports rapid problematisation, is to work with two Group Support Systems (GSS). The first presents an interactive means of exploring the probabilistic topic model (e.g., using pyLDAvis the topic model for the 2012 Olympics data set discussed in a previous post can be explored here) combined with a means of investigating the text as coded by the topic model i.e., selecting text that is coded by a specific topic and inductively generating a topic label that has meaning to the group. In effect, replicating some of the features of a Computer Aided Qualitative Data Analysis Software (CAQDAS). The fully labelled model can then be taken into a strategy-making workshop supported by the second GSS, in this case Strategyfinder. 

The prospects of these two GSS merging into a single Problem Structuring Platform is the subject of my upcoming talk at OR66, see my previous post on AQA.

Barua, A., Widmer, C., & Hitzler, P. (2024). Concept Induction using LLMs: a user experiment for assessment. https://doi.org/10.48550/arXiv.2404.11875

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? FAccT 2021 – Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, https://doi.org/10.1145/3442188.3445922

Croidieu, G., & Kim, P. H. (2017). Labor of Love: Amateurs and Lay-expertise Legitimation in the Early U.S. Radio Field. Administrative Science Quarterly, 63(1), 1-42. https://doi.org/10.1177/0001839216686531

Hannigan, T. R., Haan, R. F. J., Vakili, K., Tchalian, H., Glaser, V. L., Wang, M. S., Kaplan, S., & Jennings, P. D. (2019). Topic modeling in management research: Rendering new theory from textual data. Academy of Management Annals, 13(2), 586-632. https://doi.org/10.5465/annals.2017.0099

Augmented Qualitative Analysis (AQA)

I first started working on the semi-automated construction of Causal Loop Diagrams (CLDs) as part of a process of qualitative analysis for my MBA project. This work was developed into a full paper and published in the European Journal of Operational Research in 2013.

Yearworth, M., & White, L. (2013). The Uses of Qualitative Data in Multimethodology: Developing Causal Loop Diagrams During the Coding Process. European Journal of Operational Research, 231(1), 151-161. https://doi.org/10.1016/j.ejor.2013.05.002 [post-print version]

Judging by the citations, the methodology described in this paper has been used to develop CLDs across a wide range of application areas.

At the time of writing, the methodology was supported by the use of conventional Computer Aided Qualitative Data Analysis Software (CAQDAS) and, specifically, the use of matrix queries to compute the number of times pairs of concepts (codes) are related by the fact that they co-code paragraphs of text in the sources. The resulting adjacency matrix could be interpreted as a preliminary CLD and an input to further analysis.

Since then, the emergence of probabilistic topic modelling based on the Latent Dirichlet allocation (LDA) means that it is now possible to automate the coding process for very large collections of documents (hundreds to thousands), comprising millions to hundreds of millions of words. The co-coding technique described in Yearworth and White (2013) can be similarly applied by feeding the corpus into the topic model and counting and thresholding the resultant classification of paragraphs by topic(s). This process still requires an inductive1 bridge – in deciding i) a meaningful number of topics (k) to be used for the topic model, ii) the labelling of the topics based on an exploration of the term lists and a re-reading of the text coded by each topic, and iii) a decision about the meaning of the links – can they be interpreted as causal? In practice the last two processes are bound together and interactive once a topic model of particular size has been chosen.

An example graph is shown below based on a corpus of documents assembled from archival material about the 2012 London Olympic Games. The corpus is relatively small for a machine learning technique, 170 documents and just over a million words. However, this is of the sort of volume of data that is starting to get beyond the abilities of a single qualitative researcher to analyse. I’ve called this territory a hinterland for qualitative analysis and hence the motivation for an augmented approach. The graph presented here has been coloured according to betweenness centrality.

The graph was automatically translated into the JSON format supported by the Strategyfinder platform and is shown as a potential causal map below. Once imported into Strategyfinder it is then a further process to discuss the meaning of the relationship between statements i.e., the nature of the links, whether they represent a causal relationship, and their directionality. An export filter to create MDL files suitable for import into Vensim also exists.


I discuss this technique further in Chapters 14 and 15 of my forthcoming book Problem Structuring : Methodology in Practice in the context of a deployment of the technique via a problem structuring platform as a type of Group Support System (GSS). This is also the subject of my upcoming talk at OR66.

Annual OR Conference OR66
Details of the my talk at OR66
  1. I’ve labelled this an inductive step because at this stage in a process of problem structuring this is what it feels like you’re doing. However, situated inside a wider problem structuring process loop that includes modelling and taking action then it could be considered as abductive reasoning. ↩︎

Problem Structuring : Methodology in Practice

My new book is now available!

Current perspectives on approaches to problem structuring in operational research and engineering and prospects for problem structuring methods applicable to a wide range of practice.

Despite the myriad successes of Operational Research (OR) in government and industry, critique of its continued relevance to complex, wicked problems led to the emergence and evolution of Soft OR as a more humanist orientation of the discipline centred on a methodological framing of techniques known as Problem Structuring Methods (PSMs). These have enabled OR practitioners to broaden the scope of OR to address complex problem contexts that require transforming, planning and strategising interventions for their clients. The original core PSMs of Soft Systems Methodology (SSM), Strategic Options Development and Analysis (SODA) and the Strategic Choice Approach (SCA) are presented using a new analytical framework based on constitutive rules, epistemologies, and affordances of the modelling approach. Practical considerations in PSM based interventions are discussed emphasising trust-building, stakeholder identification, facilitation and ethical practice. A wide range of PSM applications are surveyed demonstrating clear intersections with communities of practice grounded in the applied social sciences. The development of a new PSM based on Hierarchical Process Modelling (HPM) of purpose arising from a processual turn in engineering practice offers additional insights for the practice of Soft OR. New developments in PSM practice built on use of Group Support Systems (GSS) and exploiting developments in machine learning are presented. Prospects for bringing the Soft OR project back into better alignment with mainstream OR are discussed in the context of new education programs and a possible processual turn in OR.

Problem Structuring: Methodology in Practice contains four linked sections that cover:

  1. Problem formulation when dealing with wicked problems, justification for a methodological approach, the emergence of soft OR, the relevance of pragmatic philosophy to OR practice.  
  2. Traces debates and issues in OR leading to the emergence of soft OR, comparative analysis of PSMs leading to a generic framework for soft OR practice, addressing practical considerations in delivering PSM interventions.
  3. Charts the emergence of a problem structuring sensibility in engineering practice, introduces a new PSM based on hierarchical process modelling (HPM) supported by teaching and case studies, makes the case for a processual turn in engineering practice supported by HPM with relevance to OR practice.
  4. Evaluation of PSM interventions, survey of applications, use of group support systems, new developments supported by machine learning, re-contextualising soft OR practice.

Problem Structuring: Methodology in Practice is a thought-provoking and highly valuable resource relevant to all “students of problems.” It is suitable for any UK Level 7 (or equivalent) programme in OR, engineering, or applied social science where a reflective, methodological approach to dealing with wicked problems is an essential requirement for practice.

Note that some institutions may have access via their usual eBook provider (e.g. ProQuest, Perlego), if not by default on the subscription then by a separate order for this specific title.

Wicked problems and category mistakes

Wicked problems and category mistakes

This is a brief introduction to the notion of a wicked problem. It is based on the highly-cited paper by Rittel and Webber (1973). The following characterise wicked problems:

  1. There is no definitive formulation. In a sense, formulating a wicked problem is the problem
  2. There are no stopping rules. The process of intervening is also the same as understanding the nature of the problem – the intervention is “good enough” or the best that can be achieved within other limitations (e.g. of time, budget…)
  3. Interventions are not right or wrong, they can only be viewed as making things better or worse for certain interests i.e. the intervention has made things both better and worse depending on who you ask
  4. There is no immediate or ultimate test of an intervention. Interventions will generate “waves of consequences” over a period of time
  5. Interventions are “one-shot operations”, experiments are difficult to conduct, every intervention counts significantly, they are essentially unique in nature
  6. No enumerable, exhaustively describable, set of possible interventions
  7. Every wicked problem is essentially unique. “Essentially” implies that aspects may be common, but to think in terms of categories or “classes” of wicked problems with common “solutions” is misleading
  8. Wicked problems can be considered as symptoms of other problems i.e. there is inherent systemicity in the world
  9. Can be contested at the level of explanation, there is likely to be conflicting evidence or data

The corollary of this definition is that certain statements about problems are likely to be rendered false or meaningless if it can be shown that the problem is actually wicked, in effect the statement is demonstrating that a category mistake is being made. The following is not an exhaustive list:

  1. ‘Solving’ or ‘curing’ a wicked problem is a contradiction; there are no ‘solutions’, ‘cures’…
  2. Words that suggest an objective point of view used in the context of the problem at the very least need to be debated e.g. words like optimal, best, right, smart, correct, … all suggest the question – for whom? Alternatively, no decision taken should ever be considered wrong.
  3. Any statement of measurable quantity that supports an argument for the problem getting better or worse without acknowledging the dynamic complexity that systemicity implies i.e. “…worse then better…” is a more believable statement given dynamic complexity
  4. Statements that appear to deny the systemic nature of the problem e.g. ignoring requisite variety
  5. Containing irrefutable assertions of fact e.g. “…this proves conclusively that…”
  6. Use of binary choices, any mention of “silver bullets”
  7. Misrepresenting or ignoring plurality e.g. “The public…”
  8. Emphasis on producing plans rather planning as a process

If any of these corollaries are contested e.g. if someone claims to have a solution to a wicked problem, then they are likely to be making a claim about only an aspect of the problem, or only from a certain viewpoint; or their formulation is not that of a wicked problem i.e. they are talking about something ‘tame’. Statements that contain phrases like “…optimal solution…” or “…this proves conclusively that if we do this we will have the best outcome…” in the context of a wicked problem definitely signal a likely category mistake.

Category mistakes are a warning sign – be sceptical of claims being made. They suggest either misunderstanding or partiality.

It’s worth reading the Rittel and Webber paper. Despite its age, it still does an exceptionally good job of reminding us of the characteristics of wicked problems that’s just as relevant today.

The first steps towards a coherent approach to problem formulation can be found in Rosenhead’s (1996) introduction to Problem Structuring Methods.

Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155-169. doi:10.1007/BF01405730
Rosenhead, J. (1996). What’s the problem? An introduction to problem structuring methods. Interfaces, 26(6), 117-131. doi:10.1287/inte.26.6.117

A short guide to System Dynamics

A short guide to System Dynamics

This guide was produced to help explain System Dynamics modelling to a group of interested stakeholders for a modelling workshop. However, if you have the time I would really recommend reading John Sterman’s textbook for a definitive account:

  • Sterman, J.D. (2000). Business dynamics : systems thinking and modeling for a complex world. Boston: Irwin McGraw-Hill.

Otherwise this short paper summarises the key points

  • Sterman, J.D. (2001). System dynamics modeling: Tools for learning in a complex world. California Management Review, 43(4), pp. 8-25. doi: 10.2307/41166098

And if neither are available, or time is really short, then try this SD-Introduction-MY-20200527.pdf from me.

Hard Systems and Soft Systems

Hard Systems and Soft Systems

A frequent problem I come across when discussing hard and soft systems views with engineers is that the terms ‘hard’ and ‘soft’ are rarely defined clearly. Based on conversations I’ve had over the years at the University of Bristol a common position can be characterised by the statement “all hard systems are embedded in soft systems.” I used this myself in a CSER conference paper in 2011 when talking about the EngD in Systems programme where I teach. However, since then I have arrived at the position that the epistemic shift that Peter Checkland and Susan Holwell describe is a more useful way of characterising hard and soft systems views [1]. Instead of the rather vague association of soft with the social world, people, and human intentionality, the soft systems view moves away from this ontological commitment and treats the definition as a question of epistemology, i.e. what can we know or find out about the world? The following quote from [1] spells out this epistemological position in a way that I find compelling. It is:
“phenomenologist, social constructivist, avoiding ontological commitment – sees the perceived (social) world as: culturally extremely complex; capable of being described in many different ways; and sees the “system” as one useful concept in ensuring good-quality debate about intentional action. The two observers both agree that the notion “system” can be useful, O seeing it simply as a name for (parts of) the real world, E seeing it as a useful intellectual device to help structure discussion, debate and argument about the real world.”
Where observer O corresponds to the ontological position and observer E to the epistemological. This is all usefully summarised in a table that I use with my students adapted from the original in [1]:
Hard and Soft Systems Viewpoints

Checkland and Holwell’s paper appears in a volume edited by Michael Pidd [3], which brings together the ideas developed in the Interdisciplinary Research Network on Complementarity in Systems Modelling (INCISM) Network that was funded by the Engineering and Physical Sciences Research Council (EPSRC).

John Morecroft was part of the network too and in his work on System Dynamics modelling [2] reflects on how it should be used in this soft systems sense. He paraphrases Checkland to state “… system dynamics modellers do not spy systems. Rather they spy dynamics in the real world and they organise modelling as a learning process, with the project team, to discover the feedback structure that lies behind the dynamics“.

Reflections on this hard/soft complementarity and the work of the INCISM network at the System Dynamics conference in 2004 are captured in the notes from the record of the plenary session Working Ideas, Insights for Systems Modelling: The Broader Community of Systems Thinkers.

All of this is neatly summarised by Peter Checkland himself in a colloquium delivered at Lancaster University in 2012. In this short video, Checkland outlines the development of Soft Systems Methodology emphasising its role as methodology, not method, and the origin of this particular definition soft systems as the systemic learning system designed to help us deal with the complexity of the world.

[1] Checkland, P., & Holwell, S. (2004). “Classic” OR and “soft” OR – an asymmetric complementarity. In M. Pidd (Ed.), Systems Modelling: Theory and Practice. Chichester: John Wiley & Sons, Ltd.
[2] Morecroft, J.D.W. (2007). Strategic modelling and business dynamics : a feedback systems approach. Chichester : John Wiley & Sons, Ltd.
[3] Pidd, M. (2004). Systems modelling : theory and practice. Chichester: John Wiley & Sons, Ltd.

Systems modelling in engineering

Systems modelling in engineering

The wider and more pervasive use of appropriate systems modelling techniques would have a beneficial impact on the way in which engineers deal with messy socio-technical problems. This class of problems is commonly defined by the following characteristics; i) difficulty on agreeing the problem, project objectives, or what constitutes success, ii) situations involving many interested parties with different worldviews, iii) many uncertainties and lack of reliable (or any) data, and iv) working across the boundary between human activity systems and engineered artefacts. All systems models attempt to conceptualise, via appropriate abstraction and specialised semantics, the behaviour of complex systems through the notion of interdependent system elements combining and interacting to account for the emergent behavioural phenomena we observe in the world.

Engineers have developed a multitude of approaches to systems modelling such as Causal Loop Diagrams (CLDs) and System Dynamics (SD), Discrete Event Modelling (DEM), Agent Based Modelling and simulation (ABM), and Interpretive Structural Modelling (ISM) and these are all included in my programme of research.   However, despite their extensive use, there still exists a number of research challenges that must be addressed for these systems modelling approaches to be more widely adopted in engineering practice as essential tools for dealing with messy problems. These systems modelling approaches as used in current engineering practice provide little or no account of how the process of modelling relates to the process of intervention (if any). This is in part due to the wider challenge to address the poor awareness and uptake of Problem Structuring Methods (PSMs) in engineering, the current inadequate way of integrating these more engineering-focussed systems modelling approaches into PSMs, and lack of understanding in how to deploy them appropriately in addressing messy problems in specific contexts. There is also the need to interpret the current state of the social-theoretic underpinning to systems modelling into a form that is appropriate for use in engineering. This need arises from the endemic atheoretical pragmatism that exists in engineering practice. The lack of methodology supported by suitable theory to counter this i) hinders the development of understanding why methods work or not, and also what it means for them to work, ii) acts as a barrier to communication between practitioners and disciplines, and iii) has ethical consequences, as pragmatic use of methods raises the problem of instrumentalism.

Addressing this methodological challenge is currently a central core of my work. I believe this research is transformational in that it integrates academically disparate areas of expertise in engineering, management, and social science, into a coherent articulation of systems modelling for engineers.