Red Hat Summit 2019 — BPM

At Summit 2018 one of the main focuses in the RH Process Automation Manager presentation was personas. They came up with several personas or user profiles, that were believed to be the main users of PAM, especially Business Central.

I’ve always wondered where they came up with these personas. A marketing study would have been wise. I think it would have showed that while many enterprises intend to have business users develop and maintain processes and rules, it almost always ends up being an IT function.

End users like case managers really need a custom interface built for their specific operation. The Business Central experience isn’t really optimal for that.

At Summit 2019 I was gratified to see that jBPM is headed in a direction that will optimize the experience for the developer. I saw demos of building process and business rule apps using Visual Studio Code using the Quarkus framework. Although not quite ready for general use, they have a stand-alone bpmn editor that is a plugin to VSC. That along with the Quarkus will make jBPM development a real joy.

Now we can interact with a standard git repo and host it on github or elsewhere and not be held back by the limited support in Business Central. The coding environment on Business Central was never a passable experience even for writing short scripts. Using VSC or any other IDE is so much better. Yes, there is a BPMN plugin for eclipse but that seems to have fallen behind the capabilities of the process engine and doesn’t seem to be under active development anymore.

Developing with Quarkus along with the VSC plugin is going to be a great experience. If you open a terminal on your project and run mvn compile quarkus:dev, it will continuously compile your project as the files change (even bpmn and drl) and let you interact with it as you continue development. There’s no manual build, deploy cycle taking your time.

If you are developing a web app, you just refresh the page and try out your new code. You can also run tests at any time. No need to do the build. If you are building for OpenShift or Kubernetes, you can make use of the new Operators to automatically deploy to the platform when the code is checked in.

It’s a big step forward since last year. The jBPM developers have done a great job.

BPM in a Microservice World — Part 2

Back in the early days of “workflow” we had control of the transaction, usually a document, from the start of the process to the end. As IT evolved into the SOA/ESB era, we had a little bit less control but for the most part the process engine orchestrated everything.

There were frequent hand-offs to message queues but normally the message would come back to the process engine so it would continue to orchestrate the process.

The microservice world is different. Instead of having a process engine or an ESB controlling a small number of large services, we have many small services that can potentially send and receive messages or respond to events from any of the other services.

It’s more like a web. One initiating message or event to a particular service could affect the exchange of many hundreds of messages between the microservices before the initial request is considered complete. That can make BPM practitioners a bit uneasy due to the loss of control.

We may not have control any longer but we still can have visibility into the process. We can still apply our usual patterns for SLA and exception management, and human and compensating workflows. This can be accomplished through what I call a “tracking” process.

I have a process running today that interacts with microservices written with the microservices framework, Vert.x.

Vert.x includes an Event Bus and a cluster manager, among other features. A Vert.x cluster is made up of one to many nodes. A microservice is packaged as a jar module that includes a number of what they call Verticles (British spelling I guess). The verticles are deployed to any number of Vert.x nodes.

Once the verticles are deployed, the Event Bus manages the flow of messages and responses throughout the cluster. This all happens asynchronously, so there is no way us to control that flow from the process manager.

We can still create a process in BPMN that looks like the traditional process. Here is an example.

This is a simplified version of a real process that’s been running for a couple of years on Vert.x. It receives business opportunities from an outside source. Once one is received, we need to save it locally. Then we run it through a machine learning classifier to see if it is the type of opportunity the client might be interested in. If it is, then a human needs to have a look at it. Otherwise, it is rejected.

We receive thousands of these every day. Due to the parallel nature of Vert.x we are able to spawn many requests over the cluster and get this work done quickly.

The persistence part a quite performant so we don’t need many instances of that verticle in the Vert.x cluster. The classification part is slow and requires more resources. So, we have many instances of that verticle over the cluster.

The process above looks like a traditional process but in fact, we are not in control of the transaction here. In each activity, we are sending a message using the Vert.x Event Bus and then waiting until an event happens at a future time. Once that event is received we move on the the next activity which does the same.

Unfortunately, the classification activity doesn’t always complete in a timely manner. In this example we added a boundary timer so that if the classification takes too long, we notify a user and then terminate the process.

The activities that involve microservices in the main process are modeled as subprocesses. Here is an example of the Persist Opportunity subprocess.

The first activity is a custom work item handler I created for Vert.x. It will send a message to the Vert.x cluster using the Event Bus.

That message may cause a number of other services to be called within Vert.x. We don’t care about that, all we need to know is when it’s all finished. I created a customization for Vert.x so that the process manager will be sent a signal when a particular Vert.x service is complete. When that happens the Catch Signal will be executed. At that point, control will be returned to the calling process which can move on the next activity.

So, there you go! We can model processes as we are accustomed to even though we are not in control of the transaction as it moves through the various microservices. You can definitely use these pattern to combine microservice activities with traditional ones and apply our usual process management patterns to all of it.

BPM in a Microservice World — Part 1

First some background.

Generally when the topic of BPM comes up we think of the BPM software suite. There’s another side to BPM though, and that’s the practice of Business Process Management which doesn’t require any software at all.

Traditionally the BPM Practice has focused on continuous process improvement. There are various methodologies but it generally comes down to this:

  • Collect metrics on the existing process
  • Analyze those metrics
  • Propose an optimization
  • Simulate the optimization with the collected metrics
  • Institute the validated optimization
  • Do it all again

There’s nothing wrong with that. We’ve occasionally had good results with continuous improvement for processes that are core to a business.

A good candidate would be a fee-for-service health insurance claim process. It’s a process that’s been around for decades and will likely be around for additional decades. It’s also high volume, so even the smallest improvement can have a major impact.

The unfortunate truth about applying the practice in this way, is that it is very time consuming even with the best software. It can take at least weeks, if not months or years, to get anything meaningful done.

That may be acceptable for a core process, but for most other processes that lead time is just not acceptable. Even if a program or project is approved with estimates made by the most experienced practitioners, it will likely be considered a failure after the inevitable missed deadlines and cost overruns.

We need to provide obvious value much faster. By fast, I mean within hours for something critical, to no more than a few weeks for something significant.

Unfortunately, many of the BPM software suites have become unwieldy behemoths that include a variety of sophisticated functions in a very complicated package. They are generally architected such that the workflow transaction (request) is under complete control of the process engine. This can’t work in a MSA where the various microservices can send messages to one another in an ad hoc manner. We need to change our viewpoint from being an orchestrator of services to a monitor of milestones.

We can still apply BPM patterns like SLA management to processes that include microservices. The process diagrams will look basically the same. The difference is that instead of controlling the movement of a transaction through activities, we will be monitoring milestones as the transaction moves through services. I’ll call them “tracking processes.”

If the services or message bus can ping events to the process manager at certain times, we can create processes that wait for these signals before proceeding on the the next milestone. In the case of SLA management, if the SLA deadline is approaching and we haven’t made progress toward a particular milestone, we can start a new workflow to deal with it. It could perhaps include a human task for some to investigate the delay, or send a message to the client notifying them of a delay.

In the next post we’ll see how to design a workflow that tracks milestones rather than orchestrating activities.

BPM in a Microservice World — Part 0

Business Process Management (BPM) enabling software has been around for decades having started as document centric workflow. It’s progressed into the Service Oriented Architecture (SOA) age to become an orchestrator of services and human tasks.

Now we see that the Service Architecture is evolving to include a relatively new concept called Microservice Architecture (MSA). That architecture along with enabling technologies like Cloud Services and Application Containers is allowing us to apply process management practices to solutions in a much more lightweight and agile way.
In this series of posts, we will see how to apply BPM principles to solutions that are implemented in a MSA as well as how to integrate workflows into the microservices themselves.

Part 1