Sitecore Dynamic Workflow Batching
Vasiliy Fomichev
In How To Posted October 16, 2019
TLDR; Skip to the implementation part of the post to see how to to implement batching yourself using the Sitecore PowerShell Extensions. If you’d like to understand why this is considered a preferred approach – read on.
Workflows have been part of Sitecore from the early days. Interestingly enough, the functionality hasn’t changed at all, except for the new “Approve with Test” action that came with the xDB and a few new actions several years back. Some of the foundational code that is powering the workflow functionality still lives in “Sitecore.Workflows.Simple, Sitecore.Kernel”; it’s been begging a question whether there was ever a .Complex namespace…
The good thing is that like with many things in Sitecore, the Workflow functionality is easily customizable and extendable, thus, over time Sitecore partners and clients have published many extensions for custom emails, deletion approval workflows, integrations with other systems….etc. In this blog post I’ll describe the approach I took to solving the lack of batching support in Sitecore Workflows.
The Case for Workflow Batching
The OOTB Sitecore workflows allow us to include individual items in workflows and promote them through the predefined states before they can be published. One very useful feature of the Workbox is that it allows performing the actions allowed for on individual items, on a selection, which I’ve seen work very well for Content Authors.
At times however, Content Authors need to make several related updates that need to move through the approval workflow together, previewed together, and finally published together. In fact simple page updates often require changes to multiple content items, which would need to all get approved at the same time. This is the use case that Sitecore OOTB implementation does not have a solution for just yet.
Existing Solutions for Sitecore Workflow Batching
When trying to solve any problem, 90% of the time, just like most others, I turned to Google to see what solutions are out there for this problem. To my surprise, there hasn’t been much published on this topic. There is a module on Sitecore Marketplace that proposes a solution through a filter drop-down in the Workbox; however, it wasn’t in a usable form OOTB – needed some styling and general TLC, plus I started getting YSODs in Workbox in Sitecore 9.0.2 almost immediately. Although, it wasn’t a turn-key solution, it got me thinking about either updating the module to work and look the way we needed, or build something different from scratch. Building something from scratch is always the last resort; over the years I’ve learned to pragmatically appreciate reuse, even if the code wasn’t pretty.
General Approach to Sitecore Customizations
Here is a bit of theory before we get into solving the problem. One of the most important things to remember when working with Sitecore, or any platform for that matter, is that it has two main types of users – external and internal, and that it’s important to focus on both. Lack of focus on internal system users leads to convoluted processes, unintended problems, surprises, or simply lengthy content authoring processes, which then start requiring additional QA and reviews due to issues from closely coupled content….etc. issues start snowballing, resulting in clients saying that working with Sitecore is expensive. There are many ways to implement a website component in Sitecore, however, there is usually one or two that would work best for the Content Authors based on their habits, existing workflows, update types, frequency…etc. In other words, Sitecore customizations and extensions, especially the ones affecting the interface must be done with content authoring process in mind.
Another customization rule that is platform agnostic is low intrusiveness. This speaks to a few things:
- Changing as little of the base platform functionality as possible
- Avoid relying on UI, i.e. avoid UI customizations (stability principle)
- Augmenting the functionality instead of changing, or replacing (adding a step versus replacing, for instance)
- Offloading API dependency on middle-ware layer (dependency injection, modules, scripting…etc.)
- Reuse existing functionality as much as possible
- Use features as intended, i.e. avoid hacking existing functionality to use it in a way it’s not intended
The last rule is one of the most important rules from the technical perspective, as it dramatically reduces the cost of long-term maintenance, making Sitecore version upgrades must easier, thus, cheaper. More on this in future blogs.
“Eat Your Own Dog Food”
Although the solution with the filter downloaded from the Marketplace seemed to make sense and it resembled my knee-jerk reaction thinking about solving this problem, the user experience in the Workbox resulting from that solution did not seem friendly, in fact it felt rather clunky, and unpolished. Next, keeping the focus on Sitecore users in mind, I started brainstorming other solutions and after some thought another option came to mind – creating workflows dynamically for each batch!
Creating New Workflows for Each Batch
The alternative approach I came up with would allow content authors create batch workflows from a branch template for a given batch then simply add items to that workflow. Once items have gone through the workflow, a scheduled job would clean up the empty “batch” workflows(the script below deletes them; however, you may choose to archive). To power this functionality, since we are not doing a lot of API manipulation, and can potentially be dealing with large amounts of content, scaffolding….etc. Sitecore PowerShell Extensions module, as in many cases, came to the rescue.
Comparing Workflow Batching Options
With the two options at hand I built a PoC using the second approach and followed Joel’s advice from “Joel on Software” and started “eating my own dog food”. I’ve gone through various use cases simulating Content Author activity of using the filter dropdown in Workbox versus using new workflows and came away with the “dynamic batch workflow” option being a winner. Here is why –
- Dynamic batch workflows create an opt-in experience in a Workbox, similar to the OOTB functionality e.i. Content Authors may select which batch workflows they want to see and work with
- We are not introducing new flows to working with workflows in a Workbox and continue to use the OOTB workflow views and functionality
- The opt-in experience of the dynamic batch workflows allowed to accomplish the same review approval tasks with fewer clicks when compared to the filter option
- Workbox with the dynamic approach seemed to perform slightly faster
- Since each workflow had fewer items, it was easier to manage them in the Workbox
- Fewer controls in Workbox made it seem less complex and intimidating
After evaluating it from the end-user perspective, I looked at these options from the technical point of view, where the dynamic batch workflow implementation approach also came out a winner:
- The entire implementation is scripted with SPE, which offloads some dependencies on Sitecore API to the module, similar to middleware
- SPE allows for easy customization without a need for code deployments
- The implementation does not customize Sitecore UI code, this is a big one! –
- The Workbox is still built using XAML and ASPX, add the fact that a Workflow overhaul is past due, building on these technologies is almost a guaranteed tech debt on arrival (Content Hub won’t replace this IMHO)
- We are not creating obscure tags in an unrelated content area that are tied to workflows, creating additional dependencies for the Workflow system, as we would have to with the filter-based approach
- Adding tagging to all templates in order to support the filtering approach would require editing a lower-level Sitecore Data Definition Template (DDT), which is frowned upon from the maintenance perspective
After discussing this functionality with other architects some concerns were brought up that are worth mentioning:
- Lack of traceability – with batch workflows getting cleaned up once they become empty (items never got assigned to it, or all assigned items are in the final state) it’s hard to trace the history of updates
- This is likely a misunderstanding, however, I’ll still address it – traceability is independent of functionality, it’s a separate problem from batching and should be solved separately. The solution to the traceability problem lays in a history report and logging.
- Lack of common publishing restrictions – the current implementation indeed does not have support for common publishing restrictions, however, if that functionality is required, I would recommend storing it on the dynamically generated workflow item and prompt users for that during the batch creation. Also, remember that in Sitecore everything is an item, so we can move our batched workflows to a location where they can be more accessible.
Although a filtering approach seems like an easier and more straight-forward approach to a technical person, it’s hacky, intrusive, more involved technically, carries technical debt, and carries lower usability factor.
Verdict: dynamic batch workflow approach comes out a winner.
Technical Implementation of Workflow Batching Using PowerShell
The following section describes the behavior and implementation of the batch workflows.
The “dynamic workflow batching” approach supports the following use cases:
- Creating a new item version and adding it to a new batch workflow
- Adding an existing item version to a workflow and switching workflows
- Empty workflow scheduled cleanup task
The beauty of this implementation is that it’s fully powered by Sitecore PowerShell Extensions, so anyone can modify the script in a way they feel works best for their unique use cases without having to get into code compilation and deployments. In fact this entire functionality can be installed without impacting Content Authors.
To make sense of the scripts below, make sure to get familiar with the SPE functionality first, Sitecore UI customizations, and SPE workflow actions.
- Creating a new item version and adding it to a new batch workflow.Support for this action is implemented using the following script. If the selected item has children the script will prompt the user if they also want to move the children to the Draft state of the newly created batched workflow as well.
Import-Function -Name Set-Workflow-Paths $selectedItem = Get-Item -path . if($selectedItem -ne $null){ $newWorkflowName = Show-Input "Please provide a name for the new batch:" -MaxLength 20 if($newWorkflowName -ne $null -and $newWorkflowName -ne ""){ $selectedItems = @() $selectedItems += $selectedItem if($selectedItem.Children.Length -gt 0){ $includeChildrenPromptResult = Show-ModalDialog -Control "ConfirmChoice" -Parameters @{btn_0="Yes"; btn_1="No"; te="Include child items?"; cp="Create Batch and Add Item"} -Height 120 -Width 450 if($includeChildrenPromptResult -eq "btn_0"){ $allChildItems = Get-ChildItem -Path $selectedItem.Paths.FullPath -Recurse $selectedItems += $allChildItems } } #create a new workflow here $newWorkflowPath = $workflowFolder+"(Batch) "+$newWorkflowName New-UsingBlock (New-Object Sitecore.Data.BulkUpdateContext) { $newWorkflowItem = New-Item -Path $newWorkflowPath -ItemType $workflowbranchTemplate $newWorkflowItem."__Display Name" = "Batch: "+$newWorkflowName # Setup draft state $initialStateItem = (Get-ChildItem $newWorkflowPath)[0] $newWorkflowItem."Initial state" = $initialStateItem.ID $finalStepItem = Get-ChildItem $newWorkflowPath | Where-Object { $_.Final -eq "1" } $submitForpreviewItem = Get-Item ($initialStateItem.Paths.FullPath + "/Submit for Preview") $draftOnSave = Get-Item ($initialStateItem.Paths.FullPath + "/__OnSave") $draftAutoSubmitActionItem = Get-Item ($draftOnSave.Paths.FullPath+"/Auto Submit Action") # second workflow state (assuming workflow has more than two states) $readyForReviewitem = (Get-ChildItem $newWorkflowPath)[1] if(!$submitForpreviewItem."Next state"){ $submitForpreviewItem."Next state" = $readyForReviewitem.ID } if(!$draftOnSave."Next state"){ $draftOnSave."Next state" = $initialStateItem.ID } if(!$draftAutoSubmitActionItem."Next state"){ $draftAutoSubmitActionItem."Next state" = $readyForReviewitem.ID } # for map accept and reject actions for all steps other than initial and final $stepsToMap = Get-ChildItem $newWorkflowPath | Where-Object { $_.Final -ne "1" -and $_.Name -ne $initialStateItem.Name } for ($i=0; $i -lt $stepsToMap.Length; $i++) { $acceptAction = Get-Item ($stepsToMap[$i].Paths.FullPath+"/Accept") if($i -ne $stepsToMap.Length-1){ $acceptAction."Next state" = $stepsToMap[$i+1].ID }else{ $acceptAction."Next state" = $finalStepItem.ID } $approveWithTestPath = $stepsToMap[$i].Paths.FullPath+"/Approve with test" if(Test-Path -Path $approveWithTestPath){ $approveWithTestItem = Get-Item ($stepsToMap[$i].Paths.FullPath+"/Approve with test") $approveWithTestItem."Next state" = $stepsToMap[$i+1].ID } $rejectAction = Get-Item ($stepsToMap[$i].Paths.FullPath+"/Reject") if($i -eq 0){ $rejectAction."Next state" = $initialStateItem.ID }else{ $rejectAction."Next state" = $stepsToMap[$i-1].ID } } $db = $selectedItem.Database $workflowProvider = $db.WorkflowProvider foreach($item in $selectedItems){ $oldState = $item."__Workflow state" $newState = $initialStateItem.ID $addedResult = $workflowProvider.HistoryStore.AddHistory($item, $oldState, $newState, "Item version moved to " + $newWorkflowItem."__Display Name" + " workflow.") $item."__Workflow state" = $initialStateItem.ID } } } } - Adding an existing item version to a batch workflow or switching workflows
The following script adds another button to Content Editor Workflow chunk of the review ribbon to help move item versions back into the generic workflow.Import-Function -Name Set-Workflow-Paths Write-Host "starting version: "+ $SitecoreContextItem.Version.Number $selectedItem = Get-item -Path $SitecoreContextItem.Paths.Fullpath -Version $SitecoreContextItem.Version if($selectedItem -ne $null){ $workflows = Get-ChildItem -Path $workflowFolder $options = [ordered]@{} $options.Add("Select workflow", "") foreach($workflow in $workflows){ $options.Add($workflow."__Display Name", $workflow.ID) } $props = @{ Parameters = @( @{Name="selectedOption"; Title="Workflow: "; Options=$options; Tooltip=""} ) Title = "Option selector" Description = "Choose the right option." Width = 300 Height = 300 ShowHints = $true } Read-variable @props if($selectedOption -ne $null){ $selectedItems = @(); $selectedItems += $selectedItem if($selectedItem.Children.Length -gt 0){ $includeChildrenPromptResult = Show-ModalDialog -Control "ConfirmChoice" -Parameters @{btn_0="Yes"; btn_1="No"; te="Include child items?"; cp="Add to Batch"} -Height 120 -Width 450 if($includeChildrenPromptResult -eq "btn_0"){ $allChildItems = Get-ChildItem -Path $selectedItem.Paths.FullPath -Recurse $selectedItems += $allChildItems } } $db = $SitecoreContextItem.Database $workflowProvider = $db.WorkflowProvider foreach($item in $selectedItems){ # setting to the first state of the workflow, i.e. starting from the beginning $oldState = $item."__Workflow state" $newState = (Get-ChildItem ((Get-Item $selectedOption).Paths.FullPath))[0].ID $addedResult = $workflowProvider.HistoryStore.AddHistory($item, $oldState, $newState, "Item version moved to " + (Get-Item $selectedOption).DisplayName + " workflow.") $item."__Workflow state" = (Get-ChildItem ((Get-Item $selectedOption).Paths.FullPath))[0].ID } } } - Empty workflow scheduled cleanup task
The scheduled cleanup task follows suit and uses SPE script task to check for workflows that do not have any items assigned to them or the ones where all assigned items are in the final state and delete them (the can be later found in the recycle bin)Write-Log -Log Debug "Workflow Custodian: Starting batch workflow retention check." Write-Log "Workflow Custodian: Starting batch workflow retention check." Import-Function -Name Set-Workflow-Paths $workflows = Get-ChildItem -Path $workflowFolder function RemoveItem([Sitecore.Data.Items.Item]$contentItem) { $deletinbatchWorkflowMsg = "Workflow Custodian: Deleting "+ $contentItem.Paths.FullPath Write-Log $deletinbatchWorkflowMsg Write-Host $deletinbatchWorkflowMsg New-UsingBlock(New-Object Sitecore.SecurityModel.SecurityDisabler){ New-UsingBlock (New-Object Sitecore.Data.BulkUpdateContext) { Remove-Item -Path $contentItem.Paths.FullPath -Recurse } } } $batchWorkflows = @() foreach($workflow in $workflows){ if($workflow.DisplayName.StartsWith("Batch")){ $batchWorkflows += $workflow } } #foreach batch workflow check presence of items $workflowsToDelete = @() foreach($batchWorkflow in $batchWorkflows){ Write-Log -Log Debug "Workflow Custodian: Checking whether "+ $batchWorkflow.Paths.FullPath +" workflow shoul be retained." $batchWorkflowStates = Get-ChildItem -Path $batchWorkflow.Paths.FullPath $cleanBatchWorkflowStates = $batchWorkflowStates | Where-Object { $_.Final -ne "1" } $itemCount=0; foreach($cleanBatchWorkflowState in $cleanBatchWorkflowStates){ $workflowInfo = New-Object -TypeName Sitecore.Workflows.WorkflowInfo -ArgumentList $batchWorkflow.ID, $cleanBatchWorkflowState.ID $items = (Get-Database -Name "master").DataManager.GetItemsInWorkflowState($workflowInfo) $itemCount = $itemCount+$items.Count } #check if Deployed state has items $deployedState = $batchWorkflowStates | Where-Object { $_.Final -eq "1" } $deployedStateWorkflowInfo = New-Object -TypeName Sitecore.Workflows.WorkflowInfo -ArgumentList $batchWorkflow.ID, $deployedState.ID $deployedItemUris = (Get-Database -Name "master").DataManager.GetItemsInWorkflowState($deployedStateWorkflowInfo) if($itemCount -eq 0 -and $deployedItemUris.Count -ne 0){ # all states are empty, except for final one $foundEmptyMsg = "Workflow Custodian: Found empty batch workflow: "+$batchWorkflow.Paths.FullPath Write-Log $foundEmptyMsg Write-Host $foundEmptyMsg $remappingWorkflowMsg = "Workflow Custodian: Remapping deployed items to general Lexus workflow." Write-Log $remappingWorkflowMsg Write-Host $remappingWorkflowMsg foreach($deployedItemUri in $deployedItemUris){ $validUri = $deployedItemUri.ToString().Replace("sitecore://","sitecore://master/") #the URI given by the DataManager is not complete, missing database Write-Log -Log Debug "Workflow Custodian: Remapping URI: "+ $validUri $deployedItem = Get-Item -Path master: -Uri $validUri $deployedItem.__Workflow = "{0DB6A7FD-ECC6-4065-A988-BEE8C99C0581}" # Lexus workflow ID $deployedItem."__Workflow state" = "{FED3B8BA-EBA0-4B15-84FE-24496B2F54C8}" # Lexus workflow deployed state ID } Write-Log -Log Debug "Workflow Custodian: Remapping complete." $workflowsToDelete += $batchWorkflow }ElseIf($itemCount -eq 0 -and $deployedItemUris.Count -eq 0){ # no items found in any workflow states $unusedWorkflowMsg = "Workflow Custodian: There are no items using this workflow: "+$batchWorkflow.Paths.FullPath Write-Log $unusedWorkflowMsg Write-Host $unusedWorkflowMsg $workflowsToDelete += $batchWorkflow } } if($workflowsToDelete.Count -gt 0){ foreach($workflowToDelete in $workflowsToDelete){ RemoveItem($workflowToDelete) } }else{ $nonFoundMsg = "Workflow Custodian: No empty workflows found." Write-Log -Log Debug $nonFoundMsg Write-Host $nonFoundMsg } $completeMsg = "Workflow Custodian: Batch workflow retention check complete." Write-Log -Log Debug $completeMsg Write-Host $completeMsg - Bonus: recursive workflow providerIf we are in a multisite environment, we are going to end up with multiple dynamically created batch-type workflows (workflows that start with “Batch:”) mixed in with workflows for other websites, thus, we need to organize them a bit better and put them in a separate location. The default Workflow provider does not support having folders under “/sitecore/system/Workflows” path, therefore we need to add our only C# customization to override the OOTB workflow. Note: it’s important to use fast query here for performance.
using System; using Sitecore; using Sitecore.Workflows; using Sitecore.Workflows.Simple; namespace CustomNamespace.Foundation.Configuration.Providers { public class RecursiveWorkflowProvider : WorkflowProvider { public RecursiveWorkflowProvider(string databaseName, HistoryStore historyStore) : base(databaseName, historyStore) { } public override IWorkflow[] GetWorkflows() { var workflowRootFolder = Database.Items[ItemIDs.WorkflowRoot]; if (workflowRootFolder == null) return Array.Empty(); var array = workflowRootFolder.Database .SelectItems($"fast:{workflowRootFolder.Paths.Path}//*[@@templateid='{TemplateIDs.Workflow}']"); if (array == null) return Array.Empty(); var workflowArray = new IWorkflow[array.Length]; for (var index = 0; index < array.Length; ++index) workflowArray[index] = InstantiateWorkflow(array[index].ID.ToString(), this); return workflowArray; } } }<?xml version="1.0"?> <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set/"> <sitecore> <pipelines> </pipelines> <databases> <database id="core"> <workflowProvider set:type="CustomNamespace.Foundation.Configuration.Providers.RecursiveWorkflowProvider, CustomNamespace.Foundation.Configuration" /> </database> <database id="master"> <workflowProvider set:type="CustomNamespace.Foundation.Configuration.Providers.RecursiveWorkflowProvider, CustomNamespace.Foundation.Configuration" /> </database> </databases> </sitecore> </configuration> - Additionally, create the following function-type script to provide values for the global variables :
$workflowFolder = "master:/sitecore/system/Workflows/CustomFolder/" $workflowbranchTemplate = "/sitecore/templates/Branches/CustomFolder/Foundation/SitecoreExtensions/Workflow"
Finally< I would recommend using Sitecore rules to specify when the buttons or context menu options, however you choose to implement the scripts as, are shown and enabled when a valid item is selected. This will make the functionality more intuitive, increase the usability and decrease the learning curve.