Overview
Most experienced Canvas App makers will be familiar with the Pen Input control; however, we recently had a client need that required a bit of out-of-the-box thinking that I wanted to share.
For the pen input, the default use case that likely comes to mind to is to capture signatures (and we have successfully used it for this several times) – but this particular client had a more unique need.
Scenario
To allow users to draw on top of a template teeth diagram and save the sketch to create a unique PDF dental report. More specifically, the user is a mobile dentist with an iPad visiting patients across many nursing homes, providing critical care where it is most needed. These dentists need to screen patients and record their dental condition by drawing on top of the diagram to show missing or damaged teeth as part of their treatment process. We created an end-to-end solution that saves down the pen input data to SharePoint and uses Power Automate to generate medical reports automatically.
How To
1. Start with a pen input component in a canvas app and an image over which you wish to add hand-drawn data. We have a more extensive app that displays a list of patients at the facility for the dentist to complete during their visit, but I have tried to omit non-relevant code and components in this tutorial. This is what our dentists see on their iPads:
***Note – The Pen Input must be on top of the diagram image and have the same X & Y Properties. The width and height matter greatly to ensure the image transfers without distortion. For this example, the ‘Teeth Image’ is 760W by 425H while the ‘Pen Input’ is 760W by 484H to allow for the black bar at the bottom. Some trial and error will be needed here by anyone attempting to replicate this solution.
2. Next, add a Submit Button and edit the OnSelect Property. We are taking the ‘Pen Input.Image’ data and converting it to usable JSON by trimming 24 characters. We then take that data and trigger a Power Automate flow that converts it into an image file on SharePoint.
//Trim the pen input data into JSON format we can use elsewhere
UpdateContext(
{
jsonImage: Mid(
JSON(
PenInputSketchEditor.Image,
JSONFormat.IncludeBinaryData
),
24,
(Len(
JSON(
PenInputSketchEditor.Image,
JSONFormat.IncludeBinaryData
)
) – 24)
)
}
);
Set(
Compose_Inputs,
jsonImage
);
//Pass the record ID (var_selectedPatientVisit.’Patient Visit’) and the jsonImage data to power automate
‘Create-TeethSketchImage’.Run(
var_selectedPatientVisit.’Patient Visit’,
jsonImage
);
***Tip – We found it useful to store the raw data in a multiline text field on the record as a backup. The below code is optional and saves data to a field called ‘mb_sketchimagedata’ on the specific ‘Patient Visit’ record.
With(
{
vJSON: Substitute(
JSON(
PenInputSketchEditor.Image,
JSONFormat.IncludeBinaryData
),
“”””,
“”
)
},
Patch(
‘Patient Visits’,
var_selectedPatientVisit,
{
mb_sketchimagedata: vJSON
}
);
)
3. Now we have sketch data in the format needed to run our Flow – called ‘Create- Teeth Sketch Image’. To build a similar flow, first make sure you have a Library set up on your SharePoint site to collect all the sketches that will be created. All the flow is doing to taking the jsonImage data and saving it as a new record in the library, then backfilling with meta-data relevant to the patient for ID purposes.
Relevant expression pasted below, using the output from the Compose block:
base64ToBinary(outputs(‘Compose’))
4. Now we have the data on SharePoint in an image file. If you open the sketch file at this point, you will see whatever was sketched in the pen input control, but without the background diagram. Here is a real example from our process showing the notation used by a dentist:
5. The next flow takes the sketch and populates a Microsoft Word template that is carefully designed to overlay the sketch on top of the diagram using the image transparency feature in Word. We found that setting the sketch transparency to 5% had the best effect. There will be trial and error to make sure the two images align accurately. Your results may vary, but here is how our template is set up:
There are plenty of blog posts detailing how to create Word and PDF documents using Power Automate, so I will not cover that here. We simply added a Picture Content Control in the Developer tab and used Power Automate to insert the sketched image in it. This is overlaid in front of a static teeth diagram image.
Here are some links to great posts I recommend for further details.
PowerApps SharePoint Images – Storing and viewing – Practical Power Apps
Power Automate: Create Word Document With A Repeating Section (matthewdevaney.com)
6. The end result is a PDF dental report that is saved in the patient’s file. Our dentists are now creating hundreds of these reports a week, all digitally, which has been fantastic considering it used to be a paper process.
If you have any questions, you can contact me at andrew@mibar.net.
