Researchers at Monash University, Victoria have filed a patent for new technology that can generate thousands of images of construction machinery, train deep neural networks and automate monitoring processes.

The new technology aims to improve construction site safety, quality and productivity through the use of AI.

By generating the synthetic images, including cranes, scissor lifts, bulldozers and dump trucks, it creates large datasets for training deep neural networks to analyse the machinery in several conditions.

It was developed by Monash University’s Dr Mehrdad Arashpour and his team of PhD students and postdoctoral researchers in Monash University’s Department of Civil Engineering.

The new method removes the need for cameras to be placed on construction machinery which comes with its own time, cost and privacy implications.

In other words, before using an excavator at a building site on a wet day, workers can identify all the safety features of this equipment within a digitised version of the working environment, including rain, clouds, as well as reduced light and visibility.

Over time, this technology could be expanded and used on any and every construction site across the world – from Melbourne to Madrid – depending on the type of machinery in operation.

Mehrdad says this application has been designed to improve safety, productivity and quality on building sites and other workplaces: “Operations involving heavy construction equipment are a critical component of most projects.

“Heavy construction equipment, vehicles and workers are often required to work closely due to spatial limitations and tight schedules, which often leads to suboptimal performance both in terms of safety and productivity.

“Preparing large training datasets for specific tasks is currently a manual process, which is highly time-consuming, labour-intensive, error-prone, and subject to privacy concerns.

He adds that the method randomises various critical features of the scene, such as equipment pose and texture, scene texture and lighting, camera location and field of view, and adds other elements to the scene such as simulated dust and occluding objects.

Presently, safety compliance training procedures involve placing cameras on-site to capture images of the machine in operation. From there, the images are labelled manually, including key points of each piece of equipment.

The generated data set is then used to train the AI models and inferences are made regarding classification of different equipment, pose estimation and activity recognition.

The novel Monash-developed method uses game engine technology to generate CAD models of heavy machinery, such as cranes, dump trucks, scissor lifts and other equipment on-site, which are then accurately and automatically annotated.