Detect and Compare: An Image-Based Deep Learning approach for comparing the As-Built situation with As-Planned BIM models

Quality management in construction is an important task where elements on the construction site are
compared to their quality requirements and building codes. Due to the Building Quality Assurance Act
introduced at the beginning of 2024, contractors must keep track of their buildings’ quality. However,
quality controls are often carried out manually, and with construction projects expanding in size, keeping
track of the quality becomes even more time-consuming. This research aims to develop an image-based
object detection method that is able to compare the as-built status of a construction project with the as
planned information from the Building Information Model (BIM). The comparison method uses Computer
Vision (CV) and Deep Learning (DL) techniques to automatically predict as-built data. A mobile robot
equipped with an RGBD camera was utilized on the construction site for as-built data collection. A transfer
learning approach was used where a Faster R-CNN model with a ResNet50 backbone was retrained
to detect construction elements, in this case, power sockets. A real-world use case was carried out in
which the comparison method was developed and validated. After training the Faster R-CNN model
and being able to successfully make predictions, the prediction results were compared to the as-planned
placement of the sockets. Using the BIM model, the socket dimensions were obtained and translated to
pixel coordinates and added as an overlay to the object detection results. The overlay was a rectangular
visualization of the socket, placed in the middle of the image, and represented the as-planned placement
of the socket. The pixel coordinates of the predictions and the expected socket region are compared to
determine the as-built placement status per socket. There are three as-built placement statuses “Within
Region”, “Partially Within Region” and “Outside Region”. The presented method, named the Detect and
Compare method, enables users to automatically detect objects in as-built data and compare the placement
with their as-planned placement. In addition, the method acts as a foundation for additional research in
progress monitoring, as-built modeling, and robot localization.