A Risk Identification Method for Power Operation Scenarios Using Image Caption and Semantic Text Similarity Analysis

Power production is a complex process that involves multiple interactions, which require rich semantic knowledge to categorize and evaluate. Utilizing high-level image understanding to accurately identify risks is significant in ensuring staff safety and grid stability. In this work, we propose image captioning and semantic text similarity analysis (IC-STSA), a risk identification method for complex power operation scenarios the employs image captioning and semantic text similarity analysis. First, image caption is used to transform an image at the site into the text that best describes the visual data. Next, a safe power production (SPP) knowledge base is established, after which semantic text similarity analysis is introduced to transform the risk identification into a text classification problem. By comparing the image caption information and the SPP knowledge base and identifying similarities, we achieve refined descriptions of operation scenarios and thereby risk identification. In the final step, actual images are used to verify the findings of the IC-STSA. Results show the proposed method performs well, generating accurate, targeted descriptions, and risk identification.