ESP32-S3 · Whisper · LLM · n8n
The challenge
At an industrial plant, inspectors follow routes around the facility and record issues they find. Writing the shift report manually takes about 30 minutes. Critical issues reach the maintenance service only after the report has been prepared and submitted.
The solution
The system has two parts: a wearable device that records speech and an enterprise server that turns recordings into a report.
An inspector approaches a piece of equipment, presses a button, and says: “Pump 3, bearing vibration.” There is no need to write anything down, so both hands remain free.
The device is designed for work on the plant floor:
- recording starts with one physical button, without looking at the screen;
- recordings are stored on a memory card and do not depend on connectivity: the device holds up to 64 recordings, and long recordings are automatically split into parts;
- a real-time clock synchronized through NTP gives every note an accurate timestamp;
- the touch screen has an archive with the date, time, and duration, and any recording can be played back;
- when Wi-Fi becomes available, recordings are sent to the server automatically once an hour or manually with a button. Files are streamed without loading them into device memory, and uploaded recordings are marked to prevent duplicates.
The server receives and processes recordings inside the enterprise perimeter:
- Whisper transcribes the speech;
- a language model maps each note to fields: unit, issue, and severity. “Pump 3, bearing vibration” becomes a record with unit “Pump 3”, issue “bearing vibration”, and high severity;
- n8n assembles the shift report and immediately sends critical issues to the maintenance system as requests.
My role
I came up with the product and built the system prototype. On the device side, I selected the ESP32-S3 hardware platform and wrote the firmware for recording, storage, archive, clock, and server synchronization. On the server side, I built the recording intake and designed the issue-processing flow from transcription to maintenance request.
Result
A working device prototype and a projected impact for a route with 20 inspectors:
- ~5 minutes for a shift report instead of ~30
- ~165 hours freed up per month for 20 inspectors
- ≤ 1 hour from an issue being recorded to a maintenance request