Enhancing Process Data in Manual Assembly Workflows
The rise of Industry 4.0 and the convergence with BPM provide new potential for the automatic gathering of process-related sensor information. In manufacturing, information about human behavior in manual assembly tasks is rare when no interaction with machines is involved. We suggest technologies to automatically detect material picking and placement in the assembly workflow to gather accurate data about human behavior. For material picking, we use background subtraction; for placement detection image classification with neural networks is applied. The detected fine-grained worker activities are then correlated to a BPMN model of the assembly workflow, enabling the measurement of production time (time per state) and quality (frequency of error) on the shop floor as an entry point for conformance checking and process optimization. The approach has been evaluated in a quantitative case study recording the assembly process 30 times in a laboratory within 4 h. Under these conditions, the classification of assembly states with a neural network provides a test accuracy of 99.25% on 38 possible assembly states. Material picking based on background subtraction has been evaluated in an informal user study with 6 participants performing 16 picks, each providing an accuracy of 99.48%. The suggested method is promising to easily detect fine-grained steps in manufacturing augmenting and checking the assembly workflow.
KeywordsManual assembly Computer vision BPM Industry 4.0
This research was funded in part by the German Federal Ministry of Education and Research (BMBF) under grant number 01IS16022E (project BaSys4.0). The responsibility for this publication lies with the authors.
- 2.Grzeszick, R., et al.: Deep neural network based human activity recognition for the order picking process. In: Proceedings of the 4th International Workshop on Sensor-based Activity Recognition and Interaction. iWOAR 2017, pp. 14:1–14:6. ACM Rostock (2017)Google Scholar
- 3.He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
- 6.Kagermann, H., et al.: Recommendations for implementing the strategic initiative INDUSTRIE 4.0: Securing the Future of German Manufacturing Industry; Final Report of the Industrie 4.0 Working Group. Forschungsunion (2013)Google Scholar
- 7.Knoch, S., et al.: Automatic capturing and analysis of manual manufacturing processes with minimal setup effort. In: International Joint Conference on Pervasive and Ubiquitous Computing. UbiComp, pp. 305–308. ACM, Heidelberg, September 2016Google Scholar
- 8.Knoch, S., Ponpathirkoottam, S., Fettke, P., Loos, P.: Technology-enhanced process elicitation of worker activities in manufacturing. In: Teniente, E., Weidlich, M. (eds.) BPM 2017. LNBIP, vol. 308, pp. 273–284. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-74030-0_20CrossRefGoogle Scholar
- 10.Lenz, C., et al.: Human workflow analysis using 3D occupancy grid hand tracking in a human-robot collaboration scenario. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3375–3380, September 2011Google Scholar
- 13.Roitberg, A., et al.: Multimodal human activity recognition for industrial manufacturing processes in robotic workcells. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ICMI 2015. pp. 259–266. ACM, Seattle (2015)Google Scholar
- 16.Thoben, K.-D., Pöppelbuß, J., Wellsandt, S., Teucke, M., Werthmann, D.: Considerations on a lifecycle model for cyber-physical system platforms. In: Grabot, B., Vallespir, B., Gomes, S., Bouras, A., Kiritsis, D. (eds.) APMS 2014, Part I. IAICT, vol. 438, pp. 85–92. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44739-0_11CrossRefGoogle Scholar