Keywords

1 Introduction

In the year 2008, the United Nations Convention on the Rights of Persons with Disabilities [1] came into force. One principle of the Convention is the ‘Full and effective participation and inclusion in society’ which results amongst others in Article 27 – Work and Employment. This article declares ‘the right of persons with disabilities to work, on an equal basis with others’. But still, handicapped people have hindered access to the job market. For instance, in Germany nearly 33,500 mandatory workplaces for severely disabled people have not been filled in the year 2013 [2]. Companies would rather pay countervailing charges than employ handicapped people.

Therefore, workplaces for persons with disabilities have to be created and designed in such a way that these disabilities have diminished or no effect on productivity or just play no role with respect to the working process. We think that, in specific fields of work, appropriate technologies can empower the employment of handicapped workers without decreasing the productivity, being as beneficial as employment of non-impaired people.

In this paper, we focus on hearing-impaired order pickers in warehouses. Due to their hearing impairment, the receiving of auditory information is not possible. The loss of this information channel can impair the work and its quality and also jeopardizes safety when alarm sirens are not heard. As information is needed during order picking, the information must be provided over the visual system. Therefore, we apply a visual technology called Pick-by-Vision that deploys smart glasses. This technology has been introduced by our institute [3]. We now step forward from basic research to a real-life application and design an overall system with regard to inclusion.

2 Order Picking

Order picking is a fundamental process in intralogistics throughout the whole supply chain. It is the upstream process for the supply of customers or production facilities with ordered articles. It is defined as the collocation of specific subsets from a total assortment of goods based on an order [4] (Fig. 1).

Fig. 1.
figure 1

Schematic representation of order picking

Full size image

There are both manual and automated order picking systems. Manual order picking systems are very common due to their flexibility; no physical infrastructure has to be built up. Because of low investments, they often are the only alternative for small and medium-sized enterprises. The principle on which they work is called ‘Man-to-goods picking’, meaning that the order picker moves through the warehouse to each storage location of the ordered articles.

One main part of a manual order picking system is the information system responsible for providing the order information to the picker and vice versa for gathering the status of the order processing. It is mostly embedded in a Warehouse Management System (WMS), which collects the orders and prepares them for the picking process. The main information are the location label, the number of articles to pick and optionally an article designation.

The information system comprises the human-computer interface in order picking, which is realized by different implementations as discussed in the following subchapters.

2.1 Paper List

The simplest implementation of an order picking information system is the paper list (Fig. 2a). The order picker gets all the needed order information from a printed list he has to carry with him. He picks all items listed and usually confirms every pick by marking it off the list. This system is very convenient, as it doesn’t use any additional equipment.

Fig. 2.
figure 2

Order picking information systems: (a) Paper list, (b) Mobile data terminal, (c) Voice picking, (d) Pick-by-Light

Full size image

Due to human nature, this picking process is error-prone. Positions of an order are skipped or omitted. There is no automated verification if the correct article is picked. Furthermore, the order picker has to handle the list and a pen with the adverse effect that he doesn’t have both hands free for the actual activity. Therefore, more sophisticated systems have been established called paperless order picking systems.

2.2 Common Paperless Order Picking

In contrast to the paper list, paperless order picking systems imply a direct interaction between the order picker and the information system. The order picker is directed through the order processing making it impossible to miss an order’s position. Each order position or pick has to be confirmed, commonly by the automated identification of an article or its storage location. Identification is an essential operation in order picking and prevents one from picking the wrong article. The error rate is decreased and performance is increased in comparison with the paper list. The identification result also triggers the information system either to show the next order position or, in case of an incorrect pick, to initiate any error process. In addition, identification data enables the WMS to update the warehouse inventory in real-time for accurate inventory management.

There are several paperless order picking systems that differ in the human-computer interaction.

  • Mobile Data Terminal: A mobile data terminal (MDT, Fig. 2b) contains a display, a code scanning unit and a keyboard. Order information is to be read on the display. The usage of MDTs is very flexible, but a main drawback is the form factor, which is very bulky. When articles are picked, the MDT must be put aside or into a belt holster, which results in a low performance.

  • Voice Picking: In voice picking systems (Fig. 2c), information is transferred vocally. The order picker wears a headset and order instructions are provided by verbalized prompts. The order picker confirms picks by speaking pre-defined commands and specific confirmation codes of articles or storage locations. As a main advantage, the usage of voice picking systems is fully hands-free.

  • Pick-by-Light: In Pick-by-Light systems (Fig. 2d) each storage location is equipped with a noticeable light, a button and, in most instances, with a numerical display. The worker picks articles from a highlighted storage location and confirms the pick by pushing its button. Due to the installation of equipment at each storage location, investment costs are very high, while the system offers low flexibility.

2.3 Pick-by-Vision

Pick-by-Vision also belongs to the paperless order picking technologies. Order information is provided by smart glasses in the pickers direct field of vision (Fig. 3). The modality of interaction for confirmation, identification and further data input is not determined. Either the smart glasses or additional devices can be used for interaction. The main benefit is the mobile, but hands-free provision of information. As an advantage over voice picking, the information is provided at once and not sequentially as it is in speech.

Fig. 3.
figure 3

Pick-by-Vision: Smart glasses provide order information in the pickers field of vision

Full size image

The technology, Pick-by-Vision, has been introduced before by our institute, and various systems with different complexity have been investigated [5, 6]. In comparison with the paper list, order picking performance is increased while the error rate decreases. Mostly due to the oversized form factor of smart glasses so far, it has been rarely applied in industry. Now, smart glasses of the latest generation, along with Google Glass, have smaller dimensions, enabling Pick-by-Vision to be used realistically and productively.

We define Pick-by-Vision as an augmented reality technology in the broader sense because the user’s perception of the real-world environment is augmented by computer-assisted information during his working process. It should be noted that the information presentation doesn’t inevitably have to be an exact superposition of virtual objects over real-world objects as in other familiar augmented reality systems that we call systems with congruent information presentation [7]. In the case of a basic Pick-by-Vision system, order information is presented by texts or pictograms while still being an augmented reality application. In this case, we speak of a context-sensitive information presentation since the information is shown in the context of the current working process. There has been research about information systems with congruent information presentation as well [8].

3 Inclusion on the Example of Order Picking

The aim of our research is the inclusion of hearing-impaired workers into the warehouse working environment using the example of order picking. We are going to achieve that on the basis of a technological system. The logistics process, order picking, and the contrary process, replenishment, shape the main use case of our system. First of all, we need to select an appropriate order picking technology. In the next step, we define further use cases that should advantage inclusion.

3.1 Selection of an Order Picking System

The selection of an order picking system always depends on the individual case. In our case, we are looking for a manual order picking system with an appropriate human-computer interface for hearing-impaired workers. Nevertheless, we derive the selection as general as possible.

With respect to profitability, order picking using paper lists and mobile data terminals is quite inefficient. Pick-by-light systems have a high throughput, but can be discarded because of high investment costs and the systems’ inflexibility. Especially in our case, the system must cope with a changing inventory structure of paper and office products. Voice picking turned out to be the most beneficial technology with regard to the properties error rate, throughput, investment and flexibility [9, 10]. However, hearing-impaired workers cannot use voice picking systems. That leads to the remaining technology Pick-by-Vision, which we apply in our research.

It should be mentioned that Pick-by-Vision hasn’t been developed with the intention of inclusion, though it shows to be a very suitable option in the case of hearing-impaired order pickers. Furthermore, the technology dissolves the incriminating situation for hearing-impaired order pickers of being suspended from their field of activity by the progressing voice picking systems. In that way, Pick-by-Vision also contributes to the prevention of psychological stress. This can be seen in the strong motivation of the hearing-impaired order pickers employed by our industrial research partner.

3.2 Further Use Cases

Beside the pure execution of logistics processes, we expand the system with more functionality regarding inclusion. Two more use cases are defined.

The first relates to the communication between users of the system. A textual call function should be implemented so that a hearing-impaired worker is able to notify coworkers or supervisors even without visual contact. That will be useful when a worker resides in a remote location of the warehouse and needs support.

The second use case incorporates messages in general, related to work, safety and social topics. Work-related messages, for example, inform employees of meetings or disruptions in the materials handling system. Disruptions can also be reported by users to the system. To enhance safety, fire alarms, which cannot be heard by the hearing-impaired, are visualized on the smart glasses in addition to the statutory mounted caution lights. Regarding the third option, messages can also contain social news such as current birthdays, weddings or other events. Hearing-impaired workers told us in discussions that they often miss informal information or the office grapevine. Because workers should not be overloaded with too much information during the work activities, it is contemplated to just send a notification that informal messages have been sent. These messages can be read at a common location like the break room.

Messages are sent from a central instance to all users of the smart glasses. It is up to the internal organization of the company to supervise the messages.

4 System Requirements

We will focus on the main requirements in this paper. It should be considered that the system will be used by both hearing-impaired and non-hearing-impaired workers. The pilot application will be at our industrial research partner, a medium-sized mail order company, who operates a warehouse for all kinds of paper and office products.

4.1 Requirements Due to Hearing Impairment

We found out that there actually are just two major requirements that are caused by the hearing impairment.

  • The first important requirement is to consider that the system does not distinguish distinctly between hearing-impaired and non-hearing-impaired users since that would reduce the degree of acceptance among the hearing-impaired.

  • The second point regards the human-computer interface. Feedbacks of operating activities should at least be visual or haptic and not only acoustic. For instance, barcode scanners beep if they have successfully read a code. This feedback could also be realized by a haptic vibration signal.

4.2 Smart Glasses

We concentrate on practical requirements regarding the implementation for a real-life application.

  • Due to a strong demand for low costs in the field of logistics, we focus on consumer products rather than military solutions.

  • The smart glasses must also be useable for spectacle wearers.

  • There are further requirements on the optical properties. A clear view of the real-world environment should be ensured. This involves factors like the position and brightness of the virtual image, contrast of the display and also the size of the frame.

  • As described above, the information presentation that we use is context-sensitive. Instructions and information are provided in a textual form or by pictograms. No augmented reality tracking for an accurate superposition of virtual objects over real-world objects is needed. Thus, sensors like a camera or inertial sensors as well as a powerful processor are not crucial.

  • Battery should last for a shift.

  • This requirement primarily affects the company organization. Investigations showed that prolonged work with smart glasses can have an effect on strain [11]. It is suggested to extend the duration of breaks.

4.3 Identification

The identification process is essential in guaranteeing high quality in order picking. In general, there is either the identification of articles or the identification of storage locations. The identification of articles could be realized with smart glasses since it is possible to hold articles in front of the smart glasses’ camera.

Though in our case, many articles are too small for barcode imprints or just don’t have a barcode so that their storage locations have to be identified. Imagine the order picker has to pick an article from a storage location at the ground-level of a shelf. The attached barcode of the storage location will also be at ground-level. It is hardly possible to identify this storage location by a smart glasses camera since the order picker cannot lead his head to the barcode without any contorted maneuver. It is much more ergonomic and also much faster to direct a scanning device to a barcode than his head. Furthermore, if there are multiple small storage locations there will also be many identification codes in the camera image, posing the question of which one to choose for identification. Yet another issue is lighting conditions. Smart glasses’ camera systems do not emit light for the scanning procedure as it do scanning devices, which therefore deliver more robust results.

Nevertheless, we implemented a barcode scanning app in our smart glasses using the open source ZXing library [12]. We found out that orienting the smart glasses to a barcode by watching the camera’s video stream in the smart glasses display is more difficult and time-consuming than just aiming a scanner’s reticle to the barcode. The direct optical feedback on the barcode is easier to handle than the indirect video feedback.

In summary, an extra scanning device is indispensable in our case. That potentially seizes a free hand as a drawback. But it also brings the positive side effect that the smart glasses’ battery capacity is saved since cameras and image processing algorithms would consume substantial energy.

4.4 Data Input for Special Cases

Only the input of identification data is necessary during the usual order picking process. But there are three situations that require further data input.

  • Error handling in the order picking process

  • Textual call function

  • Notification of a disruption in the materials handling system

To avoid complexity, only predefined entries or requests can be selected. For instance, if the order picker wants to report a disruption, he should be able to choose from a list that enumerates several typical disruptions in the warehouse facilities. No text should be entered. Only in case of an article shortage is a numerical input required to update the virtual inventory of the WMS.

Because data input with smart glasses is often not practical, an additional device can be used. This must be aggregated with the scanning device.

5 System Design

Having set the fundamentals of the overall system, we turn towards the system design. Thereby we center on the hardware that we use for the application. First, we give a short overview of the system architecture.

5.1 System Architecture

The system architecture is illustrated in Fig. 4. The central component is the WMS. It has interfaces to the clients, the Pick-by-Vision system and control terminals, as well as to peripherals, like a fire detector.

5.2 Smart Glasses

We have been working with three different smart glasses so far. These are the well-known Google Glass, the Vuzix M100 and the Sony SmartEyeglass, each having advantages and drawbacks. They represent different types of head-mounted displays (HMD), monocular and binocular as well as optical see-through and non-see-through glasses. The Vuzix M100 embeds a very open Android platform that easily allows one to implement applications and connect external devices, while it wasn’t possible to connect a barcode scanner via Bluetooth to the Google Glass. Sony’s binocular display enables a very impressive spatial information presentation as if the order information were hovering in the warehouse surroundings.

Fig. 4.
figure 4

Overall system architecture including the Pick-by-Vision system. The smart glasses communicate through Wi-Fi with the WMS and through Bluetooth to the scanning device.

Full size image

In a preliminary mock-up, we presented the glasses to our research participants and actual order pickers, both hearing-impaired and non-hearing-impaired. Most of them would prefer the Sony SmartEyeglass. This result must be handled with caution. Actually, it is mostly based on the fact that the Sony glasses, which have a stable frame, are easier to put on. The other two, especially the Vuzix, require some adjustment to fit properly. There are extra frames available for both of them that should be used for practical usage and moreover allow the use of prescription lenses. That in turn isn’t possible using the Sony smart glasses without a custom-made mounting that we constructed.

As we haven’t found the perfect smart glasses that fulfill all our requirements yet, we will not commit to one choice. The system is openly designed to integrate upcoming smart glasses as well.

5.3 Identification Device

For the identification process, we had to find a way to comply with two opposite requirements. The Pick-by-Vision system should be preferably hands-free. Otherwise, it is best that an identification device is directed by the users hand toward an identification code. We propose two alternatives, as shown in Fig. 5.

The first solution integrates a scanning device in a glove (Fig. 5a). Scanning is triggered by pressing a button at the forefinger with the thumb. Both hands can be used to pick articles. Though the scanning device on top of the glove can get stuck when the order picker grabs articles from narrow shelves. With the second alternative, a mini scanner is affixed to a cord reel that is attached to the order picker (Fig. 5b). This is possible since mini scanners don’t weigh much.

Fig. 5.
figure 5

Proposals for the identification process

Full size image

Both scanning devices give a tactile feedback when a code has been scanned successfully. We suggest using imaging scanners instead of laser scanners since they don’t have to be directed vertically to codes, which protects the picker’s wrist.

To enable complex data input, we chose a mini scanner with four buttons. One button triggers the scan, the three others are programmable and are used to click through a menu (up, down, confirm). In case of the glove, these operations must be done with the smart glasses’ interaction options as a third interaction device is not productive.

6 Conclusion and Outlook

We presented a system that allows the inclusion of the hearing-impaired in order picking processes. Pick-by-Vision, with the use of data eyewear, presents itself as a suitable technique for an order picking information system because it dispenses with the use of the auditory information channel. As a drawback, data input is limited with smart glasses, and identification as an essential process in order picking cannot be fulfilled. We give two proposals that enable this human-machine interaction and constrain hands-free working as little as possible.

Based on this system, we implemented additional functionality beside the order picking process that offers benefits to hearing-impaired workers. This will not only advance an inclusion in the specific working process of order picking, but also in the working environment.

In the next step of the project, the enhancement from a mockup to a productive system in cooperation with industrial partners is planned. In productive operation, the system will be analyzed and iteratively improved. Afterwards, the system will be deployed in another warehouse to achieve a more universal application in a heterogeneous application field.