Since you are reading this book, it’s a pretty good bet that you would like to develop applications that will run on the Microsoft HoloLens. The HoloLens is an exciting platform with potential applications that nobody has thought of yet. This is a ground-floor opportunity for software developers. The $3,000 price tag for a Developer’s Kit means that a mass-market consumer version of the HoloLens is probably several years away. In the meantime, there are plenty of enterprise-level use cases for a device that can embed realistic three-dimensional holograms into the user’s surroundings. Retailers can use the technology to help customers visualize how a product would look in their home, changing colors or features at will. The educational and training applications are many. Holographic images of complex objects, such as jet engines, can be examined in detail and even disassembled to enable scrutiny of individual parts. Step-by-step maintenance and repair operations can be practiced with a holographic representation superimposed on top of a device, requiring minimal intervention from a human instructor.

The application you choose to develop for your first project will probably depend to some extent on the kinds of applications that you have developed in the past. Microsoft initially envisioned the HoloLens as a game platform and entertainment device. They worked with the Unity development environment, popular with game developers, to create the tools for developing holographic applications for the HoloLens. As a result, the first people to get excited about the possibilities opened up by the HoloLens were game developers. With the HoloLens, game action, rather than being restricted to the flat panel of a video screen, can take place in a full three dimensions in a room, a series of rooms, or even outdoors. HoloLens is a good platform not only for any of the traditional types of games that people are accustomed to, but also for entirely new types of games that have yet to be conceived.

It Isn’t All About Games

Although game developers were among the first to get excited about the potential of the HoloLens, use cases in business, government, research, and education may have a greater impact than games do. Two HoloLens devices are already in use on the International Space Station. Currently, walking an astronaut through a complex maintenance procedure via an audio uplink from Mission Control is time consuming and subject to error. An astronaut wearing a HoloLens would be able to see right in front of her what needs to be done, and see how the holograms she sees mesh with the physical device she is working on.

The First Applications: Industrial, Commercial, and Educational

The first release of the HoloLens is the $3,000 Developer Edition. As we are still at the beginning, there are only a handful of apps available, so it is doubtful that many end users would buy one. However, there is quite a bit of diversity among developers. Some developers will be game developers who work for large game-development companies. Others will be independent game developers who sense a ground-floor opportunity. Some will be non-game-application developers who envision business or educational applications that would benefit from the ability to interact with both virtual and physical objects.

Industrial applications

A new product design, such as an automobile drive train, can be walked around and viewed from all angles before any metal is actually bent. This can save time and money by reducing the number of times a prototype must be rebuilt. Figure 6-1 shows such a design at the Volvo design center.

Figure 6-1.
figure 1

Examining a prototype drive train

Full size image

In many different work situations, an employee must perform a complex procedure of some kind. If she hasn’t performed this particular procedure in a while, she may have to refer to written work instructions or a procedure manual. This entails looking back and forth between the instructions and the item she is working on. Sometimes it is not crystal clear how the text in the manual relates to what must be done. If it were possible to show exactly what needs to be done, step by step, with both a virtual representation and the physical object right in front of the worker, there would be much less possibility of error and the job could be completed more quickly, perhaps much more quickly. Figure 6-2 shows both virtual information and a physical car door directly in front of an automotive assembly worker.

Figure 6-2.
figure 2

HoloLens on the assembly line

Full size image

An even bigger advantage occurs if a technical expert at a remote location can Skype in to the on-site employee and see exactly what the on-site employee is looking at. It is almost like having the expert looking over the employee’s shoulder and giving step-by-step instructions on what to do to complete the operation at hand. Since the HoloLens is a fully functional Windows 10 computer with a built-in video camera, it can easily support this kind of cooperative operation.

Commercial applications

In a sales situation, “try before you buy” gives a potential customer a much better idea of what a product would actually be like than would be possible from seeing the product on a store shelf, a catalog page, or a Web site screen. In an early commercial application, Microsoft is partnering with the Lowe’s home improvement chain to help sell kitchen remodeling. The HoloLens-wearing customer enters a relatively bare showroom, and holograms show various configurations and textures of countertops, tables, and cabinets, all at actual size. People can see, rather than having to imagine, how appliances and room elements might look within a room. Figure 6-3 shows a customer designing her new kitchen at her local Lowe's home improvement center.

Figure 6-3.
figure 3

HoloLens helps you visualize and then design a room

Full size image

Educational applications

Probably the biggest early application of HoloLens technology will be in educational and training applications. HoloLens is unique in its ability to enable an instructor to direct the attention of students to holographic details and animations overlaid upon physical items such as jet engines or automatic transmissions. Archaeology professors can show students what the insides of a Mayan pyramid or Pompeiian villa were like at their height hundreds or thousands of years ago, overlaid on top of the ruins that remain today. New employees can be brought up to speed on how to do their jobs in a way that seems real but without the need for individualized instruction from a human expert. Human anatomy can be experienced in a way that goes beyond what you get from illustrations in books or on life-size posters, as shown in Figure 6-4.

Figure 6-4.
figure 4

The systems of the human body

Full size image

Educational institutions such as Carnegie Mellon University, Dartmouth College, Virginia Tech, Clackamas Community College, and the University of California–Berkeley are using HoloLenses in a diverse array of fields, including interactive art, augmented reality for the visually impaired, data analysis, trade-based education, and control of airborne drones.

Tearing Down and Reassembling an Automatic Transmission

There are around 260 million passenger cars on the road in the United States, not counting trucks of any kind. Nearly all of those cars contain an automatic transmission, as manual transmissions have become an endangered species restricted to the vehicles of automotive purists and those who want to save a couple of hundred bucks. Most young adults today have never learned how to operate a manual transmission car.

An automobile transmission, like an automobile engine, is an expensive part of a car, and thus is often repaired rather than replaced and scrapped when it ceases to work. With 260 million cars on the road, a significant number at any given time will be having transmission problems, making transmission repair a much needed subspecialty of the trade of automobile mechanic.

Community colleges and other educational institutions that teach job skills in the trades favor a hands-on approach. In the case of automobile maintenance, physical contact with the systems and subassemblies being taught is critical. It is difficult to switch back and forth between an engine or a transmission being disassembled and a thick service manual on the bench nearby, particularly if the mechanic has greasy hands. With relevant information being holographically displayed right above the piece being worked on, the student can move through a disassembly or reassembly procedure without having to pause to consult the manual and without putting oily fingerprints on manual pages or computer keyboard.

An automatic transmission is a complex assembly of gears, clutches, gaskets, shafts, and other parts that fit together like a puzzle. Planetary gears operate in different modes depending on which gear (first, second, third, etc.) is selected.

The HoloLens can enable the student to see an exploded view of all of these parts and then control an animation of the parts all coming together to form a complete transmission. Callouts and labels can be added at appropriate places and times, and a verbal commentary from an instructor can help the student to understand the purpose of each part and how it contributes to the function of the complete assembly.

Summary

Originally conceived as a new gaming device, the HoloLens has evolved into a device aimed squarely at commercial, industrial, and educational institutions. Use cases in these areas can justify a higher cost than gamers are accustomed to paying. Anyone can buy a HoloLens, but Microsoft sees their best customers as being enterprises that can improve their businesses by providing their employees or customers with holographic mixed reality.

Once you have chosen a project to tackle, the next step is to assemble a team of people to bring that project into existence. This requires people with talents in a variety of disciplines who can work together to create a holographic application that fills a need.