Advertisement

Transferring Object Layouts from Virtual to Physical Rooms: Towards Adapting a Virtual Scene to a Physical Scene for VR

  • Zackary P. T. SinEmail author
  • Peter H. F. Ng
  • Simon C. K. Shiu
  • Fu-lai Chung
  • Hong Va Leong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11542)

Abstract

One crucial problem in VR is to establish a good correspondence between the virtual and physical scenes, while respecting the key structure and layout of the two spaces. A VR scene may well allow a user to move towards a certain direction, but in reality, a wall could block his/her path. The reason is that the virtual scene is often designed by designers for a large user population, being unable to consider individual physical constraints. Recent works involve redirected walking or additional VR hardware. We propose an alternative solution by adapting the virtual scene to the physical scene. The goal is to preserve the layout of the original design while allowing the virtual scene to be fully walkable. To move towards that goal, this paper proposes an object layout transfer mechanism to transfer an object layout from a virtual room to a physical room. A VR application can automatically transfer a designer’s virtual scene to fit any household room. Our solution involves key object layout features extraction and a multi-stage algorithm to shuffle the objects in the physical space. A user study is conducted to demonstrate that the proposed model is able to synthesize transferred layouts that appear to be created by a human designer.

Keywords

Object layout transfer Space adaptation Layout discrepancy measure 

1 Introduction

One of the double-edged swords of deploying virtual reality (VR) is the inherent spatial inconsistency between real and virtual environments, especially in the context of game playing and scene exploration. A virtual scene may contain a large explorable area which is good for players staying within small households. However, if the player wishes to explore by walking in reality, due to the limited space of a common physical room, s/he would eventually be blocked. Most games usually allow movement in virtual scenes via joystick controllers instead of actual walking to mitigate the constraint. The player is allowed to move via joystick control input, which is commonly known to induce motion sickness [1, 2]. Some games opt to use a point-to-point teleportation presentation to get around this nausea problem. However, this method is believed to limit the immersiveness of VR gaming.

Another approach is to use VR treadmill like Virtuix Omni by Virtuix Inc. or KAT WALK by KatVR. Via the additional hardware, the player would be able to walk indefinitely in reality and therefore also a virtual one. It is believed that this kind of hardware could provide a more immersive experience compared to joystick controllers. However, the costs and sizes of such devices may make it less desirable for home-use.

Due to the constraints of the solutions aforementioned, space warping [3] and redirected walking [4] are recently gaining attention. Instead of relying on additional hardware to bound the player, spaces are transformed from time to time such that s/he could explore freely by walking without fear of obstacles in reality. Current works achieved impressive results, but they do not guarantee a complete immersive experience as, for example, it is known for some redirecting techniques to send warning to the player to stop and to turn [5].

Previous methods in some sense attempted to “fit” a virtual environment into a physical one by avoiding the spatial inconsistency problem. An obviously preferable approach is to eliminate spatial inconsistency from the first place. If the virtual scene is spatially similar to the physical scene, the player would be able to explore freely without motion sickness or additional hardware/software to avoid physical obstacles. It is possible to design a virtual scene to spatially match that of a physical room. However, this would be a case-by-case approach that is only suitable for specific scenarios, such as a theme park creating a VR experience for a designated room. A more desirable application would be that a level designer could create a virtual room, and that virtual room would be tuned to match the physical room. Therefore, a designed virtual level could be played in any room, especially household ones.

A new perspective on addressing the spatial inconsistency is proposed. Instead of “fitting” a virtual environment to a physical one, a virtual environment could be synthesized from a physical one such that they are spatially similar. Aside from alleviating the spatial inconsistency, this methodology has an added benefit of “what you see is what you get”. If the player sees a virtual wall, it is because there is actually a physical wall. This also means that the player could touch the wall, further enhancing the immersiveness of the VR experience and fun factor of the game via sensational feedback [6]. Our concept is closely related to and inspired by Simeone’s substituting reality [7], but our’s leans towards synthesizing the virtual space from the physical space while Simeone’s focused on utilizing physical objects for interactions. We call our concept scene adaptation, i.e. adapting a virtual scene to a physical scene (see Fig. 1).
Fig. 1.

The left illustrates the idea of scene adaptation while the right illustrates an object layout transfer model which is a step towards the mentioned idea.

In this paper, we propose a step towards the idea of scene adaptation. Specifically we aim to solve a sub-problem: transferring a designer’s object layout to a physical room (Fig. 1). The outcome is that a designer could simply design a single virtual object layout and it could be transferred to any physical room. Concrete application examples include: a virtual museum could be transferred to anyone’s living room for walkable visits; or a game level’s monster placement where monsters start at specific locations of a room could be transferred to a player’s bedroom to surround him/her.

We propose a model to transfer an object layout from a virtual to physical room by utilizing a set of selected features to compute an Object Layout Discrepancy (OLD) metric between the virtual and physical room. Via searching in an object placement space, a best placement for each object could be found. It is assumed that the lower the OLD, the more similar the physical room’s object layout is to the original design. An illustration of this model is shown in Fig. 2. We highlight a specific process to achieve object layout transfer via the use of OLD metric. To conclude, the main contributions of this paper include: (1) a pipeline to adapt a designer’s object layout to a given physical room, (2) a computational process which uses selected features to search for best placements for the virtual objects, and (3) a preliminary user study on the effectiveness.
Fig. 2.

The left is the proposed model while the right illustrates the result of the proposed model. The right also shows that by extracting object layout features and computing the OLD, a physical room could be placed with virtual objects such that the layout is similar to the original one from the virtual room. As can be seen, the object layout of the bottom-right is better than that of the bottom-left (which does not follow OLD at all) in mimicking the layout at the top.

2 Literature Review

VR treadmills are good hardware solutions for freely exploring virtual realities. However, their costs and sizes impose severe limitations on their home use. They may also require additional setup. Virtuix Omni by Virtuix Inc., for example, requires the player to wear a pair of Omni shoes and also a harness [8].

There is a growing body of literature in space transformation techniques which help players avoid hitting obstacles in the real environment while exploring in virtual reality. Redirection is one such technique which timely rotates the virtual camera [9]. Combined with saccade [10], it can achieve an impressive result that only requires a room of 12.25 m2 for redirecting [3]. Another technique is to warp the virtual scene to guide the player away from physical obstacles [4]. A downside is that the virtual environment will be distorted and therefore does not reflect the original look and feel.

Our work in contrast does not attempt to allow the player to explore an unconstrained virtual reality in a limited physical space. Instead, we are interested in transferring object layout from a virtual room to any physical room which aids in creating a virtual scene based on an actual physical environment. Thus, our work is closer to furniture planning. Xu et al. [11] proposed a furniture layout system that utilizes pairwise relationships and constraints to help users place furniture quickly. Merrell et al. [12] proposed an interactive furniture layout system that uses internal interior design guidelines to rearrange furniture’s placements to make them look more natural together. More recently, Wang et al. [13] has proposed using deep neural networks to iteratively place furniture in a room, creating novel room layouts. However, in contrast to these works that aim to place furniture in such a way that is either functional or aesthetically pleasing, our work aims to transfer the object layout from one room to another such that they look similar.

Regarding works that utilize a physical environment for mixed-reality, Simone et al. [7] is first to propose using real-life objects in the player’s environment for interacting in virtual reality. Hettiarachchi and Wigdor [14] proposed a pipeline that pairs a virtual object to a real object in real-time, so that the user could wear an augmented-reality device to see and ‘touch’ the virtual object. Specific to control, Cheng et al. [15] demonstrated that daily objects could be used as novel controllers and Corsten et al. [16] has developed a tracker that enables user to assign novel objects as alternative controllers.

3 Methodology

In this section, the transfer mechanism for an object layout is discussed. The key idea is to use some selected features to compute an object layout discrepancy (OLD). The OLD metric is used to search for best placement of virtual objects in a physical room. In our prototype, each virtual object is paired with one another forming a relation and the object layout feature is the collection of captured relations. To simplify the computational problem by limiting the search space, we have also divided objects into three categories. Each of them has a different placement space to search. In this section, we divide the content into three parts. We will first discuss what exactly the object layout features are; then what exactly the virtual objects are; finally what exactly the method to place virtual objects in a physical room is.

3.1 Object Layout

In a broad sense, an object layout describes the spatial relations between objects (Fig. 2). Hence, it is useful to model the spatial relations between objects. First, the relation between object Oi and Oj is defined as R(i, j), where i and j are object indexes such that i ≠ j. We picked two relational features that could be used to form the object layout feature.

The first relational feature is the normalized distance Rr(i, j) between two objects.
$$ R_{r}^{ } (i, j) = \frac{{||\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{i} }} - \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{j} }} ||}}{{\mathop {\hbox{min} }\limits_{u, v} ||\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{u} }} - \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{v} }} ||}} $$
(1)
where u and v are also object indexes such that u  v and we present the position of an object O as a vector in the Euclidian space.
This distance term is useful to depict whether the relative distance of each object is maintained in the physical room. The reason why relative distance is picked over absolute distance is because there will be varying room sizes. The second relational feature is the direction Rθ(i, j). It is the direction of an object with respect to another.
$$ R_{\theta }^{ } (i, j) = \frac{{\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{j} }} - \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{i} }} }}{{||\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{j} }} - \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{i} }} ||}} $$
(2)
As mentioned, object layout discrepancy (OLD) is used to find the most suitable placements of objects such that the physical room’s object layout is similar to that of the original. The OLD is measuring the difference between each relation pair, one from the virtual room V and its correspondence from the physical room P such that
$$ OLD\left( {V,P} \right) = \frac{1}{N(N - 1)}\sum\nolimits_{i}^{N} {\sum\nolimits_{j \ne i}^{N} {(\left| {R_{{r_{V} }} \left( {i,j} \right) - R_{{r_{P} }} \left( {i,j} \right)} \right| \cdot \lambda_{r} + \left| {R_{{\theta_{V} }} \left( {i,j} \right) \cdot R_{{\theta_{P} }} \left( {i,j} \right) - 1} \right| \cdot \lambda_{\theta } + \left| {||\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{{V_{i} }} }} - \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{{V_{j} }} }} || \cdot \sqrt {\frac{Size\left( P \right)}{Size\left( V \right)}} - ||\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{{P_{i} }} }} - \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}}{{O_{{P_{j} }} }} ||} \right| \cdot \lambda_{s} )} } $$
(3)
where λ is a parameter that controls the weight of a term and N is the number of objects. The first term is comparing the difference in Rr. This term helps us to preserve the relative distance between each object such that if two objects are far away from each other in V, it will also try to be so in P. The second term is comparing the difference in Rθ. This term preserves the directional relations of objects such that if an object Oa is on the right of Ob in V, it will also try to be so in P. The dot product term is transformed such that if the directions are dissimilar, a positive cost value is produced. The third and last term is what we refer to as the size term. It takes into account of the sizes of the rooms by scaling the corresponding distance measures in the two rooms in such a way that the proposed model will not cluster all the objects in one part of the room simply to minimize OLD. A square root is imposed on the size ratio of the two rooms because the area grows quadratically with the length. It should be noted here that since V and P could be of different shapes, the relations from V cannot be guaranteed to be preserved in P. Experimental results in Sect. 4.2 suggest that OLD will become larger when the dissimilarity between the shapes of the two rooms increases.

3.2 Virtual Objects

In an ideal case, it should be possible to simply optimize OLD by finding the best possible positions for all objects. However, our experiments show that this is difficult to achieve practically as the large placement space has many local minima. Instead, a specific process is proposed to find an improved solution. The objects will first be separated manually into three categories, major-walled objects, minor-walled objects and non-walled objects. The main purpose of this categorization is to limit the placement space for each object. The designer would need to manually categorize the objects. Since this is a one-off process, we believe this is an acceptable tradeoff.

Major-Walled Object.

Objects in this category (OM) are objects that are walled and deemed important by the designer. They could be objects that are critical to the layout. It is expected that they are usually objects with greater sizes or simply deemed to be of greater importance by the designer. For example, when recreating a virtual museum, the virtual paintings of Mona Lisa and The Coronation of Napoleon I are likely to be of this category. By placing objects in this category, the proposed method would take precedence in finding the most suitable placements for them. Since these are walled objects, their search spaces are limited to the physical room’s walls Hw as shown in Fig. 3. Hence, the search space for a major-walled object is SM = [0, 1] where 0 and 1 denote the starting and ending point of the room’s wall respectively. Since the room is enclosed, 0 and 1 are equivalent in that they are representing the same point of the room. If a major-walled object indexed i, O M ( i) , is located midway along the room’s walls, we denote that Hw(O M ( i) ) = 0.5.
Fig. 3.

Illustrations on major-walled objects (OM), minor-walled objects (Om) and non-walled objects (Oo), from left to right with gradual inclusion.

Minor-Walled Object.

The minor-walled object (Om) category is for walled objects that are not major-walled objects. Since they are deemed less important to the designer, these objects compared to the major-walled objects are given secondary priority in finding the most suitable placements. It is assumed that a minor-walled object is bounded by two major-walled objects. So, not only is its search space simply of one dimension, it is also bounded by the two closest major-walled objects along the wall. So, if Om(i) has O M ( j) and O M ( k) as neighbours, we denote that Flk(O m ( i) ) = (O M ( j) , O M ( k) ). It means that O m ( i) is flanked by O M ( j) and O M ( k) such that Hw(O M ( j) ) < Hw(O m ( i) ) < Hw(O M ( k) ). An illustration of minor-walled objects is also provided in Fig. 3.

Non-walled Object.

This category as its name implied contains objects (Oo) that are simply not walled. They are the last type of objects to search for best placements. The reason is that walled objects are used as reference to find the best placement. As the objects are non-walled, their search spaces are not simply along the wall but within the entirety of the room. However, it is proposed that the search space could be limited to one dimension via drawing a line that cut the room in half. We refer to such a line as the half room line C. The object could then simply move along the line to find its best placement. So, the search space for non-walled objects is So= [0, 1]. But unlike SM, 0 and 1 do not meet at the same point in the spatial space. To avoid abrupt positional change along C, smoothing could also be considered. An illustration of non-walled objects is given in Fig. 3 as well.

3.3 Object Placement

There are two intuitions for layout transfer and they are: (1) we could approximate a virtual object’s placement then fine-tune it, and (2) we could make assumptions regarding the dependencies between virtual objects of different categories.

The exact method to approximate the placement of an object depends on the category, but for fine-tuning, the strategy is fundamentally the same. Figure 4 showcases the step by step result of the proposed object layout transfer procedure.
Fig. 4.

Illustration of the proposed object layout transfer procedure. Left to right: empty room, after major-walled object placement (approximation and fine-tuning), after minor-walled object placement and after non-walled object placement.

Major-Walled Object Approximation.

Major-walled objects OM are relatively few in numbers and are considered as a noticeable part of the designer’s layout. So, they are given priority in searching for the best placements. To approximate OM’s placement, it is proposed that each object \( O_{M}^{(i)} \) in OM first searches for a wall W in P to attached on that would minimize the object layout discrepancy (OLD). That is, we are trying to find the best (OM, W) pair for Eq. (4):

$$ min_{{(O_{M}^{(i)} , W_{j} )}} OLD(V, P) $$
(4)
where Wj denotes a W indexed j. Note that at this moment, since only OM have been copied from V to P, the relations to compute OLD are only the ones between each OM.

Major-Walled Object Fine-Tuning.

Searching which W is best for an OM does not minimize OLD. It is proposed that the second step is to fine-tune the position of all OM along the room’s encircled wall for improving the cost function. Thus, the goal is to minimize
$$ min_{{H_{w} (O_{M}^{(i)} )}} OLD(V, P) . $$
(5)

Note that since we are trying to find all Hw(O M ( i) ) that minimize OLD, the placement of an object in OM is not bounded to the paired wall in the previous step. Although that is the case, the previous approximation step should find a good enough placement for each OM such that the topological relationship among objects in OM remains the same.

Minor-Walled Object Approximation.

Once the best placement for OM has been found, the search space of a minor-walled object Om, Sm can be limited. We make an intuitive assumption that if O m ( i) is flanked by O M ( j) and O M ( k) in the virtual room, O m ( i) will also be flanked in the physical room. Given Flk(O m ( i) )=(O M ( j) , O M ( k) ), for approximating its rough placement, we simply compute
$$ \begin{aligned} H_{wP} \left( {O_{m}^{(i)} } \right) = H_{wP} \left( {O_{M}^{(j)} } \right) + \frac{{H_{wV} (O_{m}^{(i)} )_{ } - H_{wV} (O_{M}^{(j)} )}}{{H_{wV} (O_{M}^{(k)} ) - H_{wV} (O_{M}^{(j)} )}} \cdot \left( {H_{wP} \left( {O_{M}^{\left( k \right)} } \right) - H_{wP} \left( {O_{M}^{\left( j \right)} } \right)} \right). \hfill \\ \hfill \\ \end{aligned} $$
(6)

Minor-Walled Object Fine-Tuning.

Similar to fine-tuning placement of objects in OM, fine-tuning the placement of objects in Om is to minimize the OLD. The main difference is that the search space is limited such that

$$ min_{{H_{w} (O_{m}^{(i)} )}} OLD(V, P) $$
(7)
where Hw(O M ( j) )< Hw(O m ( i) )< Hw(O M ( k) ). However, the OLD computation does not only involve relations between OM, but also Om and across the two categories.

Non-walled Object Approximation.

To approximate the placement of a non-walled object \( O_{o}^{(i)} \), we need to first make use of the half room line of the virtual room V, CV, to find where roughly an object in Oo should be. CV(Oo) = (x, y) gives us the room width ratio x and room length ratio y of the object Oo. Both x and y are bounded by 0 and 1. For x, 0 represents one side of the room, while 1 represents the other side. Similar for y, 0 represents the start of the room while 1 represents the end of the room. The Oo’s approximation is done by copying CV(Oo) to CP(Oo) such that
$$ C_{P} \left( {O_{o}^{(i)} } \right) = C_{V} \left( {O_{o}^{(i)} } \right) . $$
(8)

Non-walled Object Fine-Tuning.

The fine-tuning for non-walled objects is also similar to the other two:
$$ min_{{C_{P} (O_{o}^{(i)} )}} OLD(V, P). $$
(9)

So, we simply consider searching along the room line to fine-tune a non-walled object’s placement.

4 Evaluation

We first showcase the results of the proposed model compared with two other models. Then, to give better insight on the limitations of the proposed model, the result of transferring the object layout to rooms with different shapes is shown.

4.1 User Study

We followed [11] in conducting a user study to evaluate the effectiveness of our proposed model. The purpose of the user study is to see if a human considers the proposed model and its associated implementation are capable of transferring the object layout in a convincing way. We have compared three different models (Fig. 5): our proposed model based on the method mentioned in Sect. 3; a baseline random placement model which randomly picks placements for objects; and a bounding box placement model which scales the virtual room to fit into the physical room.
Fig. 5.

Comparing the proposed model to random placement and bounding box placement with L-shape, D-shape and T-shape physical room.

In the survey, we inform the respondents that we have developed a model that could transfer object layout from one room to another. We ask them in a 5-point Likert scale question format regarding their certainty as to whether a resulting object layout is coming from a human designer. A score of 5 represents that they are most convinced that the object placement is done by a designer, while a score of 1 represents that they are most convinced that it is done by a computer instead. In the survey, we have indicated to the respondents the original object layout and the three empty rooms that the layout will transfer to. Each of the three rooms will be filled via the three aforementioned models. So, for each room-model pair, the respondent is asked regarding their certainty on whether the room’s layout is created by the designer. The assumption is that, the more convincing a model is, the more its result is able to fake a human on that it is created by a human designer. It should be noted that none of the three rooms have their objects actually placed by a human. In total, we have received 33 responses. A snippet of the survey could be found in Fig. 6.
Fig. 6.

The left is an example of a question in the survey. A first-person view is also given to help a respondent feel what the room would look like being physically inside it. The right is the chart on how certain the respondent believes that the resulting placement is created by a human designer.

As shown in Fig. 6, compared with the other two models, it is much more likely for respondents to choose the proposed model as the probable designer. To statistically compare whether the proposed model is more convincing than the other two models, two matched paired t-tests are performed. In Table 1, the rating of each model is shown on the left column while the rating difference of a model to the proposed model (meaning, how much higher/lower a respondent rate a model against the proposed model) is shown on the middle column. As shown in Table 1, it is statistically significant (p < 0.05) that the respondents rate the proposed model higher compared to the other two. This result indicates that the respondents believe that the proposed model could transfer the layout most convincingly out of the three. It should be noted that although the bounding box model could achieve a lower OLD as shown in Table 2, since it did not fully utilize the entire room, it generally fails to convince the respondents. This shows that simply copying the layout is insufficient for a good layout transfer.
Table 1.

Respondent answer statistics.

 

Mean score (SD)

Mean score difference (SD) with proposed model

p-value

(t-test with proposed model)

Random

2.798 (1.102)

0.909 (1.799)

0.0066600

Bounding Box

2.091 (1.094)

1.616 (1.446)

0.0000003

Proposed Model

3.707 (0.920)

Table 2.

The object layout discrepancy result for each model-room pair.

 

L-Shape

D-Shape

T-Shape

Random

31.492

28.921

41.100

Bounding Box

13.541

8.408

10.676

Proposed Model

9.527

13.793

10.349

4.2 Properties of the Proposed Model

To make it easier to observe its properties, the proposed model is tasked to transfer an object layout to different rooms similar to the user study. However this time, the focus is on comparing the different OLDs for each room shape. In Fig. 7, we first deploy the model on regular polygonal rooms with different number of sides. As the result indicates, a larger number of sides would result in a higher value of OLD. This is expected as when there are more sides, the less likely the model could find a placement that fits the directional requirement of the original layout (for example, that an object is supposed to be on the exact right of another according to the virtual room).
Fig. 7.

Comparison of the OLD when transferring an object layout to regular polygonal rooms with different numbers of sides (left) and transferring to rooms with ‘peculiar’ shapes (right).

We are also interested in testing the proposed model on rooms with ‘peculiar’ shapes. They are a cross-shape room, a u-shape room, a double-u room and a triple-u room as shown in Fig. 7. The result seems to indicate that the more complex a room is, the higher the OLD would be. For this case, since the rooms offer more corners for the proposed model to work with, the higher OLD seems to be related to the fact that the growth of the in-between object distances could not catch up with that of the growth in the room sizes.

Both of these results show an inherent problem with the proposed model that when the physical room is significantly different from the virtual room, the algorithm would simply have difficulty transferring the object layout. This outcome could be attributed to the fact that the algorithm has no component of creativity and therefore could not internalize the layout. It could simply attempt to seek for object placements which would make it as similar to the original one as possible. However, for some scenarios, it is suggested that the physical room could be segmented into smaller areas. The area most suitable for object layout transfer could be used. Although this approach could not utilize all of the rooms, it could improve the OLD of the transferred layout. This is also in line with the better OLD generated by the bounding box approach.

It is worth noting here that the proposed model does not consider whether the sizes of the objects are suitable with respect to the size of the physical room. If only using the proposed implementation here, a very large physical room will have large distances between objects due to the relatively small sizes of the objects. However, it is believed that this problem could be easily resolved by resizing the transferred objects in accordance to the size of the physical room as much as possible.

5 Conclusion and Future Work

One of the main problems in VR is how we could physically walk to explore a much larger virtual environment while residing in a household room. In this paper, we suggested that the virtual scene could be adapted in accordance to the physical room. Spatially speaking, the virtual environment is similar to that of the physical one and therefore the player could freely walk around in the virtual environment. We proposed a step towards this direction with object layout transfer, which could transfer the designed object layout from a virtual room to any physical room. This is useful because many games involve a specific virtual object layout created by a designer. Our method involves selecting a set of object layout features to compute an object layout discrepancy (OLD). The best placement for a virtual object could then be determined via minimizing the discrepancy. To evaluate the proposed object layout transfer model, a user study is conducted on whether the transferred layout could fool respondents that it is more likely created by a designer. The result seems to imply that the proposed model is rather successful in placing objects in a convincing way.

Currently, the algorithm only considers the room’s walls to help constrain the search space. It is believed that the next step to this line of work could be considering also physical obstacles. By strategically placing virtual objects where physical obstacles are, it could help the player prevent colliding with them. Once this is achieved, a virtual scene would be directly applicable to any home room and the player could play without any consideration of his/her room’s physical constraints. Thus, a player would be able to enjoy a more immersive VR experience where s/he can physically walk and interact with the virtual environment.

The physical rooms currently shown in this paper are simply simulations to test the feasibility of the proposed model. The user study only evaluates whether the generated layout is more convincing. The important question regarding how users will actually feel in a virtual space generated from a real physical room when playing VR is not known. Therefore, another important research direction would be to study and capture the user experience in interacting with a virtualized physical room. By using a scanned physical room and transferring an object layout to it, the user playing in VR could report his/her own feeling regarding safety, fun and immersiveness.

References

  1. 1.
    Ohyama, S.: Automatic responses during motion sickness induced by virtual reality. Auris Nasus Int. J. ORL&HNS 34(3), 303–306 (2007)CrossRefGoogle Scholar
  2. 2.
    Kennedy, R.S., Drexler, J., Kennedy, R.C.: Research in visually induced motion sickness. Appl. Ergon. 41(4), 494–503 (2010)CrossRefGoogle Scholar
  3. 3.
    Sun, Q., et al.: Towards virtual reality infinite walking: dynamic saccadic redirection. ACM Trans. Graph. 37(4), 67 (2018)CrossRefGoogle Scholar
  4. 4.
    Dong, Z.-C., Fu, X.-M., Zhang, C., Wu, K., Liu, L.: Smooth assembled mappings for large-scale real walking. ACM Trans. Graph. 36(6), 211 (2017)CrossRefGoogle Scholar
  5. 5.
    Azmandian, M., Grechkin, T., Suma, E.: An evaluation of strategies for two-user redirected walking in shared physical spaces. In: IEEE Virtual Reality, Los Angeles (2017)Google Scholar
  6. 6.
    Hunicke, R., LeBlanc, M., Zubek, R.: MDA: a formal approach to game design and game research. In: AAAI Workshop on Challenges in Game AI (2004)Google Scholar
  7. 7.
    Simeone, A.L., Velloso, E., Gellersen, H.: Substitutional reality: using the physical environment to design virtual reality experiences. In: ACM Conference on Human Factors in Computing Systems, Seoul (2015)Google Scholar
  8. 8.
    Avila, L., Bailey, M.: Virtual reality for the masses. IEEE Comput. Graph. Appl. 34(5), 103–104 (2014)CrossRefGoogle Scholar
  9. 9.
    Steinicke, R., Bruder, G., Jerald, J., Frenz, H., Lappe, M.: Estimation of detection thresholds for redirected walking techniques. IEEE Trans. Vis. Comput. Graph. 16(1), 17–27 (2010)CrossRefGoogle Scholar
  10. 10.
    Burr, D.C., Morrone, M.C., Ross, J.: Selective suppression of the magnocellular visual pathway during saccadic eye movements. Nature 371(6497), 511–513 (1994)CrossRefGoogle Scholar
  11. 11.
    Xu, K., Stewart, J., Fiume, E.: Constraint-based automatic placement for scene composition. In: Graphics Interface, Calgary (2002)Google Scholar
  12. 12.
    Merrell, P., Schkufza, E., Li, Z., Agrawala, M., Koltun, V.: Interactive furniture layout using interior design guidelines. In: SIGGRAPH, Vancouver (2011)Google Scholar
  13. 13.
    Wang, K., Savva, M., Chang, A.X., Ritchie, D.: Deep convolutional priors for indoor scene synthesis. ACM Trans. Graph. 37(4), 70 (2018)Google Scholar
  14. 14.
    Hettiarachchi, A., Wigdor, D.: Annexing reality: enabling opportunistic use of everyday objects as tangible proxies in augmented reality. In: ACM Conference on Human Factors in Computing Systems, San Jose (2016)Google Scholar
  15. 15.
    Cheng, K.-Y., Liang, R.-H., Chen, B.-Y., Liang, R.-H., Kuo, S.-Y.: iCon: utilizing everyday objects as additional, auxiliary and instant tabletop controllers. In: ACM Conference on Human Factors in Computing Systems, Atlanta (2010)Google Scholar
  16. 16.
    Corsten, C., Avellino, I., Mollers, M., Borchers, J.: Instant user interfaces: repurposing everyday objects as input devices. In: ACM International Conference on Interactive Tabletops and Surfaces, St. Andrews (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Zackary P. T. Sin
    • 1
    Email author
  • Peter H. F. Ng
    • 1
  • Simon C. K. Shiu
    • 1
  • Fu-lai Chung
    • 1
  • Hong Va Leong
    • 1
  1. 1.The Hong Kong Polytechnic UniversityHung HomHong Kong

Personalised recommendations