Interacting at a Distance Using Semantic Snarfing

  • Brad A. Myers
  • Choon Hong Peck
  • Jeffrey Nichols
  • Dave Kong
  • Robert Miller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2201)


It is difficult to interact with computer displays that are across the room, which can be important in meetings and when controlling computerized devices. A popular approach is to use laser pointers tracked by a camera, but interaction techniques using laser pointers tend to be imprecise, error-prone, and slow. Therefore, we have developed a new interaction style, where the laser pointer (or other pointing technique such pointing with a finger or even eye tracking) indicates the region of interest, and then the item there is copied (“snarfed”) to the user.s handheld device, such as a Palm or PocketPC handheld. If the content changes on the PC, the handheld.s copy will be updated as well. Interactions can be performed on the handheld using familiar direct manipulation techniques, and then the modified version is sent back to the PC. The content often must be reformatted to fit the properties of the handheld to facilitate natural interaction.


Mobile Device Interaction Technique Laser Pointer Text String Computerize Device 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bier, E.A., et al. “Toolglass and Magic Lenses: The See-Through Interface,” in Proceedings SIGGRAPH'93: Computer Graphics. 1993. 25. pp. 73–80.CrossRefGoogle Scholar
  2. 2.
    Eckert, R.R. and Moore, J.A., “The Classroom of the 21st Century: The Interactive Learning Wall.” SIGCHI Bulletin, 2000. 23(2): pp. 33–40.CrossRefGoogle Scholar
  3. 3.
    Horn, G.A.V., “Proxima’s new Ovation+ projection panels do up multimedia.” Byte (on-line), 1995.
  4. 4.
    Kirstein, C. and Muller, H. “Interaction with a projection screen using a cameratracked laser pointer,” in Multimedia Modeling; MMM '98 Proceedings. 1998. pp. 191–192.Google Scholar
  5. 5.
    Myers, B.A., An Implementation Architecture to Support Single-Display Groupware. Carnegie Mellon University School of Computer Science Technical Report, CMU-CS-99-139 and Human Computer Interaction Institute Technical Report CMU-HCII-99-101, May, 1999.
  6. 6.
    Myers, B.A., et al. “Extending the Windows Desktop Interface With Connected Handheld Computers,” in 4th USENIX Windows Systems Symposium. 2000. Seattle, WA: pp. 79–88.Google Scholar
  7. 7.
    Myers, B.A., Stiel, H., and Gargiulo, R. “Collaboration Using Multiple PDAs Connected to a PC,” in Proceedings CSCW'98: ACM Conference on Computer-Supported Cooperative Work. 1998. Seattle, WA: pp. 285–294.
  8. 8.
    Narayanaswami, C. and Raghunath, M.T. “Application Design for a Smart Watch with a High Resolution Display,” in Proceedings of the Fourth International Symposium on Wearable Computers (ISWC'00). 2000. Atlanta, Georgia: pp. 7–14.
  9. 9.
    Nichols, J.W. “Using Handhelds as Controls for Everyday Appliances: A Paper Prototype Study,” in ACM CHI'2001 Student Posters. 2001. Seattle, WA: pp. 443–444.
  10. 10.
    Olsen Jr, D.R. and Nielsen, T. “Laser Pointer Interaction,” in ACM CHI'2001 Conference Proceedings: Human Factors in Computing Systems. 2001. Seattle, WA: pp. 17–22.Google Scholar
  11. 11.
    Raymond, E.S., The New Hacker’s Dictionary. Second Edition ed. 1994, Cambridge, MA: The MIT Press. See also: Scholar
  12. 12.
    Rekimoto, J. “Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments,” in Proceedings UIST'97: ACM SIGGRAPH Symposium on User Interface Software and Technology. 1997. Banff, Alberta, Canada: pp. 31–39.Google Scholar
  13. 13.
    Rekimoto, J. “A Multiple Device Approach for Supporting Whiteboard-based Interactions,” in Proceedings SIGCHI'98: Human Factors in Computing Systems. 1998. Los Angeles, CA: pp. 344–351.Google Scholar
  14. 14.
    Rekimoto, J. and Saitoh, M. “Augmented Surfaces: A Spatially Continuous Work Space for Hybrid Computing Environments,” in Proceedings SIGCHI'99: Human Factors in Computing Systems. 1999. Pittsburgh, PA: pp. 378–385.Google Scholar
  15. 15.
    Richardson, T., et al., “Virtual Network Computing.” IEEE Internet Computing, 1998. 2(1): pp. 33–38. Scholar
  16. 16.
    Shim, R., “First look at MS’ stinger'-based phone.” ZDNet UK Online, 2000.
  17. 17.
    SMART Technologies, “SMART Board 580,” 2001.
  18. 18.
    Weiser, M., “Some Computer Science Issues in Ubiquitous Computing.” CACM, 1993. 36(7): pp. 74–83. July.Google Scholar
  19. 19.
    Winograd, T. and Guimbretiere, F. “Visual Instruments for an Interactive Mural,” in ACM SIGCHI CHI99 Extended Abstracts. 1999. Pittsburgh, PA: pp. 234–235. http://graphics.Stanford.EDU/projects/iwork/papers/chi99/.

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Brad A. Myers
    • 1
  • Choon Hong Peck
    • 1
  • Jeffrey Nichols
    • 1
  • Dave Kong
    • 1
  • Robert Miller
    • 1
  1. 1.Human Computer Interaction InstituteCarnegie Mellon UniversityPittsburgh

Personalised recommendations