On August 11, 2011, the US Patent & Trademark Office published a grand patent application from Apple that reveals their plans of integrating mini or pico-like projectors into future iOS devices while introducing a likely projector accessory for MacBooks. Apple has been working on this idea for some years now. In June 2009 Apple first hinted that they would one day integrate a projector system into iOS devices. In March 2010 Apple's broad patent application came to light concerning networked projection systems and later in April 2010 we learned that Apple's research teams had begun working on integrating a projector into a MacBook. Today's incredibly detailed patent application reveals how they're working on pico-like projectors for iOS devices and how these projectors will work with a shared workspace in presentations. Apple's patent even details how they'll introduce advanced gesturing that will be able to interpret shadow and silhouette gesturing associated with presentations in a darkened environment. The level of detail associated with this patent would suggest that Apple's development teams are moving full steam ahead on the projection system project which will further push the likes of Apple's iPad into the enterprise.
Patent Overview of Shared Projected Workspace
Consumers frequently store many types of data on electronic devices, such as cellular telephones, portable media players, laptop computers, and personal data assistants. For example, consumers may store photographs, movies, presentations, messages, documents, databases, and spreadsheets on electronic devices. Due to the increasing portability of electronic devices, consumers frequently share data stored on electronic devices with other people. Many electronic devices include display screens that allow a consumer to display the data and allow viewing of the data by other people. However, to facilitate portability, these display screens may be small, which may inhibit viewing by larger groups of people. Further, to facilitate viewing on the display screen, the electronic device may be transferred between viewers, which may be inconvenient.
Apple's invention generally relates to electronic devices that may be connected to one another over one or more communication links to produce a shared projected workspace. In accordance with certain embodiments, each electronic device may include a projector that may be used to produce a projected display.
Two or more projected displays, each from a separate electronic device, may be displayed proximate or adjacent to one another to produce the shared projected workspace. Images displayed on one of the projected displays within the shared workspace may be transferred and/or copied to another projected display within the shared workspace.
Apple's invention is directed to electronic devices and systems that may be employed to create a shared workspace. The shared workspace may include two or more projected displays, each generated by a separate electronic device. Images may be transferred between the projected displays within the shared workspace. For example, the electronic devices may communicate with each other over a communication link that allows data for images to be transferred between the electronic devices for display on different projected displays within the shared workspace. In certain embodiments, the electronic devices may communicate directly with one another. However, in other embodiments, the electronic devices may communicate through an intermediary device, such as a server.
Device Cameras that Could Detect Gestures
According to certain embodiments, the electronic devices may include a graphical user interface (GUI) for transferring images between the projected displays. The electronic devices each also may include a camera that detects user gestures on the projected display. In these embodiments, a user may perform a gesture to transfer images between the projected displays.
The electronic devices may include projectors for producing the projected displays, as well as cameras for detecting gestures made with respect to the projected displays. According to certain embodiments, the electronic devices may include a library of gestures that correspond to image sharing commands. For example, different gestures may represent different image sharing commands, such as an image transfer command for moving an image from one projected display to another projected display and an image copy command for displaying a copy of an image on one projected display on another projected display. To share images between projected displays, a user may perform a gesture with respect to the projected displays. The camera for the electronic device producing that projected display may detect the gesture, and in turn, the electronic device may interpret the gesture and perform the associated image sharing command.
Apple first revealed that they were working on camera-related gesturing in February 2010.
Overview of System: One Example of a MacBook with Projector Accessory
As shown in patent FIG. 1, electronic device 10 may include various internal and/or external components that contribute to the function of the device which could be any form of MacBook, any form of desktop computer or any iOS device.
Apple's communication interface 28 could include access to 3 and 4G networks; use device identification protocol "Bonjour;" use an NFC device that may be used to receive information, such as the service set identifier (SSID), channel, and encryption key, used to connect through another communication interface, such as a WAN, LAN, or PAN interface. The NFC device may allow for close range communication at relatively low data rates (424 kb/s) or it may allow for close range communication at relatively high data rates (560 Mbps) which comply with Sony's TransferJet protocol.
Apple's patent FIG. 2 illustrates a projector accessory connected to a MacBook Pro. Apple may introduce their own and/or work with various third party developers to make their pico projectors compatilbe with the Apple's gesturing system. Time will tell how that transpires.
Advanced Camera Feature that Detects Shadow and Silhouette Gestures
One of the wilder aspects to Apple's advanced camera 30 that will be used in this new system is being designed to detect gestures in the form of shadows and/or silhouettes shown on the projected display created by projector 16. For example, the camera may be designed to detect spatial patterns, such as shadows, produced on the projected display. The camera may also be designed to detect movement of an object, such as hand, that produces gestures within the viewing area of the camera.
The Camera may operate in conjunction with image processing software included within storage 24 to interpret gestures for sharing or moving images between projected displays of the shared workspace. For example, the storage may include a library of gestures that may correlate to image sharing commands that allow a user to move images between different projected displays of the shared workspace.
Apple's iOS Devices with Integrated Projectors
Apple's patent FIG. 3 shown below depicts a system of electronic devices 10B (iPhone) and 10C (iPad) that may be employed to produce a shared workspace. Although the cameras may be integrated into these iOS devices, Apple does mention alternative ways to clip or snap a projector onto the housing of various other devices – such a Mac Mini.
Electronic devices 10B and 10C may be placed on a table to produce shared workspace 40 on a surface 44 such as a wall, screen or whiteboard, among others. In other embodiments, electronic devices 10B and 10C may be held by a user or mounted on a tripod.
To produce shared workspace 40, each projector 16B and 16C may produce a projected display 48B and 48C that may be produced within the same location to allow a user to interact with both of the projected displays. According to certain embodiments, the projected displays may be disposed adjacent to each other to form a unified display. However, in other embodiments, the projected displays may be produced farther apart and/or on separate walls of a location. Further, in certain embodiments, the projected displays may be positioned above and below one another.
Using Image Sharing Mode
According to Apple's invention, users will be able to interact in a shared workspace. For instance, a user may select images 50 and 52 on their touchscreens initiating an "image sharing mode where images may be transferred between projected displays 48B and 48C.
To share images between projected displays 48B and 48C, a user may perform one or more gestures on a projected display. Each device 10B and 10C includes one or more cameras 30B and 30C that may be used to detect gestures performed in front of the projected displays. For example, a user may stand adjacent to shared workspace 40 and gesture in front of one or both of projected displays. The user's gesture may cast a shadow on the projected displays that the respective cameras may detect. In other embodiments, the camera 30B or 30C may detect the gesture itself, rather than the shadow of the gesture. Electronic device 10B or 10C may interpret the detected gesture and may correlate the gesture with an image sharing command.
Image Sharing Commands
According to Apple's invention, Image sharing commands may include various types of commands, such as copy or transfer commands, for sharing an image shown on one projected display 48B with one or more other projected displays 48C within the shared workspace. Each image sharing command may be represented by a different gesture. For example, a user may place a hand or other suitable object in front of an image 50 and may flick his hand to the right to produce a gesture that electronic device 10B recognizes and uses to transfer image 50 from projected display 48B to projected displays 48C.
In another example, a user may hold onto an image with a fist and move the fist to the right to produce a gesture that electronic device 10B recognizes as a command to copy image 50 and transfer a copy of image 50 to projected display 48C. In this example, image 50 may then be displayed on both projected displays 48B and 48C. Any number of various types of gestures may be associated with different image sharing commands or functionalities.
Gesture Specifics
Apple's patent FIG. 4 shown below is a front view of the system of FIG. 3 illustrating a gesture for sharing images within a shared workspace; Patent FIG. 5 is a front view of the system of FIG. 3 illustrating another gesture for sharing images within a shared workspace,
According to certain embodiments, electronic devices may interpret the gesture based on the shape, direction, speed, acceleration, and/or magnitude of the gesture. Accordingly, a user may vary the shape of object 54, the orientation of object 54, the direction of the gesture, the speed of the gesture, the acceleration of the gesture, and/or the magnitude of the gesture to perform different image sharing commands. For example, a user may display an open palm or a closed fist to perform gestures for executing other types of image sharing commands, such as copying images, displaying images on multiple projected displays, displaying an image on a unified space that includes the projected displays, or exchanging images between projected displays, among others. In another example, a user may change the number of fingers that are extended during the gesture.
Apple's patent FIG. 5 depicts another example of a gesture that may be used to share images between projected displays 48B and 48C. In this example, a user may arrange object 54 into a fist that moves from a first position 58 to a second position 59 as shown by an arrow 60. Camera 30B may detect the gesture, and electronic device 10B may interpret the gesture to retrieve an imaging sharing command associated with the detected gesture.
Unified Display Mode
Apple's patent FIG. 6 shown below is a front view of the system of FIG. 3 illustrating a unified display mode for a shared workspace; FIG. 7 is a front view of another embodiment of a shared workspace system between three devices.
Various Graphical User Interfaces Associated with the Shared Workspace
Apple's patent FIG. 8 is a front view of a GUI screen that may be employed to assign electronic device positions within the system of FIG. 7; FIG. 9 is a front view of a GUI screen that may be employed to select images for display on the shared workspace; and FIG. 10 is a front view of a GUI screen that may be employed to transfer images between projected displays within the system of FIG. 7.
Virtual Bins
As shown in Apple's patent FIGS. 11 and 12, components of the GUI also may be displayed on projected displays 48 to facilitate performance of gestures for sharing images within a shared workspace. For example, as shown in FIG. 11, a graphical representation 108 of shared workspace 40 may be shown on projected display 48B. Graphical representation 108 includes virtual bins 110, 112, and 114, each corresponding to one of the projected displays within shared workspace. As shown, virtual bin 110 represents projected display 48B; virtual bin 112 represents projected display 48C; and virtual bin 114 represents projected display 48A (as shown in FIG. 7 above).. Although graphical representation 108 is shown at the bottom of projected display 48B, in other embodiments, graphical representation 108 may be shown in other locations, such as the top and/or sides of projected display 48B.
To share an image with another projected display 48, a user may perform a gesture that moves an image to a virtual bin 110, 112, or 114 shown within graphical representation 108. For example, a user may perform a gesture that moves image 98 to virtual bin 112 to display image 98 on projected display 48C (FIG. 7).
Apple's patent FIG. 15 is a schematic view of methods for sharing images within a shared workspace.
Apple's patent application 20110197147 was originally filed in Q1 2010 by inventor Anthony Fai.
It should be noted that MacRumors noted last Friday that Apple had registered the domain name Applepico.com. The timing of that news is very interesting indeed.
Other Noteworthy Patent Applications Published Today
We may list other patent applications here later today.
Notice: Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. Apple's patent applications have provided the Mac community with a clear heads-up on some of Apple's greatest product trends including the iPod, iPhone, iPad, iOS cameras, LED displays, iCloud services for iTunes and more.
About Comments: Patently Apple reserves the right to post, dismiss or edit comments.
Here are a Few Great Community Sites covering our Original Report
MacSurfer, StumbleUpon, The Guardian UK, Inside Mobile Apps, MobilBranche Germany, Aberto ate de Madrugada Portugal, Fox News Houston, Paper.Li Switzerland, FierceWireless, Agoracom, Samsung.net/Service, Wall Street Select, Huston Chronicles' TechBlog, Technology Headlines from Around the Web NY Times, Reddit,Twitter, Facebook, Apple Investor News, Google Reader, UpgradeOSX, TechWatching, Macnews, iPhone World Canada, StockTwits and CBS MarketWatch.
MacDailyNews, Techmeme, Hear Netherlands, 9to5 Mac, Geeky Gadgets, MacRumors, iPhoneclub Netherlands, Apple Bitch, T3 UK, iFun iPhone Germany, PicoPOPdisplay, Gizmovil Spain, Karthik Geek Center, Mac Life, iGadgets Report, and Pad Gadget, Venture Beat, App Advice, CNET, SlashGear, iPhoned Netherlands, Blogrunner NYT, Gizmodo, Technologizer, iDownloadBlog, I4U News, and Computerworld Norway (English)
Wired Gadget Lab, TUAW, PC Magazine, and Exame Brazil, Tech.Nologia Spanish, iPhoneinCanada, CNET Japan, Washington Post, One Gadget Russia, Developpez France, Automatisering Gids Netherlands, V2 Finland, iPhones.Co Israel, ITProPortal, CiberPais EL Pais Spain, Digit Russia, Monkey Business Portuguese, Trusted Reviews UK, Macity Italy, BetaBeat and Krystian MacKozer Kozerawski Poland
The Week, Tietokone Finland, Gad Russia, The iPhone 6 News Blog, Video Technology, Folha Brazil, MacViki Hungary, The Atlantic Mobile, iPhone Hacks, The iPhone 5 News Ticker, iPadd France, Engadget Germany, FWD Magazine Belgium, Huffington Post, The Center Hungary, Kompyulenta Russia, Memes Gestao Brazil, Windows ISO, Blog TV InOff Brazil, EXPreview China, Simonblog, Know Your Mobile India, Canoe French-Quebec, The Gadget Guy Tampa Bay, Computerworld Denmark, and more.
Note: The sites that are linked to above offer Apple community members with an avenue to make comments about this report in many original languages. Additionally, many of these sites provide our guests with different takes on any given patent, concept or lawsuit that is presented in our reports to make it more fun, interesting and/or personal. If you have the time, join in!
