Today, the US Patent and Trademark Office has officially granted Apple a patent for a potential future mixed reality headset with three internal displays. What’s new is that Apple is considering adding two lower-resolution “peripheral displays” located on both the left and right sides of the central display area to provide a more natural immersive experience for viewing content such as games. This is something that is lacking in other headphones on the market today. A second patent was granted this week for hints at how AR/VR manual controllers can now adopt a combination of buttons and a clickpad on the Apple TV 4K remote.
In the background of Apple’s patents, they note that challenges can arise in presenting virtual reality content to the user. If caution is not taken, The display structures used to deliver content will not cover the user’s peripheral vision, detracting from the immersive nature of the user’s VR experience.
The patent granted by Apple covers a head-mounted device that provides a screen. The head-mounted device can be used to display virtual reality content to the user. To enhance the user’s immersion in the content being presented, the screen may have a peripheral portion that covers the user’s peripheral vision.
Surround display part It may use individual LEDs or other low-density pixels that display content at a lower resolution than the central part of the screen. Due to the reduced visual acuity of the user at the periphery of the user’s visual field, a reduction in the resolution of the peripheral display portion relative to the central portion will not be noticeable to the user.
The presence of the content in the terminal screen will help to cover all parts of the user’s vision and thus will enhance the immersive effect of the head-mounted device when using the head-mounted device to present VR content to the user.
Furthermore, the headset may contain a gaze-tracking system that monitors the user’s eyes in eye boxes to gather information about the direction of gaze of the user’s eye. During operation, the control circuits of the electronic device may use the gaze direction information to adjust the surround content on the surround portion to correct mismatch caused by the parallax between the surrounding content and the central content on the central portion of the screen. The control circuit may also remove peripheral pixels that are marked as not displayable based on gaze direction information.
Diffuser structures can be used to help hide the boundaries between the central and peripheral parts of the display. Diffuser structures can be formed from lens mount structures that support the lenses or a separate diffuser layer. The Neutral Density Filter can be used to reduce the brightness of the pixels in the surround portion of the display. PWM plots can also be used to regulate pixel density.
Apple FIG patent. 10 below is a front view of an illustrative screen showing areas with content that can be expanded across the peripheral display panes to hide the interfaces between the central and peripheral display panes; Figs. 11 is a top view of an illustration screen showing how the pixels on the screen can be selectively removed or otherwise modified based on the direction of gaze.
Apple FIG patent. Figure 14 above shows how the #14C center display and/or the #14P surround display can be formed from solid display panels (such as planar panels). A #14C display may be, for example, a flexible screen or a rigid screen made up of a rigid substrate. A 14P display may be, for example, a printed circuit board (for example, a rigid printed circuit board made up of a solid printed circuit board material such as a fiberglass-filled epoxy) on which the individual #24P pixel LEDs are mounted.
Apple FIG patent. Figure 15 above shows how the #24P pixels in the #14P peripheral part can be mounted on a molded polymer support structure or other support structure with a curved inner surface profile (support structure #86).
For more details, see Apple granted patent 11,450,297
David Lum: Senior Systems Engineer (19 years veteran)
Shubham Gandhi: Instrumentation Engineer, Displays
Tyler Melhem: mechanical design engineer
The profiles of Apple engineers William Sprague and Pretesh Mascarenhas have not been found.
Continuous touch input on multiple independent surfaces
Apple also received a patent on Tuesday that essentially covers the new Apple TV remote sold in a 4K box. So technically, a patent was fulfilled.
Buried in the patent, however, is a small patent point that states, as with most patents, that the invention is not limited to the Apple TV remote control. Apple states in Patent Point No. 0031: “Figures 1A-1C show various examples of controllers (#100, #102, and #104) to provide remote control for another electronic device such as, for example, a separate computing device, a display ( For example, a TV or computer monitor) or a device connected to a display (for example, a DVD player, BLU-RAY(R) player)Augmented reality (AR) device, Virtual reality (VR) devicea streaming device connected to the Internet (for example, an APPLE(R) TV, laptop or desktop computer), a game console, and similar devices and combinations thereof).
While it’s currently a tip from Apple’s engineering team and nothing more, it’s an interesting and timely tip that wasn’t part of previous Apple TV remote patents. For more information on this topic, the review was granted patent 20220291778
Many Apple fans already know the clickpad that was part of the original iPods and is now part of the 4K remote. Therefore, it would be a fairly natural approach to adopt.
Apple could easily assign action buttons along with its clickpad to gaming headset controllers for a future mixed reality headset, if that’s the direction Apple decides to take. In the bigger picture, Apple’s patents show us that their engineering teams are working on several possible accessories for the headphones including rings, finger sticks and VR gloves filled with sensors. Time will only tell what accessories Apple will offer at launch time.