Tag: MacOS

Boosting Productivity with Quest 3: The Best Apps I’ve tried for Using it as a Virtual Display on Mac

Having tried a few VR headsets in the past including the original Oculus Rift and Quest 2 and finding them uncomfortable to wear or the display quality being too poor for productivity use, I decided to take the plunge with the Quest 3 when it came out. Compared to the Quest 2, the Quest 3 has a much improved optical design (pancake lenses vs fresnel on the Quest 2) and weight distribution making it comfortable to use for extended periods of time. The display sharpness is also good enough for general productivity use for many hours a day. While the display quality isn’t as good as on a Mac (sharpness, contrast, colour accuracy, lens artefacts), the ability to use virtual displays in a virtual space is nevertheless great for focus, productivity and privacy, especially when working while travelling. 

In an ideal world…

As I use the Quest 3 mostly as a virtual monitor for my M1 Max MacBook Pro, the ideal scenario would be to connect the Quest 3 directly to the Mac and have it output a DisplayPort signal, which treats the Quest 3 as a standard external display, perhaps with some processing to keep the display fixed in virtual space (something like the XReal Air 2). Unfortunately, the Quest 3 doesn’t support USB-C alternate modes and DisplayPort input, so this is probably out of the question. Alternatively, it would be great if we could stream the virtual displays over the local network, or to a hotspot created by the Quest 3 for portable use. 

While there isn’t a native way of using the Quest 3 as a virtual display on Mac, there are many apps which can help with this function. These are the ones that I’ve tried:

Immersed

This is my go-to app for Quest 3 productivity use at the moment. Much like all the solutions in this list, Immersed uses a desktop streamer app installed on the Mac and an app installed on the Quest 3 available on the Quest store. In normal use with Mac and Quest 3 connected to the same Wifi network, the virtual desktop is streamed over the local network. There is a fall-back option which routes the streams over the internet if Mac and Quest 3 are unreachable over the local network, but this option adds significant latency as well as eating into any data allowance you might have. Finally, there is a “USB mode” which is in beta and requires you to enable developer mode on the Quest 3, and gives you the lowest latency option by wiring the Mac directly to the Quest 3 via USB-C. Immersed is free to use, and the “Pro mode” gives you more features such as support for 5 virtual displays and better team collaboration features for $4.99 a month. 

Pros:

  • Immersed allows you to create virtual monitors (up to 5 in “Pro” mode) in addition to your primary displays. Like in Horizon Workrooms, these are treated as normal monitors by MacOS, i.e. open windows will be moved to their last position on the virtual monitors upon reconnection
  • While the streaming bitrate can be set quite low (to ~1MB/s while playing video), the latency is good (~50ms depending on network conditions and stream quality). You can increase the stream quality and even retina oversampling on the Mac client for even better quality at the expense of higher network utilisation and encoding latency.
  • Immersed sometimes works without an internet connection, in a plane or hotel room for example. You can set up a personal hotspot with a phone or portable router to connect the Mac and Quest 3, then stream the virtual displays over this local network, or use the low latency USB mode. However, there are some issues and caveats to this (more in cons section)
  • You can change the size/resolution of your virtual displays
  • There is an option to turn off the built-in Mac display automatically while using the Immersed virtual displays in VR, and the built-in display will automatically turn back on when you exit Immersed or take the headset off. This is a convenient power saving feature when travelling
  • There is an option to enable a “virtual cursor” which is an overlay over the “real” cursor in the Immersed app and gives the impression that the latency of your mouse movement is lower than it actually is
  • You can arrange the virtual monitors almost any way you like, including changing the position, distance and curvature
  • There is a lot of support, tips and discussion on their Discord server which is very well organised
  • Keyboard passthrough allows you to see your desk and keyboard even if you are in a virtual environment
  • You can join public rooms where you can work and chat in the presence of other Immersed users
  • In a pinch, you can use a virtual keyboard and controllers as a mouse and keyboard, but it is not comfortable enough for daily use

Cons

  • Currently, “offline mode” feels a bit like an afterthought and doesn’t always work as expected. The Immersed app on Quest 3 and Mac needs to connect to Immersed servers every now and then to authenticate your account before you could use it, even if the virtual displays will be streamed locally over wifi or USB. The staff at Immersed have confirmed that they are looking into improving the offline experience and iron out some of the bugs currently in offline mode, but as of now (October 2023), I wouldn’t rely on Immersed if I’m travelling on a plane or somewhere without an internet connection. 
  • In USB mode, while the monitors are streamed over USBC, the Mac and Quest 3 still need to be connected to the same network (ideally with an internet connection if the apps need to authenticate with Immersed servers) in order to start the session. You can disconnect from the network once the USB streaming is in session
  • While there is flexibility in arranging your displays in virtual space, Immersed currently doesn’t do a great job of saving your arrangements or having the displays show up the same way each time. I find myself adjusting the monitors slightly every time I start an Immersed session to get the position just right. 
  • The loading time on the Immersed Quest 3 app is looooonnngg. It usually takes around 35 seconds from starting the app to getting the monitors connected and positioned before I can start working.
  • There is a feature on Quest 3 which automatically detects compatible keyboards and does a passthrough for it, however I found it doesn’t work consistently (this is more of a problem with the keyboard detection feature on the Quest 3 though rather than an Immersed issue).
  • There is no easy to use “desk passthrough” feature like on Meta Horizon Workrooms. You could set up custom passthroughs to see your desk, but these are all reset if you use the “reset positions” button, which I have to use every time I start a session to get the monitors to quickly snap to their proper positions
  • While there are many quirky virtual environments to choose from, I do wish there was a nice and simple office/desk environment like in Meta Horizon Workrooms. 

Virtual Desktop

Another popular desktop streaming app for Quest 3 which supports Mac. Virtual Desktop works slightly differently to Immersed in that you can’t set up multiple virtual monitors in VR (you are limited by the number of physical monitors you have, and multi monitor support in VR is not supported in Virtual Desktop on Mac anyway). Virtual Desktop streams over the local network, or over the internet as a fallback option. However, unlike Immersed, there is no wired/USB mode on Mac. Virtual Desktop is a one-time purchase of £14.99 on the Quest store.

Pros

  • Virtual Desktop can work consistently without an internet connection, hurrah! Both Mac and Quest 3 must be on the same Wifi network (could be a hotspot on a phone or portable router), but an internet connection is not required upon startup from my testing. This is great if you are on a plane and want to get some work done in VR. Sometimes the Quest 3 and Mac have difficulty finding each other in offline mode, but restarting the apps on either end fixes this for me
  • Rather than simply mirroring the laptops display, Virtual desktop can change the resolution of the virtual display depending on its size in VR. Not only could you get a larger display in VR, but also more display real estate
  • On the “void” or passthrough modes, you can change the size, position, distance and curvature of your virtual monitor, and these settings are saved so your monitor appears exactly the same way each time you use Virtual Desktop
  • Virtual Desktop loads fairly quickly and I can usually connect to my virtual monitors within 10 seconds of launching the app on Quest 3.
  • The quality of the virtual environments is great
  • The feature where you can use the controller as mouse input works surprisingly well (although double clicking is hit and miss)
  • Virtual Desktop supports 3D video playback, and Steam VR features on Windows although I haven’t tried these

Cons

  • The quality of the virtual display video stream is really high on MacOS, even when set to the lowest setting in the app (4Mbit/s). Playing a video on a VR monitor can easily reach 4-5Mbytes/s over the network (which is around 32Mbit/s, higher than 4Mbit/s, so perhaps this is a bug?). Using MacOS mission control or scrolling through large areas of the display results in a lot of network activity compared to Immersed or Meta Horizon Workrooms. Unless you are connected by ethernet or on Wifi 6E, there will be a lot of latency and stuttering. This limits what you can comfortably do on Virtual Desktop while connected to a personal hotspot while travelling for example. I find that for emails/word documents/spreadsheets, the added latency is tolerable. While it is great that the quality of the stream is high for home use with ethernet/Wifi 6E, I do wish there was a way of lowering the stream quality even further so latency and network performance is comparable to Immersed. Much of the discussion on Virtual Desktop Discord server revolves around optimising your network and eeking out the highest bitrates and best image quality you could get, which is great, but for productivity use, a reliable stream is more important than a marginal improvement in quality you get from higher bitrates. 
  • Virtual Desktops does not support multiple virtual displays on Mac
  • There is no desk/keyboard passthrough in virtual environments. To see you keyboard, you have to be in passthrough mode (where the virtual displays are overlaid onto the passthrough), which could ruin the immersion depending on how messy your room is
  • While using Virtual Desktop, you need to turn off your Mac’s primary display manually (by turning the brightness all the way down) while in VR if you want to preserve battery life or for privacy

Meta Horizon Workrooms

While the main feature of Horizon Workrooms is for collaboration, there is also a nice virtual desktop feature too. The Virtual Desktop environment on Horizon Workrooms feels the most refined compared to Immersed and Virtual Desktop, and the desk passthrough feature is simple and intuitive to use. Horizon Workrooms streams your desktop over the local network, with fallback to internet if the devices can’t reach each other locally. Meta Horizon Workrooms is currently in beta and free to use. 

Pros

  • There are currently 3 desktop environments available, and they’re all beautifully designed
  • The default 3 monitor setup can’t be moved/changed, but is very comfortable to use at just the right size, distance and curvature for me
  • The desktop passthrough setup is amazing and simple to use on Quest 3. The passthrough appears in exactly the same place each time so there is minimal setup every time you use the app
  • The bitrate of the stream is quite low so it is not too taxing on congested local networks

Cons

  • Horizon Workrooms does not work at all without an internet connection, even if Mac and Quest 3 are on the same local network. This is a shame as it could have easily been my daily driver for VR desktop use.
  • If you set up all 3 virtual monitors, Horizon Workrooms doesn’t remember this so you have to add the two extra monitors every time you start the app. This adds about 4 seconds to the startup time, although the app launches and connects quickly so despite this I can still get to work faster than with Immersed
  • Perhaps the streaming bitrate of the virtual desktops on Horizon Workrooms is too low, as I notice a lot of frame drops and stuttering even on an uncongested Wifi 6E network/ethernet. It would be great to be able to adjust this manually
  • You need to turn off your Mac’s primary display manually (by turning the brightness all the way down) while in VR if you want to preserve battery life or for privacy
  • You can’t change the resolution or size of your virtual displays

Final Notes

  • This post was written in late October 2023, and I’m hoping that these apps would be improved over time, especially support for offline use
  • The screenshots from the Quest 3 shown here are a lot more pixellated than what I can see in real life. Also the field of view I can see is much wider than the screenshots, as they are from the left eye only and cropped
  • My setup is: MacBook Pro 14 M1 Max running MacOS 14.1 Sonoma, Meta Quest 3 with OS version 57, Wifi 6, 6E router running on DFS channel for 5GHz, or iPhone 14 Pro Max/Android AP hotspot on 5GHz

Re-visiting “Reduce Transparency” in MacOS

The introduction of translucency effects in MacOS Yosemite back in 2014 was a step backwards in usability and power efficiency as I discussed here. Translucency effects were computationally intensive to render and on older systems, can result in high GPU utilisation and frame drops during normal everyday activities like scrolling or browsing the web.

Fast forward to 2022 and the Mac hardware landscape has changed dramatically. We now have Apple Silicon SoCs that can achieve levels of performance only seen in high end systems with discrete GPUs, but with power consumption orders of magnitude lower. M1/M2 Macs can sustain high frame rates of >60fps in the MacOS UI despite translucency effects smattered all over the interface, and the sheer power efficiency of Apple Silicon should make these effects essentially free from a power consumption perspective. Right?

I decided to take a look at the impact of translucency effects on power consumption of various functional units of the M1 Max SoC including CPU, GPU and DRAM controller. To do this, I used the powermetrics utility (Terminal, sudo powermetrics), which helpfully lists clock speed of the CPU/GPU clusters and power consumption at regular intervals. asitop was used to estimate memory bandwidth. All non-essential background apps were quit and the system was monitored to ensure things like Spotlight indexing were not running during the test.

Translucency effects were toggled in System Preferences (or “System Settings” under MacOS Ventura) > Accessibility > Reduce Transparency. 

System tested: MacBook Pro 14, M1 Max 10/32, 64GB RAM

Based on preliminary testing, I found that the biggest performance/power impact of translucency effects in MacOS occur when a translucent window UI is placed over a video. In this scenario, I left a video playing in IINA (a great media player for MacOS by the way) and placed the sidebar of a Finder window over it.

Memory bandwidth of GPU, E core cluster, P core cluster and Media Engine determined by asitop during video playback with Finder overlaid. “+T” is with transparency on, “-T” is with transparency turned off. Average of 10 readings taken during the test, error bars = SEM.

First, let’s start with memory bandwidth. We would expect that translucency effects will increase demands on DRAM due to the additional rendering of window UI, and for GPU and P Cores this is exactly what we see. Interestingly, E Core memory bandwidth decreased slightly which was unexpected, and the Media Engine was unaffected by translucency effects. Although there were differences in memory bandwidth requirements between translucency on or off, the numbers here are quite small considering that the M1 Max DRAM is capable of just over 400GB/s. To be honest, I wouldn’t expect these results to make any meaningful difference in performance of everyday workloads. 

I suspect that after usability concerns, power consumption would be a big reason to turn translucency off in MacOS. For example, in light everyday workloads, ensuring that the SoC is able to stay in idle/power gated states by reducing CPU utilisation of background tasks etc can make the difference between 5 or 9 hours of battery life. So what impact does translucency have on power consumption of Apple Silicon?

Power consumption of GPU, CPU (E + P core clusters), DRAM (controller most likely?) and package power determined by powermetrics during video playback with Finder window overlaid. “+T” is with transparency on, “-T” is with transparency turned off. Average of 10 readings taken during the test, error bars = SEM.

With translucency turned on, the power consumption of all the functional units I could see on powermetrics was higher, especially the CPU and GPU which consumed around 50mW more compared to translucency turned off. DRAM power increased slightly, although the difference was masked by variation in the readings. The impact of translucency on CPU, GPU and DRAM (as well as other blocks of the SoC which I was unable to measure) resulted in a ~200mW increase in total package power. A 200mW increase in power consumption equates to around 20 minutes less battery life on a 14 inch MacBook Pro.

“Reduce transparency” option in System Settings on MacOS Ventura

So, should you turn off translucency in MacOS?

The answer depends on your personal preference (subjective) and the impacts on system performance (a little more objective). Prior to the Apple Silicon transition, I opted to reduce transparency effects on my Intel Macs and would have recommended others to do the same based on the impacts on system performance and power consumption alone. However, those on Apple Silicon need not worry so much about the performance/power consumption impacts of translucency effects (unless squeezing every last minute of battery life is the top priority). 

Power Consumption Implications of Liquid Retina XDR/MiniLED on MacBook Pro

MiniLED, baby

LCD Backlight comparison (source TCL, via CNET)

Apple’s shift towards MiniLED display technology on the 2021 M1 Pro/Max MacBook Pro’s represents a huge leap in image quality as many reviewers have found. Contrast and colour accuracy are best in class, and blooming (a common issue with Full Array Local Dimming (FALD) displays) is well controlled.

However, one aspect of MiniLED which I haven’t seen discussed in as much detail is power consumption. The platform power consumption of Apple Silicon Macs (and to a lesser extent, recent Intel/AMD based notebooks) has become so low that for many everyday tasks, it is the display, not the SOC, which is the primary power draw. I regularly notice massive battery life fluctuations on my 14 inch MacBook Pro despite workload being the same, which can be attributed to display brightness.

So how do MiniLED displays compare to non-FALD LED displays or OLED panels? Well, you won’t find the answer here, but hopefully my observations below will bring us a little closer to the answer.

Method

To obtain display power, I measured total system power consumption at idle (iStat Menus) and subtracted platform power (which was calculated as total system power with display off). A better way to do this would have been to probe the display power rails directly, but this was out of the scope of a lunch-break project (and my expertise).

FALD displays work by dimming or turning off backlight zones in areas of the image which are dark or black, which can reduce display power consumption. Therefore, the impact of Average Picture Level (APL) was also measured by using test images with solid blocks of black and white at various ratios.

Results

Display power consumption under various brightness and APL conditions on 2021 MacBook Pro 14

As expected, higher display brightness corresponds to higher power consumption up to a maximum of ~6.1W for SDR (500nits) content. The difference between 100% brightness and 6.25% (one step above “off” on the brightness controls) corresponds to approximately 7.5 hours vs 20 hours of battery life under light usage, which is a huge range.

Power draw increases dramatically after ~50% brightness. The almost-exponential increase in power consumption at higher display brightness can be attributed in part to the luminous efficacy of LEDs, which decreases logarithmically with increasing power. Furthermore, our eyes are less sensitive to changes in brightness at higher brightness levels, and so the display brightness controls on the MacBook Pro are non-linear to accommodate for this.

Viewing HDR content pushes the display up to 1600nits peak brightness or 1000nits sustained. I don’t have a way of quantifying brightness or APL of my HDR sample videos yet, but clearly running the display at 2 or 3X the brightness of SDR will cause power consumption to skyrocket, as can be seen in the graph above.

Saving power with dark mode?

When taking APL into account, lower APLs result in lower power draw at every brightness level, which I am sure is a surprise to no-one. However, this result got me thinking, maybe we could reduce display power (and increase battery life) by changing the UI of the display to reduce APL. Fortunately, MacOS features Dark Mode which does just that.

Opinion Alert: I much prefer light mode over dark mode in MacOS and iOS, but that’s a rant for another post. Anyway, now I that I have offended half the MacOS userbase..

Display power consumption on 2021 MacBook Pro 14 in Light and Dark mode at maximum brightness viewing a static document on Microsoft Word

Well would you look at that, the difference between dark and light mode is 0.4W, which is probably the first surprising result of this post. For context, this translates to a measly 20 minute difference in battery life, and not what I expected based on the brightness/APL test graph above. My guess is that the high contrast white text causes the backlight in those areas to stay on at high brightness, which is what would have happened anyway in light mode (black text on white background). However, there are large areas of the display devoid of text which are darker in dark mode than light mode, and I suspect the small power saving I observed in the dark mode scenario would be due to this. It’s important to note that an OLED display would see a much greater relative power savings in this scenario, as OLEDs have per-pixel brightness control.

So, the MiniLED display of the MacBook Pro can be more frugal in terms of power consumption when the backlight is given an opportunity to turn off in certain parts of the display (as can be seen in the brightness/APL graph testing methodology). However, in mixed everyday use, even in dark mode, the power savings were less pronounced. Based on my brief observations, it would appear that the main benefit of Apple using MiniLED in the 2021 MacBook Pros was for image quality and HDR support, rather than power savings.

Fan Control on MacBook Pro

MacBooks tend to run their fans conservatively, opting to prioritise low noise at the expense of higher surface temperatures. This is particularly true for Intel Macs which can get much warmer during “normal” use unlike their Apple Silicon counterparts.

Fan control applications like Macs Fan Control of TG Pro allow you to set custom fan curves for your Mac. Setting the custom fan curve is as simple as selecting a temperature sensor to key off, then setting max and minimum temperature.

You could even set the fan curve to be less aggressive than the default, for example, by setting a constant RPM. This may be useful if you absolutely need low noise, for example if you’re recording audio during CPU intensive tasks.

At high temperatures and low fan speeds, the CPU/SoC and GPU will throttle down which keeps them within safe operating temperatures, but other components such as the battery will likely degrade faster, just something to keep in mind.

An interesting tidbit is that all modern MacBooks with two fans tend to have very different max RPM values for each fan, and both fans never seem to run at the same speed when ramping up. I suspect that Apple is intentionally spinning the two fans at different speeds to smooth out the sound signature and to prevent the high pitched noise so common on other laptops. MacBook fans tend to sound like a “whoosh” rather than a “whine”.