“Act On Press” - John Carmack's Hot Take On UI

This John Carmack tweet got me thinking. Like, REALLY thinking. onMouseDown might be my goto in the future

SOURCE

S/O Ph4se0n3 for the awesome edit 🙏

[tdotgg]

My video sparked debate on Twitter, and John Carmack left some (really good) comments. Also pointed out that Chrome changes tabs on press, not on "click".

"To perform an action with act-on-release you need to both hit the button, then keep the pointer on target until you perform the release action. If you have a jittery point location, as you do with hand tracking, controllers held away from the body, or old age, this can result in many fail-to-activate cases.

You also want generously sized and spaced buttons, but that is an orthogonal effect."

[BrandonFurtwangler]

Since I was the person that implemented the original OpenGL based keyboard on Quest (originally GearVR), I thought I’d provide some additional context.

John is correct wrt static UI, but I was also one of the people pushing back on his requests to apply this more broadly (such as to nearly everything). Although, I was not the person requesting a study, as the keyboard case was obvious to me ONLY because a text keyboard can easily roll-back characters if you end up holding the key (try holding “n” on the stock iphone keyboard and also note they do act on release).

The main issue is still how often buttons are in scrollable UI. At one point I sent him several screenshots with all scrollable containers tinted red (most of the screen) to illustrate how common it was.

In this video, the impression was given that scrollable containers aren’t a concern, but that was only because the mobile browser ignored your request to act immediately on press/down, and delayed invoking your handler until it could determine you weren’t scrolling. Thus you also weren’t getting the full latency improvements John was advocating for.

In the end, there is some validity to John’s argument but it’s not as simple as just switching things over to press/down events.

* UI frameworks would need to be updated to do a delayed down event while detecting intent-to-scroll

* not-currently-scrollable containers are more common that never-scrollable so they could certainly benefit opportunistically at the cost of minor inconsistency

* any press-and-hold interfaces (think context menus) would need to be replaced or support roll-back logic.

* any drag & drop UI would need similar intent-to-drag delays

Also note - on mobile, intent-to-scroll (or drag) is significantly easier/better than in VR because a touch digitizer samples input at 300-1000hz vs 72-120hz in VR for controllers and significantly less for hand tracking. The precision of your finger is also higher on mobile vs. VR controllers and esp. hand tracking (requiring bigger deadzones and/or delays).

So there you have it - it’s not like the eng at Meta were dumb and didn’t want to have latency improvements, it’s just that we actually have to implement/support all this along with all the edge cases and it’s competing with a million other requests. If it was as trivial as just replacing click with onmousedown, we would have done that years ago.

[imaron]

The whole "undoing" a click is HUGE for me, and I'd hate to see it go...

[rikschaaf]

HTMX supports "fetch on press, show on release", which massively cuts back the time it takes to navigate between webpages. It also supports "fetch on hover", which saves even more time, because it usually takes ~100ms before you press after a hover. It does risk fetching too much, like when you quickly swipe your cursor across a menu, but I believe they thought of that too and solved it for most cases.

[yapdog]

No. Don't just do this everywhere. Understand your game/app. Then apply it where it's needed.

[joostkersjes]

Assistive technologies have far better support for the click event than the mousedown event. The click event also includes handlers for keyboard interactions with the space bar or the return key. Please don't forget about accessibility!

[WorstDeveloper]

I think it's better to trigger on mouse up most of the time due to the user being able to cancel accidental clicks.

[Supra]

This not only goes against accessibility, but also user expectation

Imagine 99.9% of websites follow one standard, but you decide to go against it, just so your contact button has a potential saving of ~10ms

I especially hope no one uses on press for important actions, such as confirmations

[nevaknowmanamesame]

He is wrong. So many times I've held down on a button, then realized I've changed my mind. So I can release it without clicking.

[SuperGylden]

And then there's WCAG, Success Criterion 2.5.2 Pointer Cancellation

[Winnetou]

One thing to mention: on things that should be on release, you can still have a little effect on press. Buttons are like that, to have 4 states: normal, hover, active (aka on pressed, no released) and disabled .... if I remember correctly. Having something like a color change on press would still maintain the feeling of very fast interaction.

[CottidaeSEA]

I agree with him in a game.
I do not agree with him on the web.
People tend to forget that Carmack is a game developer, do not apply his wisdom outside of games, that does not really work.
You can absolutely do things on press even on the web, increment and decrement quantity in a shopping cart for example. If it is easy to revert the change, go ahead. If not, use the release event.

[MrHerbalite]

I really like Cormacks advice. I get the point of cancelling clicks, and also about people being used to the current patterns. However many of us would use the Intenet more conciously if there was an immediate action. There might be even better UI expereince invented than what we have today. To those who disagree with Cormack, I have a question. Did you think through how UI needs to be adjusted? Here is something that I suggest browser vendors could give us, no matter what opinion we have on this topic on accessibilty: Let the browser users decide in user settings how the mouse-down event should be applied. Some want an immediate response, some want to keep the ability to be able to cancel the event. Everyone will be happier if we can customize the expierence, and I could think of even other ways to give each one of us a better UI experience than what we have today, if we had more choices. Configure the mouse-down event to acct as a double click, colors, gradients, , form of button, etc. accoustic feedback amd what else..

[-ok-]

Mixed feelings. I can see the value in some games / high speed situations... But for most work apps etc, it speeds me up to be able to click at the speed of thought, and be able to "undo" the click by dragging off the button before letting go...
That way, each click is less deliberate / more optimistic, and the few extra millisecond before I let go of the button are enough time to hit the brakes on the action if it's wrong.
I feel like with "act on press", I'd need to be way more deliberate with every interaction, and it's stressing me out just thinking about it.

Although to be honest I'm sure this is something one would adjust to quite quickly, and is probably less significant than any of us here is making it out to be.

[M_]

The best option: make it a setting. If some people like act-on-relase, they can use that, anf if they like act-on-press they can use that. A lot of problems can be solved by just introducing a setting. (Yes I know that most people don't bother going into settings, so choosing a right default is good too).

[SahilP]

Guys he's mostly talking about VR, not websites. For VR I think it makes ton of sense.

[simonwxyz]

Navigation on mouse down sucks when you just want to grab a link and drop it into a new tab or a separate window as the url. The same goes for buttons that navigate you with js without being anchors so you also can't middle click / ctrl click them into a new tab that way.

[seebradrun]

This is really good content. I also follow John, didn't see that tweet, and now also got your commentary on it. Thanks for upload!

[jordanbrennan]

Was just thinking about this pattern because of several unintentional interactions with the like button on LinkedIn. Was able to cancel with the decades-old OS-level UX of moving away. Act on mouse up is a very good default. For the love of good UX do NOT just started making ALL interactions act on press because you treat the internet as gospel truth.

[syncrossus]

I think I disagree for a good chunk of cases because of what Theo mentions at 3:09 : having events happen on mouse-up/release events is a layer of protection against misclicks. I'm more tolerant to unresponsiveness than I am to "hyperresponsiveness" if that makes sense. I think you can get 90% of the feeling of responsiveness by creating a visual acknowledgement of the press event, e.g. changing the color of the button. You can even change the color on hover (if using a mouse), which feels very responsive and telegraphs the consequences of a click.