restore from history requests now break documented hx-request behaviour

[MichaelWest22]

TLDR:
Documentation for caching and the implied use of full/partial responses when detecting HX-Request header leads to bad back page navigations and broken behavior if you use these caching guidelines after a PR from last year.

In this now merged PR request last year:
#2013
The behavior of the restore from history on cache miss (or history disabled) was changed so that the simulated full page reload done by htmx on back navigation now represents the request as a full htmx request via the HX-Request header being set to true. This kind of makes some sense as it is htmx making the request. However this also breaks the standard convention used in many existing websites and also documented in the htmx documentation https://htmx.org/docs/#caching and the hypermedia systems book https://github.com/bigskysoftware/hypermedia-systems-book/blob/main/ch06-more-htmx-patterns.typ . Many major youtube guides for htmx also teach this now broken pattern which is not ideal.

Users sometimes return partial htmx responses when HX-Request header is true but full page responses when its missing for a full page reload will sometimes experience random bugs where excessive back navigation use will lead the user to a full page reload of just a partial response. If you disable history feature in config and navigate back when using this partial/full standard pattern it will trigger this issue every single time. Most users would have history enabled and not see this often and when it happens intermittently write the issue off as htmx being a bit unstable.

Also the documented caching guidelines in the htmx docs linked above and in the hypermedia systems chapter 6 point to using a Vary: HX-Request response header when returning cached responses and in the back navigation example above the local browser cache will return the last partial response instead of the full page response which will be swapped into the body causing the same issue as above which is also not ideal.

If we want to keep the change in place so that history loads are treated as full hx-requests as the PR requested then we probably need to update the caching documentation to point out this limitation and revise all external documentation talking about the common pattern of using hx-request header to return partial responses. You can for example implement something on the backend to detect the problem and respond correctly with a partial or not by checking both headers like this:

function is_htmx_request(context) {
  if (context.header('HX-History-Restore-Request')) { return false }
  return !!context.header('HX-Request')
}

However when testing this change this then returns full page reloads fine on back navigation and then if you have Vary: HX-Request caching enabled as documented it then caches the full response in the browser and so normal htmx navigations after this point break by placing the full page reload into the target in error which is annoying. To resolve this you have to change to using Vary: HX-Request, HX-History-Restore-Request which is more added complexity to deal with these edge cases.

The other simpler solution for end users is to just turn the following config item to true htmx.config.refreshOnHistoryMiss which will bypass the problem history restore code. Adding this as a recommendation in the documentation might help. Could even make this the new default with no real impact on users probably.

It may also be possible to just safely revert the #2013 PR as I can't see any valid reason to return this header for non htmx requests like back navigation. Anyone relying on these back navigation full reload requests to be treated as hx-requests will already have to deal with identical full page reloads when the user pushes f5 so having a few of the requests including this header at random I don't think leads to ideal outcomes. Also the small percentage of users that might want to know if the request is a a full page history reload already have the HX-History-Restore-Request request header they can use to account for this behaviour.

[yawaramin]

Would this issue by fixed by htmx not sending the HX-Request header if it also sends the HX-History-Restore-Request header?

[MichaelWest22]

Yes reverting the change to return HX-Request header done in #2013 would resolve the issue. However it would change the behavior that some like the creator of the PR think they depend on. My last paragraph above details my thoughts on this.

One other option to possibly resolve this issue is to add a new config item like htmx.config.hisotryRestoreIsHxRequest where we optionally return the problem header or not based on this config value and then we could decide if we want to leave this config with a default of true to retain current behavior or default to false and revert to the old behavior to support users returning partials based on HX-Request header. Either way it would allow additional documentation for this config value so people can discover how this might impact their application and choose the right value for them.

Also Maybe the original requester @xhaggi who requested this change for htmx-spring-boot could comment on why they needed this change and exactly what problem it solved for them so we know the impact.

[xhaggi]

Any request made by htmx should contain the HX-Request header to easily detect a request made by htmx. And that's what the documentation points out by saying “always true”, if I understand it correctly 😄. However, I understand your objection if you have an endpoint that returns a full or partial page when HX-Request is present. If the presence of the header causes such impairment, it is fine with me to remove HX-Request from the history request. But after that, you also need to check for HX-Request and HX-History-Restore-Request to know if the request was done by htmx.

[yawaramin]

OK, but the point of wanting to know whether a request is from htmx or not is that we want to know whether we should render a partial or full page. That's why the HX-Request header exists. I don't think anyone wants to know which requests are from htmx just for the sake of knowing 🙂