folder_contents broken when using Root Navigation behaviour on folder clone

[ginetsl]

I am having a problem where folder_contents breaks when in a folder inside a subsite. The contents of the folder don't show up, and a 404 error trying to load @@qsOptions is logged in the browser console. It sounds a bit like issue #103, but not exactly. I am also using virtual hosting and nginx.

These are the steps to reproduce:

1. Create new plone site (mysite)

2. Create a new "Subsite" dexterity content type as a clone of "Folder". Add the "Navigation root" behaviour to it.

3. Add a new "Subsite" in the root of mysite (mysubsite) and publish it.

4. Create a page inside mysubsite, publish it and make it the default view for mysubsite

5. Create a folder called "Folder 1" inside mysubsite (folder1) and folders, pages or any other content type objects inside folder1. Publish them all.

6. Create virtual host configuration on separate webserver (in my case, a nginx reverse proxy) for mysubsite:

server {
  listen 80;
  server_name
    mysubsite.something.com
    mysubsite.somethingelse.something.com
    ;

  return 301 https://$server_name$request_uri;
}

server {
  listen 443 ssl http2;
  server_name
    mysubsite.something.com
    mysubsite.somethingelse.something.com
    ;

  if ($host != $server_name) {
        return 301 https://$server_name$request_uri;
  }

  include /etc/nginx/conf.d/local/0-proxy-headers;

  location / {
    proxy_pass http://plone.something.com:53080/VirtualHostBase/https/mysubsite.something.com:443/mysite/mysubsite/VirtualHostRoot/;
  }
}

0-proxy-headers contains this:

proxy_request_buffering off;
proxy_buffering off;

add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;" always;

proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;

7. Log into mysubsite.something.com
8. Click on "Folder 1" in the top menu
9. Click "Contents" in the admin panel

• No content is shown
• Navigator content shows "Page 1 of NaN shown"
• The broswer console shows a 404 error trying to load "https://mysubsite.something.com/folder1/@@qsOptions"

10. The problem doesn't appear when logging into mysubsite.something.com, then clicking on "Contents" at the root, and then clicking on "Folder 1"

Info about my setup:
Plone 5.1.2.1 (5112) (Although it breaks in 5.0.6 as well)
CMF 2.2.12
Zope 2.13.27
Python 2.7.14 (default, Aug 17 2018, 09:45:23) [GCC 4.4.7 20120313 (Red Hat 4.4.7-18)]
PIL 5.1.0 (Pillow)

[flipmcf]

I can also reproduce this, but without virtual hosting. 5.0.5
I also use a custom content type known as a 'subsite' which is just a folder with some special fields.

1: Create a 'subsite' (or a top level folder) Let's call it 'root' (so /root is now the real site top level)
2: Create folders under this. 'nav1' 'nav2' 'nav3' These shall be the navigation folders.
3: Site-Setup->Navigation set 'Root' to '/root'

Navigation works as expected! folders under /root are now my navigation.

However:

1. content listing is broken. No items show up.
2. createObject() is broken. New items appear under '/' not under '/root'
3. Selecting an 'item as a default view' has some very strange effects. So goodbye to collective.cover

set navigation back to '/' and folder listings come back, but navigation is broken. (you only see 'root')

This might be reproducible without a special 'subsite' content type, but that's unknown. I just find it interesting to see another person using subsites other than myself.

I am accessing my zope instance directly:
http://localhost:8080/Plone/root and seeing the behavior.

I can add read access to our buildout with a very small trust relationship - but can't make it public b/c people wearing suits and ties said I can't.

[flipmcf]

One can re-create this issue without special content types.

create Plonesite 'plone' as usual ( localhost:8080/plone/ )
create a folder 'folder' under plone ( localhost:8080/plone/folder )

I see 'folder' appear in my navigation

Create another folder called 'subfolder' under that ( localhost:8080/plone/folder/subfolder )

Now, go to folder view of the subsite: localhost:8080/plone/folder/folder_contents )

I see 'subfolder' listed in the folder_contents

---

Go to site-setup->Navigation
Go to 'root path' and set it to '/folder'

Navigation changes as expected. Navigation now has 'subfolder' and "home" links to localhost:8080/plone/folder/

go to localhost:8080/plone/folder/folder_contents

Empty. This is the bug. It should list 'subfolder'

Set navigation root back to '/' and folder contents goes back to correct behavior.

Adding objects is also broken, but I'm going to focus on 'folder_contents' view and see what I shake loose.

[flipmcf]

The workaround I'm using is to avoid this setting altogether (leave configuration registry setting 'plone.root' alone ) and tag the navigation root object with the 'INavigationRoot' interface marker. The behavior seems to work well, the navigation displays correctly, listing contents does not break, and adding content goes where you expect it.

[ginetsl]

Unfortunatelly, that workaround doesn't apply to my case. I have never touched the "root path" setting in site setup -> navigation. I am seeing the problem by simply adding the "Navigation root" behaviour ('INavigationRoot' interface marker) to the subfolder/subsite.

Everything shows up as expected when going to http://localhost:8080/plone/subsite/folder_contents

but not when visiting http://subsite_url/folder_contents , having subsite_url defined as a virtualhost.

Were you able to reproduce this case?

[flipmcf]

I cannot reproduce using INavigationRoot interface maker. I am also not using VirtualHostMonster, and don't plan on using it, so I might not be able to reproduce.

I have not setup virtual hosting yet, so my problem is exactly the opposite of yours. It's possible you could use the configuration registry setting that I found troublesome.

[flipmcf]

I now believe we are describing separate issues, so I'll open my own.

[Rudd-O]

I can reproduce the issue with both Plone 6 and Plone 5. It's been like that forever, since I use VirtualHostMonster and only subportions of the site are visible on each domain. If I access my site without the VHM:

https://editor.site.com/Plone/es/archives/page/folder_contents

Then the folder listing works fine because the @@qsOptions URL that the browser loads is https://editor.site.com/Plone/@@qsOptions, and of course that view is accessible just at the site root.

If, however, I access the site using the VHM:

https://es.site.com/archives/page/folder_contents
(translated to https://site.com/VirtualHostBase/https/site.com/Plone/es/VirtualHostRoot/archives/page/folder_contents)

Then the folder listing does not work at all, and that's because the @@qsOptions URL that the browser wants to load then becomes https://site.com/archives/page/@@qsOptions, which is not a view accessible via pages other than the site root.

I believe the bug may be in site.absolute_url() when used with anything that alters the site root (such as the plugin OP is using, or the VHM stanza VirtualHostRoot, and this has been broken for close to half a decade now. From file plone.app.content.browser.contents.__init__.py, line 248 (as of Plone beta):

    def get_options(self):
        site = utils.get_top_site_from_url(self.context, self.request)
        base_url = site.absolute_url()  # <<--- HERE
        base_vocabulary = "%s/@@getVocabulary?name=" % base_url
        site_path = site.getPhysicalPath()
        context_path = self.context.getPhysicalPath()
        columns = self.get_columns()
        options = {
            "vocabularyUrl": "%splone.app.vocabularies.Catalog" % (base_vocabulary),
            "urlStructure": {"base": base_url, "appended": "/folder_contents"},
            "moveUrl": "%s{path}/fc-itemOrder" % base_url,
            "indexOptionsUrl": "%s/@@qsOptions" % base_url,  # <<--- HERE
            "contextInfoUrl": "%s{path}/@@fc-contextInfo" % base_url,
            "setDefaultPageUrl": "%s{path}/@@fc-setDefaultPage" % base_url,
            "defaultPageTypes": self.default_page_types(),
            "searchParam": "Title",
            "availableColumns": columns,
            "attributes": [
                "Title",
                "path",
                "getURL",
                "getIcon",
                "getMimeIcon",
                "portal_type",
            ]

[Rudd-O]

I can confirm — via some elite printf() debugging — that absolute_url() simply returns the wrong thing, at least when VHM is in use for a subfolder. From the example above, without VHM absolute_url() returns https://site.com when accesing https://site.com/es/archive/page, and with VHM enabled it returns https://site.com/archive. Which is wrong!

This is because the method above site = utils.get_top_site_from_url(self.context, self.request) is returning the es/archives folder as the top site, instead of returning es as it should be.

[Rudd-O]

I don't seem to be able to edit my Github comments right now (it's slow), so I will clarify that only paths handled by plone.app.multilingual suffer from the problem I noted above.

[Rudd-O]

I've instrumented plone.base.utils.py:get_top_site_from_url as follows, with this output:

def get_top_site_from_url(context, request):
    """Find the top-most site, which is still in the url path.

    If the current context is within a subsite within a PloneSiteRoot and no
    virtual hosting is in place, the PloneSiteRoot is returned.
    When at the same context but in a virtual hosting environment with the
    virtual host root pointing to the subsite, it returns the subsite instead
    the PloneSiteRoot.

    For this given content structure:

    /Plone/Subsite

    It should return the following in these cases:

    - No virtual hosting, URL path: /Plone, Returns: Plone Site
    - No virtual hosting, URL path: /Plone/Subsite, Returns: Plone
    - Virtual hosting roots to Subsite, URL path: /, Returns: Subsite
    """
    logger.warn("---ACCESSING %s", context.absolute_url())
    site = getSite()
    logger.warn("---SITE: %s", site)
    try:
        url_path = urlparse(context.absolute_url()).path.split("/")
        logger.warn("---URL_PATH: %s", url_path)
        for idx in range(len(url_path)):
            _path = "/".join(url_path[: idx + 1]) or "/"
            logger.warn("---_PATH: %s", _path)
            site_path = "/".join(request.physicalPathFromURL(_path)) or "/"
            logger.warn("---_SITE_PATH: %s", site_path)
            _site = context.restrictedTraverse(site_path)
            logger.warn("---_SITE: %s", _site)
            if ISite.providedBy(_site):
                logger.warn("---BREAK")
                break
        if _site:
            site = _site

Here is the output:

2022-09-07 12:59:27,646 WARNING [Plone:272][waitress-1] ---ACCESSING http://site.com/archives
2022-09-07 12:59:27,646 WARNING [Plone:274][waitress-1] ---SITE: <PloneSite at Plone>
2022-09-07 12:59:27,646 WARNING [Plone:277][waitress-1] ---URL_PATH: ['', 'archives']
2022-09-07 12:59:27,646 WARNING [Plone:280][waitress-1] ---_PATH: /
2022-09-07 12:59:27,647 WARNING [Plone:282][waitress-1] ---_SITE_PATH: /site.com/es
2022-09-07 12:59:27,647 WARNING [Plone:284][waitress-1] ---_SITE: <Container at es>
2022-09-07 12:59:27,648 WARNING [Plone:280][waitress-1] ---_PATH: /archives
2022-09-07 12:59:27,648 WARNING [Plone:282][waitress-1] ---_SITE_PATH: /site.com/es/archives
2022-09-07 12:59:27,649 WARNING [Plone:284][waitress-1] ---_SITE: <Folder at archives>

That is wrong. The _site variable should be /site.com/es because that is technically the site root for that folder when using Multilingual.

[Rudd-O]

Haha! I think I have a fix. Holy balls, a fix for an issue that has been open since 6 years! Wait a few minutes for PR please...

[Rudd-O]

HAHA FIXED plone/plone.base#18

[flipmcf]

So, let me get this straight.

The "canonical way" (bigger than 'best practice') to get a site root is to traverse upwards until you find an object that implements ISite.

You are proposing that we add a utility to do just this. And use that utility.

I tend to agree with this, I've seen it documented, somewhere, and I was taught this way back in Plone 2 also.

I have custom code that uses this method also.

Questions:

@Rudd-O How did you arrive at ISite as the interface marker? I think there are more, like ISiteRoot and INavigationRoute that are also good candidates.

Docs say use ISiteRoot https://docs.plone.org/develop/plone/serving/traversing.html#traversing-back-to-the-site-root

Does this play well with zope.component.hooks.getSite? Should it? Does it even matter?

I concur, you are on the right track here. Just throwing out some ideas.

[Rudd-O]

If the comment is addressed to me, all I have to say is that I am preserving the behavior of the code, just including the topmost accessible object (rather than the bottommost only) in the list of objects to be considered as "site roots" to return to the caller.

[flipmcf]

I'm going to move this discussion to the PR where it belongs. Sorry. I was doing review questions on the issue, not the PR.