Universal listings (search and filtering)

[benenright]

Summary

Exploring design solutions for making sites / sections with 1000s of pages and little to no tree structure easier to navigate in the CMS.

The aim is to enhance the searching / filtering capabilities of the main page listing view (and to eventually roll out the same patterns and functionality to all listings within the CMS).

Background

• See RFC 82: Treeless page listings rfcs#82
• See Universal listings roadmap#33

Considerations

• An accessible and scalable mechanism for filtering
• How to show when filters/search terms are applied
• Approach to fetching results and impact on performance
• Adding additional information such as 'X results found' based on filtering/search criteria
• Can users 'save' a filtered view to quickly find it again in the interface
• UI patterns that can be consistent across all listings (not just Pages)
• Impact on the listing item information (eg. show no. of direct descendants, show location in the tree in search results)
• Impact on the listing item styling/actions (eg. reduce vertical height, improve a11y of the actions (Edit, Add child page etc), add a folder icon to help identify parent pages)
• Impact on the role and functionality of the fly-out page explorer

Design approach

Default page listing

Search results

Item hover state

Filter dialogs

Filter pills

Save view

Open questions

• What is the best approach for the visibility of the listing item actions? It feels heavy in the UI to have an omni-present 'Edit' button, and since this action can also be achieved by clicking on the item title, there is an argument for the 'Edit' button to appear on hover
• Whether we introduce the 'Add child page' shortcut action to each item, where child pages are allowed to be added
• Impact on the role and functionality of the fly-out page explorer - and whether we consider this as part of this work
• Impact on Chooser modals

Welcome all feedback and thoughts people would like to contribute.

[pySilver]

That looks really really good!

[tbrlpld]

Yes please!

[enzedonline]

Superb :)

Request for multi-lingual sites - please add 'results limited to current listed locale' in considerations. Thanks!

[benenright]

Good idea, will see where this feels natural. It does open the question around whether we want to display the current locale upfront on the listing view (it's currently now hidden away in the status panel). But if it were more visible, it may negate the need to display 'results limited to current listed locale' since that may be more apparent.

[ababic]

This is looking superb @benenright! Especially loving the filters 😍!

A couple of thoughts that occurred to me are:

1. What is the default action when clicking on the page title? Will that continue to always go to the explorer view for that page?
2. Do we have any research on the actions most performed from the explorer listing? I feel like the "edit" action should certainly be easier to see and access. Would icon-only buttons work for the key actions?
3. Have you played with repurposing the right-most column for actions? I feel like a series of small buttons might work nicely there (instead of next to the page title) as it doesn't disrupt the visual flow of the row as much. The "explore this section" action feels a little less important outside of the tree view, but is using up prime real estate.

[benenright]

Hi Andy

1. Yea, the default action when clicking a page would stay as 'Edit'
2. Agree 'Edit' should be easy to access, but since the default action when clicking a page title is 'Edit', we were considering still keeping the additional Edit button only appear on hover (to reduce clutter). Icon-only buttons have been considered and maybe we could revisit... although there was something quite strange about the edit 'pen' icon on it's own without a label :).
3. Yes we considered moving the actions further to the right of the table (I think we can try this again based on your thoughts about row disruption). A reason we kept them nearer the page title was that it felt like in general it was an easier place on the screen to get to (less distance to move for mouse users) when scrolling down the table (where in general we expect people to be hovering more on the left side while reading the page titles). We still need a place for the 'explore this section' button, and the paradigm of the right arrow (on the far right side) to dive down a level still feels natural (akin to mobile menus etc.)

[cnk]

I like this! I have a site that doesn't have a particularly main content tree but we do have years of imported news and events pages so we have built custom versions of what you are proposing here (though not as nice because I don't have your design skilz). From that perspective, it would be great if the filtering component were configurable since sites like ours will have immediate uses for custom filters - and because all snippet listings that eventually use this will want to set up their own filtering options.

[ababic]

Just thought of another challenge that isn't explicitly addressed in the designs so far...

What does the switch between "only show children of the current explorable page" and "show matches from anywhere" look like? And how does that work for multisite / translated projects?

Maybe we need two completely separate "Page explorer" and "All pages" menu items, and the latter would go to somewhere with a completely different header and site / language filters?

[benenright]

Yea, I think we could still consider how the search/filter can be either "show any descendants below this point in tree" or "show matches from anywhere". Where in the latter we will need to find a way to show locale for each result (and also consider the need for a 'locale/language' filter).

However, so far our mvp version of this is not to create a seperate view for 'All pages' but rather to enhance the existing tree view that people are familiar with. In our designs we are currently limited to showing results 'below' your point in tree - which in most cases I believe negates the need to show different locales because (unless you are at Root) in most cases you are already within a specific locale.

@gasman @thibaudcolas may want to chime in on this?

[thibaudcolas]

Yup. In a tree structure world, the "treeless" experience of "show matches from anywhere" will essentially amount to applying filters from the root level. Which is accessible from the top of the page explorer. It’d be nice if this was more easily accessible perhaps.

[ababic]

the search/filter can be either "show any descendants below this point in tree" or "show matches from anywhere".

Okay. In the explorer view, we're currently showing only 'direct children' of the current point in the tree. So, I think automatically playing with that rule when searching or filtering could cause a bit of confusion. I would suspect we need to stick with showing direct children in both cases by default, but have some opt-in mechanism for showing more 'levels' (e.g. a toggle switch or checkbox).

the "treeless" experience of "show matches from anywhere" will essentially amount to applying filters from the root level. Which is accessible from the top of the page explorer. It’d be nice if this was more easily accessible perhaps.

Okay, one thing to bare in mind there is: There's not really any reason for most editors to care about the root page at all, as content is never at that level. So, I worry it wouldn't be an obvious place for them to navigate to, even if that option were more visible. A design challenge, for sure.

If similar options were available on the general/global 'Search' view too, the "View results from anywhere" option could just link out to that, potentially?

[thibaudcolas]

Just wanted to capture another use case we discussed with colleagues today (cc @helenb) – for QA purposes, it’s really common to need to find a page of a given page type. And potentially doing that for multiple page types.

I believe the current designs as proposed would cater for that scenario well – go to the root level, filter by page type, pick the first one. But it’s such a common scenario I thought it’d be worth mentioning anyway.

[ababic]

I added this fly-out menu to Tate using ModelAdmin, which they love. It would be awesome if Wagtail could roll it's own version of this:

[ababic]

Just spitballing here: If we DID happen to have a separate "View all" option for pages, the "View by type" options could simply link to a pre-filtered version of that.

[easherma-truth]

Its also common (IMO) to be able to search by page slug and not just title or page type 🤔

[pySilver]

And filter by Site

[SebCorbin]

Another use case from us, we have a site where the page tree is not very relevant: some editorial pages like "Legal mentions" and another page-based model (a bit like landing pages) for which we want page features like workflow, previewing, and so on, along with our business configuration.

In our case the Page explorer is too far-fetched for this kind of site structure so we would love to get rid of it entirely. So far we are stuck with it, but by using ModelAdmin we have a kind of basic list so this feature would help a lot.
A thing to note, the menu breadcrumb (which item is active) would need a bit of thinking, do we want to retain the path (list) from which we come when editing a Page ?

[jhonatan-lopes]

Hi folks, I've just submitted a PR for a new built-in report focusing on page types, which answers the following questions:

• Which page types do we have on our CMS?
• How many pages of that page type do we have?
• When was a page of that type last edited? By whom? Which page was that?

It can also show a list of all the pages of a particular type.

[benenright]

Hi

During development of the new listings, some UX issue have come to light so we have adjusted some of the UI.

Notes

• Having the search and filtering bar underneath creates a very tall header and since this header is sticky (to enable consistent access to and behaviour of the side panels), we have created a version where we move the search and filter button up alongside the panel icons, to achieve a shorter sticky header (details below)
• We also reduced the size of the page title (which is also the 'current page' in the collapsed breadcrumb)
• The filter 'pills' will still appear in a row underneath, making the header taller

General listing header would share similar patterns:

[gasman]

Just a note that a request has come up to add "is a parent page or not" to the set of filtering options. (That should be a simple one to implement - filter on the 'numchild' field being greater than 0.)

[thibaudcolas]

Discussed this week with Ben – it’d be nice to have a way to view the "treeless" version of a specific view without having to trigger a search or applying a filter. For example, if I’d like to see all of the site’s pages.

[Scotchester]

Assuming it's all controlled by query string params in the URL, perhaps a hook could be used to add permanent menu links pointing to specific views? These could have specific search queries, or filters applied, or neither.

For bonus points, a way for users to add their own saved views to the sidebar for easy access would be pretty cool.

[thibaudcolas]

Ah yep! Since the sidebar is for navigation, it’d probably be a very natural place for those "saved views" / bookmarks.

[enzedonline]

Not sure if this is the right place for this -

It would be a great boost in UX if the current view settings were retained after performing an action on the listing page. This applies to all the listing views I've seen so far (i.e. pages, snippets, users, forms ...).

Currently, after performing an action, the returned page is the first page of the default view. This can be quite frustrating when you've got a filtered, ordered or paginated view you're working with.

The view could be all pages created after X, user list sorted by last login date or page x of submitted forms for instance. Ideally, you'd want all those attributes of the view retained after performing the menu action (delete, unpublish etc.).

[thibaudcolas]

Interesting, thank you. Do you have examples of other tools that work like this? This would make perfect sense to me if those operations were AJAX-based, but when moving from page to page I can imagine it would be confusing if you returned to a filtered view.

[cnk]

Actually the PR my team submitted to add next_url to most page actions was so we could do exactly what @enzedonline suggested. We have a snippet index view where our editors search for items to review and edit. We had multiple requests for "can't this go back to my search results?" so we customized that listing to return to the filtered list.

[enzedonline]

I think people are far more used to AJAX type behaviour these days, there's not much around that still does a refresh for list actions like this. Feedback is definitely that the expected behaviour is to keep the view you're already on, not have it reset each time.

The best example I can think of are the more old-school webmail interfaces (c-panel type thing) - sort email by name/date, be on page x of items. If you were to delete an email on that for example, you wouldn't want your inbox view to return to the first page sorted by newest first, you'd expect you view to stay as you were.

[laymonage]

Thank you, it looks like we have this behaviour in some places but not all. For example, the documents index view does this:

wagtail/wagtail/documents/templates/wagtaildocs/documents/list.html

Lines 42 to 44 in b7beb36

	<td id="document_{{ doc.pk|unlocalize }}_title" class="title">
	<div class="title-wrapper"><a href="{% url 'wagtaildocs:edit' doc.id %}{% if next %}?next={{ next|urlencode }}{% endif %}">{{ doc.title }}</a></div>
	</td>

wagtail/wagtail/documents/views/documents.py

Lines 78 to 81 in b7beb36

	next_url = reverse("wagtaildocs:index")
	request_query_string = self.request.META.get("QUERY_STRING")
	if request_query_string:
	next_url += "?" + request_query_string

If we want to keep this behaviour, we'll need to make sure it's applied everywhere.

[thibaudcolas]

What do you all think of #9177?

[jsma]

Hi all,

Per the Migrating from ModelAdmin to Snippets guide:

ModelAdmin allows the registration of Page models, but the main use case is to create custom page listing views. The create and edit views are not supported by ModelAdmin, as there are page-specific operations in those views that are best handled by Wagtail’s page views. For this reason, registering a Page model as a snippet is not supported.
In a future release, Wagtail will introduce a new “treeless” listing view for pages, as outlined in RFC 082: Treeless page listings and the Universal Listings discussion. This feature will allow for custom page listing views and will be the recommended way to achieve this use case. Feedback to this upcoming feature is welcome.

I'm not clear where we're at based on this thread. I can see that it's possible to filter by page type in Universal Listings but it's not clear to me how one would replicate the ModelAdmin functionality of creating a custom menu item e.g., Marketing -> Blog Posts so that staff can jump right to the page types that matter to them. Another use case we could handle with ModelAdmin was to add custom filters to this index view. For example, the Blog Posts ModelAdmin might have Blog Category filters on the right. With Universal Listings, the filters appear to be limited to fields that are common to all page types.

Is the solution to make a custom view/template and wire up the URL(s) using register_admin_menu_item? Previously it was a lot more clear to me how to add a mix of items to a custom menu. Now I'm thoroughly confused how we're supposed to wire things together if we have a custom menu item that may include models registered with SnippetViewSet and some with ModelViewSet plus custom views with register_admin_menu_item. Do we put them all in a ModelViewSetGroup or SnippetViewSetGroup or should we not mix items?

[laymonage]

I'm not clear where we're at based on this thread. I can see that it's possible to filter by page type in Universal Listings but it's not clear to me how one would replicate the ModelAdmin functionality of creating a custom menu item e.g., Marketing -> Blog Posts so that staff can jump right to the page types that matter to them. Another use case we could handle with ModelAdmin was to add custom filters to this index view. For example, the Blog Posts ModelAdmin might have Blog Category filters on the right. With Universal Listings, the filters appear to be limited to fields that are common to all page types.

This was added in #11485, which will be in 6.1. A follow-up to allow creating a page from such a list is #11734 (if we can get a PR in before feature freeze, that'll be in 6.1 as well).

Is the solution to make a custom view/template and wire up the URL(s) using register_admin_menu_item? Previously it was a lot more clear to me how to add a mix of items to a custom menu. Now I'm thoroughly confused how we're supposed to wire things together if we have a custom menu item that may include models registered with SnippetViewSet and some with ModelViewSet plus custom views with register_admin_menu_item. Do we put them all in a ModelViewSetGroup or SnippetViewSetGroup or should we not mix items?

I wouldn't recommend mixing the items, but I believe there isn't really anything that prevents you from doing so. The safest option would be to use the ViewSetGroup base class:

wagtail/wagtail/admin/viewsets/base.py

Lines 134 to 143 in 77d783b

	class ViewSetGroup(WagtailMenuRegisterableGroup):
	"""
	A container for grouping together multiple :class:`ViewSet` instances.
	Creates a menu item with a submenu for accessing the main URL for each instances.
	For more information on how to use this class, see :ref:`using_base_viewsetgroup`.
	"""
	def on_register(self):
	self.register_menu_item()

All the ModelViewSetGroup does is add a reasonable default menu label if one isn't supplied:

wagtail/wagtail/admin/viewsets/model.py

Lines 662 to 679 in 77d783b

	class ModelViewSetGroup(ViewSetGroup):
	"""
	A container for grouping together multiple
	:class:`~wagtail.admin.viewsets.model.ModelViewSet` instances.
	All attributes and methods from
	:class:`~wagtail.admin.viewsets.base.ViewSetGroup` are available.
	"""
	def get_app_label_from_subitems(self):
	for instance in self.registerables:
	if app_label := getattr(instance, "app_label", ""):
	return app_label.title()
	return ""
	@cached_property
	def menu_label(self):
	return self.get_app_label_from_subitems()

And the SnippetViewSetGroup only sets menu_item_is_registered to the items to ensure the Snippets menu item is properly hidden when all items have their own menu items:

wagtail/wagtail/snippets/views/snippets.py

Lines 1290 to 1310 in 77d783b

	class SnippetViewSetGroup(ModelViewSetGroup):
	"""
	A container for grouping together multiple
	:class:`~wagtail.snippets.views.snippets.SnippetViewSet` instances.
	All attributes and methods from
	:class:`~wagtail.admin.viewsets.model.ModelViewSetGroup` are available.
	"""
	def __init__(self):
	menu_item_is_registered = getattr(
	self, "menu_item_is_registered", bool(self.menu_hook)
	)
	# If the menu item is registered, mark all viewsets as such so that we can
	# hide the "Snippets" menu item if all snippets have their own menu items.
	for item in self.items:
	item.menu_item_is_registered = menu_item_is_registered
	# Call super() after setting menu_item_is_registered so that nested groups
	# can inherit the value.
	super().__init__()

So, as long as you have the menu_label set for the menu group, and you set menu_item_is_registered = True in your SnippetViewSets (or leave it if you don't care about hiding the Snippets menu item), you should be OK with using the base ViewSetGroup.

Or, alternatively, you might be able to use SnippetViewSetGroup. I think the code should handle non-snippet/non-model items gracefully (and if not, we should probably change that).