Do any of these organization methods make page lists load faster?
Permalink
I'm trying to understand how page lists load. I know someone out there will know the answer to this.
So how do page lists scan the full database? Or are there ways to minimize the area it scans? Like when I make a page list, if I select a specific page type along with my topic type, will that reduce the amount of database it has to scan through per query?
Alternatively, if I organize my sitemap really well, and then I tell the page list to look for a topic but only search for pages underneath of X page, does that do anything for the speed of each database query to build the page list?
Are there ways to clean up or organize the database to make the queries more efficient?
Does anybody knows how it works? Does a page list just search through literally all the pages on a website in the database for each query? Or are there ways to narrow down the search so that big batches of queries go more quickly?
So how do page lists scan the full database? Or are there ways to minimize the area it scans? Like when I make a page list, if I select a specific page type along with my topic type, will that reduce the amount of database it has to scan through per query?
Alternatively, if I organize my sitemap really well, and then I tell the page list to look for a topic but only search for pages underneath of X page, does that do anything for the speed of each database query to build the page list?
Are there ways to clean up or organize the database to make the queries more efficient?
Does anybody knows how it works? Does a page list just search through literally all the pages on a website in the database for each query? Or are there ways to narrow down the search so that big batches of queries go more quickly?
So now I'm making my page lists only for a certain page type and only pages beneath the page that has the pages I'm looking for. I have no idea if it actually makes them load faster though. For all I know it actually loads slower now.
Page lists are generated through a specialised query builder that, when executed, generates and executes a single (growingly convoluted) SQL statement.
There is some rigid sequencing of filters in the generated SQL, but there is also scope for applying the more efficient or greater reducing filters first, then the less efficient filters later in the sequence where they have less data to wade through. MySQL also optimises SQL filters internally on a similar principle, but maybe not as much as we would hope.
Up to this point each page of data on a page list should be one SQL query for the list and then repeated queries for each page object in the list, then further repeated queries for each page's attributes.
That is complicated by permissions. Permissions are not evaluated within the SQL list, but applied in php by checking if a user is permitted to view each page object in turn.
Autonav works in a similar way. Except it builds a tree of pages, so makes one query for the top level, then one for each page in the first list, and so on downwards. Hence the overhead of autonav grows exponentially with the depth the navigation is configured to. That is a bit of a naive description. Its not quite like that. But if you bear that in mind it helps understand the massive cost of an unconstrained autonav or of generating an autonav to a depth greater than the menu actually needs.
In general, for a large site where speed is becoming an issue:
- Keep control of the depth of the tree autonav generates.
- Don't use attributes in navigation or page lists unless you really need them.
- Don't use permissions where you don't need to.
- If coding a page list, take care over the order of filters.
- Enable the cache
- For a stable top-level nav, consider nested manual nav instead of autonav.
What you are now doing is likely faster. Use the Speed Analyzer addon to check.
There is some rigid sequencing of filters in the generated SQL, but there is also scope for applying the more efficient or greater reducing filters first, then the less efficient filters later in the sequence where they have less data to wade through. MySQL also optimises SQL filters internally on a similar principle, but maybe not as much as we would hope.
Up to this point each page of data on a page list should be one SQL query for the list and then repeated queries for each page object in the list, then further repeated queries for each page's attributes.
That is complicated by permissions. Permissions are not evaluated within the SQL list, but applied in php by checking if a user is permitted to view each page object in turn.
Autonav works in a similar way. Except it builds a tree of pages, so makes one query for the top level, then one for each page in the first list, and so on downwards. Hence the overhead of autonav grows exponentially with the depth the navigation is configured to. That is a bit of a naive description. Its not quite like that. But if you bear that in mind it helps understand the massive cost of an unconstrained autonav or of generating an autonav to a depth greater than the menu actually needs.
In general, for a large site where speed is becoming an issue:
- Keep control of the depth of the tree autonav generates.
- Don't use attributes in navigation or page lists unless you really need them.
- Don't use permissions where you don't need to.
- If coding a page list, take care over the order of filters.
- Enable the cache
- For a stable top-level nav, consider nested manual nav instead of autonav.
What you are now doing is likely faster. Use the Speed Analyzer addon to check.
I'm a little confused about attributes as it relates to page lists. So in order for my sites to work, I need to be able to make individual pages, tag them for every appropriate category using topics, and then make page lists that show all pages for each topic.
Are you saying that if I check a bunch of different topic categories for a page that it then takes longer to query it for a page list even though the page list only picks pages based on one topic? Or are you referring to attributes as in page name, meta title, etc.?
My main concern here is for the future of my site. In the future, when building website sections, if I know everything I can do to minimize page list load times then I can start from the very beginning organizing in such a way as to minimize the burden on the database queries.
If someone ever makes an add-on that creates a second page list type that only queries the database for the first page of pages in a pagenation list, I would buy that. Then if users click page 2, it loads the page again but only queries the database for the first two pages of page list pagenation, and so on. If the user tries to go to page 50 then it will take 10 seconds to load a page, but at least the first page will load quickly and if they just go to pages 2, 3, or 4 then the page list load time is negligible. I would straight up by an add on that does that.
Are you saying that if I check a bunch of different topic categories for a page that it then takes longer to query it for a page list even though the page list only picks pages based on one topic? Or are you referring to attributes as in page name, meta title, etc.?
My main concern here is for the future of my site. In the future, when building website sections, if I know everything I can do to minimize page list load times then I can start from the very beginning organizing in such a way as to minimize the burden on the database queries.
If someone ever makes an add-on that creates a second page list type that only queries the database for the first page of pages in a pagenation list, I would buy that. Then if users click page 2, it loads the page again but only queries the database for the first two pages of page list pagenation, and so on. If the user tries to go to page 50 then it will take 10 seconds to load a page, but at least the first page will load quickly and if they just go to pages 2, 3, or 4 then the page list load time is negligible. I would straight up by an add on that does that.
The complexity of the query generated increases with the number of different attributes you filter on or number of different values of a single attribute. But that isn't a major cost. It makes a single query a bit more complicated. But doesn't escalate the number of queries.
When you display the items on the list, if you show just name+description+link, that is no extra cost. If you also display N different attributes, each attribute displayed with the list generates a few SQL queries. For most page lists showing 10 to 20 pages, that isn't enough to be problem.
For faster page lists, what you have already done - single page type, immediately beneath a a parent page etc, are all good measures for speed. The other thing you can do is ignore permissions. That eliminates extra queries for each page. But to do that safely you have to make sure your pages are organised so that pages beneath a page list share the same view permission, so you don't end up exposing information you don't want to.
When you display the items on the list, if you show just name+description+link, that is no extra cost. If you also display N different attributes, each attribute displayed with the list generates a few SQL queries. For most page lists showing 10 to 20 pages, that isn't enough to be problem.
For faster page lists, what you have already done - single page type, immediately beneath a a parent page etc, are all good measures for speed. The other thing you can do is ignore permissions. That eliminates extra queries for each page. But to do that safely you have to make sure your pages are organised so that pages beneath a page list share the same view permission, so you don't end up exposing information you don't want to.
Okay so like adding thumbnails and page dates to the items in a page list make it take longer to load. Am I understanding that correctly? Also I did not know about the ignoring page permissions, I will definitely use that. I'm not sure how it works exactly so if you don't mind me asking, what kind of information can it expose and how?
Right now all the pages checked for different topics have nothing on the page itself that isn't already public, but I'm not sure what ignore page permissions will do. So if you, or someone could explain more about that to me I would greatly appreciate it.
Also if anyone can think of literally anything else on how to speed up page lists I would greatly appreciate it. I'm thinking though that everything stated above is about everything that is available to speed it up.
Right now all the pages checked for different topics have nothing on the page itself that isn't already public, but I'm not sure what ignore page permissions will do. So if you, or someone could explain more about that to me I would greatly appreciate it.
Also if anyone can think of literally anything else on how to speed up page lists I would greatly appreciate it. I'm thinking though that everything stated above is about everything that is available to speed it up.
'Ignore' will list pages even if they are not supposed to be able to see them. The title/description may expose information that shouldn't be public.
If they then click the link to such a page, they will usually get redirected to the login page.
If they then click the link to such a page, they will usually get redirected to the login page.
Okay I think I understand. In that case I shouldn't run into any problems, I don't think I have a single page that has anything that's meant to be private or hidden. Every page on my website is meant to be public and read. Thanks for the clarification!
