An Interconnection between Data Provider Limit and Filters

[vlakula]

Hi!

I have a little problem that I'm not sure how to resolve.
At the moment my app is loading too slow and I think it happens because of too many rows of data that loading at once. There ara 1500 rows. And strictly speaking I don't need all of them at once - I need only 50 or even 20. When I take a limit in 20 rows in Data Provider the loading speed starts to feel better. But! If I take so limit with a pagination all of my filters start to see just only rows from the first page but not from the whole data in 1500 rows. And this is a point of problem. I need to filter data by all rows but I don't need to show all rows to user.
Is there some resolve for this case?

Here are the screenshots of Data Provider Settings and Filters:

Thank you!