Polling interval algorithm and extra property called ldes:timeToLive

[pietercolpaert]

Currently, the expected behavior is as follows:

1. When a Cache-control: max-age=x and Age: Y has been set, poll the page again in X-Y seconds, or when it is immutable, don’t poll it ever again.
2. When there are is no Cache-control header present, poll the page according to a ldes client specific strategy, that could be influenced by a configuration flag

In the reference implementation of the LDES client, there is a flag on a pollingInterval. This is a problem as it requires specific configuration for a specific kind of LDES.

There should be an algorithm in the LDES spec that indicates when you MUST poll again, even when caching headers are not set. This thus can first elaborate on how to interpret max-age and age, but I would like to have an LDES specific property on a tree:Node to either overwrite the max-age, or set the polling interval when it’s not set.

# If set, you can ignore HTTP headers and not visit this page again
<> ldes:timeToLive "immutable" .
# If this is set, the time to live of the page is 60 seconds. It may however be possible that the page was generated 59 seconds ago. For more fine-grained control, you will need to check HTTP headers to then check the `Age` header.
<> ldes:timeToLive 60 . 

This may also be useful for hosting LDESes in systems in which the caching headers are not easy to change.

When there is no timeToLive, or max-age present, we should have a fall-back of a default value. I’m unsure what this default value should be then.

[sandervd]

This is something we need to look into more in depth.
This issue looks at it from an individual node, but I would also like to introduce something on the view level:
a minimum polling interval. This becomes important in replication scenarios, where the server applies compaction, and clients could fall too far behind, and get into a state where the only solution is starting over with replication.