Wake Up interval cannot be longer than 3600s

[N20Visuals]

I've set thePHOTOPRISM_WAKEUP_INTERVAL to 84600s so the workers would only start once a day to reduce power consumption (just in case the workers need to spin up the drives). Did this as stated in the documentation, but i just discovered that i get an error message when trying to tag new faces that the wake up interval cannot be longer than 3600s. I assume either the documentation needs to be changed or this is some kind of bug.
Easily reproducable by setting PHOTOPRISM_WAKEUP_INTERVAL to 86400s and try to name a face that has not been recognized or is new to the library.

I'm running the most recent docker version on TrueNAS SCALE on a modified HP Z440, 8-Core CPU and 64GB RAM. Container is running on SSD, Photos are stored on HDD.

[lastzero]

This works as intended. Face recognition works asynchronously. So if you don't have a background worker running more than once a day, you may run into various issues. You can find various discussions on GitHub about this, and there may also be a note in our docs, for example in the "Known Issues" section.

Please only submit bug reports for confirmed issues. Thank you very much! 🙏

[N20Visuals]

that would be amazing, no hurry though.

[xrad]

Tagging along here as well, after also having run into this snake pit. I read known issues. I understand the general reasoning and I appeciate the fear of an increase in support issues. Still I think it would be acceptable to make the regular worker an opt-in for the tinkerers, who should know what they're doing anyway. For the rest of us, the worker should just be scheduled on an as needed basis to conserve energy and wear.

[lastzero]

I think it would be acceptable to make the regular worker an opt-in for the tinkerers

That's possible! However, not all tinkerers read the code, so they may not know the PHOTOPRISM_UNSAFE flag that has been added for them? 😂

photoprism/internal/api/markers.go

Lines 25 to 29 in 5bd822e

	// Checks if background worker runs less than once per hour.
	func wakeupIntervalTooHigh(c *gin.Context) bool {
	if conf := get.Config(); conf.Unsafe() {
	return false
	} else if i := conf.WakeupInterval(); i > time.Hour {

or the rest of us, the worker should just be scheduled on an as needed basis

We've done that, and of course we will continue to improve it over time:

• Faces: Improve performance and strategy for manual tagging #3124

[xrad]

Hm.... touché on not reading the code 😆 Though PHOTOPRISM_UNSAFE just seems to allow for longer than 1h intervals which is not really what I was after. The diff you referenced though looks much more promising, but ony my install it did not seem to catch on. With a stable set of photos the system reliably woke up in intervals of 15 minutes. As far as I could interpret the logs it did not do anything productive on each wakeup and fell right back to sleep. But it ran over all folders. I'll try to dig more into the code but I am not at all fluent in reading Go.

[lastzero]

What do you mean by woke up? Yes, the worker is "running", but it should only check if something needs to be done, and otherwise skip all/most of the tasks (that might create load). If your goal is to avoid any activity and resource usage, maybe it's best to stop the photoprism service when you're not using it? Of course, the worker could also benefit from further improvements and be split into smaller workers for more fine-grained control, but we don't have unlimited resources available for this and don't want to break existing functionality by rushing it.

[xrad]

Indeed, the worker wakes up every 15 minutes just to see there's nothing to do. This check will wake up drives which would otherwise be sleeping, causing wear and energy consumption. The worker should only be scheduled for a wakeup if there is actually something to do OR someone specificially asks for a regular "preventive" wakeup because they do asynchronous modifications to the database.

Taking down the server entirely is not really a viable path with multiple independent (albeit rarely active) users.

[lastzero]

I agree with you that waking up traditional magnetic hard disks is not ideal. Based on the code I just checked for you again, it could be caused by any of these workers:

photoprism/internal/workers/workers.go

Lines 54 to 65 in a5dd491

	case <-stop:
	log.Info("shutting down workers")
	ticker.Stop()
	mutex.MetaWorker.Cancel()
	mutex.ShareWorker.Cancel()
	mutex.SyncWorker.Cancel()
	return
	case <-ticker.C:
	RunMeta(conf)
	RunShare(conf)
	RunSync(conf)
	}

As you can see, the other workers are responsible for syncing with external services (if configured, e.g. Nextcloud/WebDAV server). So they need to check if a service (which could also have been manually added to the database) exists and then e.g. check this service for new/updated files.

Logs may also be created. PhotoPrism sends these to stdout, where they are picked up by Docker. It is possible that Docker writes them to disk so that it may spin up. However, this is beyond our control and could be configured by you.

Whether the meta-worker is running is determined by this line (the effective interval is otherwise entity.IndexUpdateInterval, so 3h, to check the index for entries that need to be updated - for example, because it was updated by running "photoprism index" in a terminal or manually, e.g. by scripts or some database tool):

photoprism/internal/workers/meta.go

Lines 50 to 51 in a5dd491

	// Check time when worker was last executed.
	updateIndex := force || w.lastRun.Before(time.Now().Add(-1*entity.IndexUpdateInterval))

Finally, I wouldn't put magnetic hard disks in a server unless I can make sure they are constantly running, because that's what server hard disks should do for highest reliability and durability - after all, I have a server to protect my data and perform background tasks, so I don't need a RAID on every single desktop computer and can shut down any desktop computer whenever I want, e.g. at night. But that's just my personal opinion and if it's crucial for you to have a quiet server with magnetic hard drives that you can put to sleep (and you definitely don't want to buy SSDs which would also perform much better), I'll do my best to point you in the right direction to investigate this further :)

[xrad]

Actually I am using an SSD, but I think the wear argument applies to those as well. I'm certainly not disputing actual and potential needs to run the worker, and your examples make a lot of sense. All that I noticed on my relatively small instance were regular wakeups which didn't do anything productive. This one I captured the other day as I wasn't sure if there wasn't any incremental improvement say on the face detection being done:

root-photoprism-1  | time="2024-02-12T14:41:56Z" level=debug msg="index: running face recognition"
root-photoprism-1  | time="2024-02-12T14:41:56Z" level=debug msg="faces: found no orphan markers [551.663542ms]"
root-photoprism-1  | time="2024-02-12T14:41:56Z" level=debug msg="markers: found no invalid references [128.855269ms]"
root-photoprism-1  | time="2024-02-12T14:41:56Z" level=debug msg="markers: found no missing subjects [988.6µs]"
root-photoprism-1  | time="2024-02-12T14:42:24Z" level=debug msg="faces: found no ambiguous subjects [27.603433331s]"
root-photoprism-1  | time="2024-02-12T14:42:24Z" level=debug msg="faces: found no clusters to be merged [33.602084ms]"
root-photoprism-1  | time="2024-02-12T14:42:24Z" level=debug msg="faces: skipped clustering"
root-photoprism-1  | time="2024-02-12T14:42:24Z" level=debug msg="faces: found no new faces [25.843564ms]"
root-photoprism-1  | time="2024-02-12T14:42:24Z" level=debug msg="faces: found no unmatched markers"
root-photoprism-1  | time="2024-02-12T14:42:24Z" level=debug msg="faces: updated 0 markers, recognized 0 faces, 0 unknown [245.846029ms]"
root-photoprism-1  | time="2024-02-12T14:42:24Z" level=debug msg="index: updating metadata"
root-photoprism-1  | time="2024-02-12T14:42:24Z" level=debug msg="moments: analyzing 15944 photos and 534 videos, with threshold 15"
... crawling over a ton of moments ...
root-photoprism-1  | time="2024-02-12T14:42:28Z" level=debug msg="index: updating counts"
root-photoprism-1  | time="2024-02-12T14:42:28Z" level=debug msg="counts: updated 0 places [49.136997ms]"
root-photoprism-1  | time="2024-02-12T14:42:28Z" level=debug msg="counts: updated 0 subjects [32.948339ms]"
root-photoprism-1  | time="2024-02-12T14:42:28Z" level=debug msg="counts: updated 0 labels [122.473482ms]"
root-photoprism-1  | time="2024-02-12T14:42:28Z" level=debug msg="index: updating covers"
root-photoprism-1  | time="2024-02-12T14:42:28Z" level=debug msg="covers: updated 0 albums [8.240239ms]"
root-photoprism-1  | time="2024-02-12T14:42:28Z" level=debug msg="covers: updated 0 folders [130.571102ms]"
root-photoprism-1  | time="2024-02-12T14:42:28Z" level=debug msg="covers: updated 0 months [59.63433ms]"
root-photoprism-1  | time="2024-02-12T14:42:29Z" level=debug msg="covers: updated 0 labels [131.459878ms]"
root-photoprism-1  | time="2024-02-12T14:42:29Z" level=debug msg="covers: updated 0 subjects [17.164729ms]"

In terms of efficiency, this simply doesn't feel right to me. Bear with me. I am biased by age, and probably also by the experience of working a lot in the embedded and real-time space. But I just hate cycles go to waste. :-)

[lastzero]

Nothing wrong with that! You are welcome to contribute improvements where possible without impacting functionality. While I personally enjoy such optimizations and would love to work with you on this, many of our users are waiting for the features on our public roadmap to get released, so we need to focus on those and can't spend a lot of time on other tasks at the moment.