How to handle threads that shouldn't be killed

[michaelpj]

We have been (ab-)using ki in the following way:

• We have a medium length job that runs
• During the execution of the job, we have some tasks that need to be completed but not promptly (e.g. updating the database)
• To handle this, we pass around a Scope which lives for the lifetime of the job and into which we fork background tasks

This isn't very tidy:

• It's unclear what the lifetime of the scope is
• You had better call awaitAll on the scope before you terminate, otherwise you might kill things that need to finish

What we're doing seems not ideal, but I'm not sure how to do it better.

Ideas:

• Track it on the thread level: some threads are okay to kill, some aren't. A Scope could track which ones can be killed and which ones have to be awaited.
• Track it on the scope level: some scopes kill their children, some await them. We could have different scoped functions that create Scopes that behave differently. Maybe these should be different types of Scope.
• This is just not an appropriate use of structured concurrency, and you should have something more like an explicit job queue.

[mitchellwrosen]

For starters, not that it's particularly relevant here, I do tend to view passing around a Scope as a bit of an anti-pattern.

Ki.scoped \scope ->
  XXXXXXXXXXXXXXXX
  X scope XXXXXXXX
  XXXXXXXXXXXXXXXX
  XXXXXXX scope XX
  XXXXXXXXXXXXXXXX

The scope object does just correspond to the lexical scope drawn in Xs, so if you can visually scan and see that scope is only passed into "trusted" functions like Ki.fork (and whatever other variants you've created locally), then there's no mystery about the structure of the concurrency / thread tree.

Since you have background jobs that are meant to outlive the "scope A" in which they were created, there indeed could be a larger "scope B" that they are not meant to outlive at all, i.e. the Scope that you're passing around. You could also just fire-and-forget these background jobs if they need not be managed in any particular way, but it sounds like you do want them scoped to the job itself (i.e. if some part of the job fails with an exception, all of those background workers should be killed, too).

You had better call awaitAll on the scope before you terminate, otherwise you might kill things that need to finish

Indeed, this is just how the API works (as you know ;)). There's no implicit blocking. It's the caller's responsibility to ensure children threads are allowed to complete before the parent "falls through" the bottom of the scoped block.

But I can provide some motivation for the decision: if there was an implicit block at the end of all scoped blocks, we couldn't implement race-like functionality without an explicit killThread function, too:

race :: IO a -> IO a -> IO a
race action1 action2 =
  Ki.scoped \scope -> do
    resultVar <- newEmptyMVar
    _ <- Ki.fork scope (action1 >>= tryPutMVar resultVar)
    _ <- Ki.fork scope (action2 >>= tryPutMVar resultVar)
    takeMVar resultVar

In this example, we only wait for one thread to tryPutMVar before falling through the bottom of scoped to kill the other thread. And I've argued against a killThread function in the past: #12

Track it on the thread level: some threads are okay to kill, some aren't. A Scope could track which ones can be killed and which ones have to be awaited.

This is an interesting suggestion. My gut reaction is I don't think we should do this, but I'm willing to hear out more thoughts about it.

It seems to me that this is getting at the question of responsibility. For example, ought the scope/parent have complete control over when the children it spawns are terminated, as it does now? Or ought some part of the code with a handle to some scope be allowed to spawn a child into it, declaring it immune to being killed prematurely?

But to me, as expounded upon above, these two "parties" are one and the same. If you have a handle to a scope, you are the parent thread of that scope. So, it's just as easy to remember to call awaitAll as it would be to call forkThreadThatCantBeKilled. Passing a scope around certainly blurs the boundaries here, though.

Track it on the scope level: some scopes kill their children, some await them. We could have different scoped functions that create Scopes that behave differently. Maybe these should be different types of Scope.

I like the ideas you are having. This is another I hadn't heard about before. But my inclination is the same: it seems just as easy to remember to call awaitAll as it would be to call the scopedThatAwaits variant. So, that's a point against expanding the API in that way.

What do you think?

[michaelpj]

Thanks for the detailed reply! (I missed the notification...)

It seems to me that this is getting at the question of responsibility.

Yes, I think you're exactly right.

But to me, as expounded upon above, these two "parties" are one and the same. ... Passing a scope around certainly blurs the boundaries here, though.

I think the passing the scope is the key thing that makes this tricky. Once the creation of the scope and the use of the scope are not very close, it becomes much easier to mess things up. That's a classic place for us to use the type system to help us! Hence the idea of having a different kind of scope: now you just can't get it wrong.

I do agree that the current API is better as a base. Indeed, I can implement the one I want on top very simply:

newtype BlockingScope = BlockingScope Scope

blockingScoped :: (BlockingScope -> IO a) -> IO a
blockingScoped act = scoped $ \scope -> do
  act (BlockingScope scope)
  atomically $ awaitAll scope

forkBlocking :: BlockingScope -> IO a -> IO (Thread a)
forkBlocking = coerce fork

-- etc.

I just wonder if this is something we want to provide to other people.

---

I do think that a pretty reasonable response is just: is this really structured concurrency? Is it a good idea in the way that we think structured concurrency is a good idea? Do I really want to be able to spawn an arbitrary amount of jobs into a big scope that could contain a potentially arbitrary amount of subcomputation?

Or perhaps I want a job queue: an execution service that I provide over a wide scope with bounded concurrency and perhaps some more visibility. Perhaps the lesson is that structured concurrency and job queues are a good pairing?