Skip to content

Tentacule is an uncomplicated library to deal with a pool of worker processes

License

Notifications You must be signed in to change notification settings

BinarSkugga/tentacule

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

29 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

tentacule

Tentacule is an uncomplicated library to deal with a pool of worker processes

How to install

pip install tentacule

How to use

  1. Create a pool and start it
  2. Send a new task to it
  3. Fetch the task result
pool = ProcessPool(workers=3)
pool.start()

def task(arg: int):
    return 2 + arg

task_id = pool.submit(task, 3)
result = pool.fetch(task_id)  # 5

pool.close()

You can also do this in a single step:

pool = ProcessPool(workers=3)
pool.start()

def task(arg: int):
    return 2 + arg

result = pool.submit_and_fetch(task, 5)  # 7

pool.close()

It is possible to stream a generator from a process:

pool = ProcessPool(workers=3)
pool.start()

def task(max: int = 10):
    for i in range(max):
        yield i

for result in pool.submit_and_stream(task):
    print(result)  # Will print 0 through 9 inclusively

pool.close()

Restrictions

  • Anything dill can't pickle, can't be used as a task.
  • Arguments are pickled using the regular pickling for processes.

About

Tentacule is an uncomplicated library to deal with a pool of worker processes

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published