Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add abstraction of Azos.Sky.Fabric.Fiber - an orchestration context (e.g. result mailslot address) to execute distributed cloud services/units of work #813

Open
itadapter opened this issue Nov 21, 2022 · 5 comments
Assignees
Labels
CONCEPT Needs conceptual design session CRITICAL Critical Issue affecting the whole system PROJECT A project which is comprised of multiple tasks

Comments

@itadapter
Copy link
Contributor

itadapter commented Nov 21, 2022

Processes provide:

  1. Unique mutext ID whih can be used for
  2. Result/mailslot address for actors
  3. Chronicle execution log correlation
  4. Management, status tracking
  5. Supervisor actors ensure process final state machine transition

This new model supersedes prior art here:
https://github.com/azist/azos/tree/master/src/Azos.Sky.Server/Workers

Process - a global context of the larger task execution, establishes context for multiple instances of asynchronous Todos,
captures the final result of multiple Todos; reacts to Signals

IMPORTANT The system should be able to handle millions of active executing processes (i.e. not keep all in ram), as processes may represent certain business activities, such as a contract execution paused on a customer signature step) etc.. Sharding and Fail-over is 100% required as processes represent the ultimate distributed process supervision orchestration primitive

@itadapter itadapter changed the title Add abstraction of Sky.Process - a context (e.g. result mailslot address) to execute distributed cloud functions Add abstraction of Sky.Process - an orchestration context (e.g. result mailslot address) to execute distributed cloud services/units of work Nov 21, 2022
@itadapter
Copy link
Contributor Author

itadapter commented Nov 21, 2022

  GDID
  GUID

  parent_ID
  priority
  scheduled_time
  
  Creator_User  EntityId
  Create_Origin Atom
  CreateUtc 
  Tenant   string
  ShardKey
    
  Description
  Note

  Namespace   string
  Image  string;// fully qualified exeutable package (e.g. assembly)

  Status
  State  
  
  Result:()

ProcessResources (CPU, Memory, Disk)
ExitResult : Json
Memory(PID, Segment): (ctype, bin[])

@itadapter itadapter self-assigned this Dec 28, 2022
@itadapter
Copy link
Contributor Author

image

@itadapter itadapter changed the title Add abstraction of Sky.Process - an orchestration context (e.g. result mailslot address) to execute distributed cloud services/units of work Add abstraction of Sky.Jobs - an orchestration context (e.g. result mailslot address) to execute distributed cloud services/units of work Dec 28, 2022
zhabis added a commit that referenced this issue Dec 28, 2022
zhabis added a commit that referenced this issue Dec 29, 2022
zhabis added a commit that referenced this issue Dec 30, 2022
zhabis added a commit that referenced this issue Dec 30, 2022
zhabis added a commit that referenced this issue Dec 30, 2022
zhabis added a commit that referenced this issue Dec 30, 2022
zhabis added a commit that referenced this issue Jan 3, 2023
zhabis added a commit that referenced this issue Jan 3, 2023
zhabis added a commit that referenced this issue Jan 3, 2023
zhabis added a commit that referenced this issue Jan 5, 2023
zhabis added a commit that referenced this issue Jan 6, 2023
zhabis added a commit that referenced this issue Jan 6, 2023
zhabis added a commit that referenced this issue Jan 6, 2023
zhabis added a commit that referenced this issue Jan 6, 2023
@itadapter itadapter added CRITICAL Critical Issue affecting the whole system CONCEPT Needs conceptual design session PROJECT A project which is comprised of multiple tasks labels Jan 6, 2023
@itadapter itadapter changed the title Add abstraction of Sky.Jobs - an orchestration context (e.g. result mailslot address) to execute distributed cloud services/units of work Add abstraction of Azos.Sky.Fabric.Fiber - an orchestration context (e.g. result mailslot address) to execute distributed cloud services/units of work Jan 6, 2023
zhabis added a commit that referenced this issue Jan 6, 2023
zhabis added a commit that referenced this issue Jan 6, 2023
zhabis added a commit that referenced this issue Jan 6, 2023
zhabis added a commit that referenced this issue Jan 6, 2023
zhabis added a commit that referenced this issue Jan 9, 2023
zhabis added a commit that referenced this issue Jan 26, 2023
zhabis added a commit that referenced this issue Jan 26, 2023
zhabis added a commit that referenced this issue Jan 26, 2023
zhabis added a commit that referenced this issue Jan 27, 2023
zhabis added a commit that referenced this issue Jan 27, 2023
zhabis added a commit that referenced this issue Jan 27, 2023
zhabis added a commit that referenced this issue Jan 27, 2023
zhabis added a commit that referenced this issue Jan 27, 2023
@itadapter
Copy link
Contributor Author

itadapter commented Jan 27, 2023

todo

  • Validate State before commiting slots back to shard
  • Fiber.WriteLog()... add these methods
  • What is the very first current step????

zhabis added a commit that referenced this issue Jan 27, 2023
@itadapter
Copy link
Contributor Author

itadapter commented Jan 28, 2023

Need to test case: When BIX atr is forgotten on slot - we need not crash the Fiber... need to review exception types

incorrect, this must crash fiber as it is impossible to handle this otherwise,
the exception may surface only WHEN accessing certain state slot which is not decorated...
so it is a regular fiber exception and program can not continue

zhabis added a commit that referenced this issue Jan 28, 2023
zhabis added a commit that referenced this issue Jan 31, 2023
zhabis added a commit that referenced this issue Jan 31, 2023
zhabis added a commit that referenced this issue Feb 15, 2023
zhabis added a commit that referenced this issue Feb 16, 2023
zhabis added a commit that referenced this issue Feb 16, 2023
zhabis added a commit that referenced this issue Feb 16, 2023
zhabis added a commit that referenced this issue Feb 16, 2023
zhabis added a commit that referenced this issue Feb 17, 2023
zhabis added a commit that referenced this issue Feb 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CONCEPT Needs conceptual design session CRITICAL Critical Issue affecting the whole system PROJECT A project which is comprised of multiple tasks
Projects
None yet
Development

No branches or pull requests

2 participants