Improve resilency of iudx systems in K8s

[abhi4578]

Motivation

What happens when one/more of the dependent systems say 'y' are down/unavailable of a particular system x and how would the system 'x' behave and improve the system 'x' resiliency accordingly. This is also needs to be coupled with lessons learnt/feedback from monitoring/incidents in k8s testing and production environment

This maybe a far fetched, but can look at the chaos mesh, use that principles for testing and may be slowly adopt the chaos mesh for testing the resiliency of systems in K8s