Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fatal error link detached due to IdleTimerExpired #50

Open
lohmanndouglas opened this issue May 1, 2021 · 0 comments
Open

Fatal error link detached due to IdleTimerExpired #50

lohmanndouglas opened this issue May 1, 2021 · 0 comments

Comments

@lohmanndouglas
Copy link

lohmanndouglas commented May 1, 2021

Version:

github.com/amenzhinsky/iothub v0.7.0

Error:

Link detached, reason: *Error{Condition: amqp:link:detach-forced, Description: The link 'G2:...' is force detached. Code: ServerError. Details: AmqpEventHubConsumer.IdleTimerExpired: Idle timeout: 00:30:00.

I am getting this error message and restarting the pod Kubernetes container frequently. I had this code working for a long time and it did not happen so frequently. The first time, I thought that it was idle, but I checked the last message received was one minute before and not 30 minutes, maybe I misunderstand the Azure concept of idle in the Iothub bus and be observing the wrong parameters.

My contantainer log is:

...
{"time":"2021-05-01T06:17:58.627919145Z", "message received"}
{"time":"2021-05-01T06:17:58.633331806Z", "message received"}
{"time":"2021-05-01T06:17:58.638524857Z", "message received"}
{"time":"2021-05-01T06:18:02.227702684Z","level":"FATAL","prefix":"-","file":"main.go","line":"47","message":"link detached, reason: *Error{Condition: amqp:link:detach-forced, Description: The link 'G2:omited' is force detached. Code: ServerError. Details: AmqpEventHubConsumer.IdleTimerExpired: Idle timeout: 00:30:00.}

My golang code is:

parentCtx, _ := context.WithCancel(context.Background())
log.Fatal(iotHub.SubscribeEvents(parentCtx, func(msg *iotservice.Event) error {
	err = json.Unmarshal(msg.Payload, &identifier)
	log.Info(" identifier :", identifier)
	if err == nil {
                 HandleID(identifier)
	} else {
		log.Error("could not marshing identifier: ", err)
	}
       return nil
}

I am considering doing something like the following draft, but a little afraid of losing messages, now if there is an error I see due to the container reinitialization and fix it as soon as possible to do not lose messages.

draft:

for {
       err = iotHub.SubscribeEvents(parentCtx, HandleFuncHere)
       if err != nil {
	    if err == context.DeadlineExceeded { // IdleTimerExpired?
                     log.Errorf("iothub idle timeout")
	    } else {
		    log.Fatal(err) // another error
	    }
       }
	continue
}

This is a reasonable solution? Did someone experience something similar?

I appreciate any help that someone provides to workaround or understand better this issue, thanks :)

@lohmanndouglas lohmanndouglas changed the title Handle without fatal error when link detached due to IdleTimerExpired Fatal error link detached due to IdleTimerExpired May 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant