Account fails to reconnect using Refresh Token after Access Token expires

[pungys97]

Issue:
When utilizing the O365 library in Python, I have encountered an issue where the connection is not re-established or refreshed when the access token has expired, even though a valid refresh token is present. If I attempt to use the Account object after an inactivity of more than 60 minutes (post the expiration of the access token), the connection fails. I could not find an inbuilt mechanism within the O365 library to handle this. However, as a workaround, I am able to manually utilize msal to obtain a valid token using auth_app.acquire_token_by_refresh_token(), and then re-establish the connection.

Steps to Reproduce:

1. Set up a Python application with the O365 library.
2. Authenticate and establish a connection to O365.
3. Wait for more than 60 minutes without performing any operations, ensuring the access token expires.
4. Try making a request using the Account object.
5. Observe that the connection fails and doesn't automatically use the refresh token to obtain a new access token.

Expected Behavior:
The library should automatically use the available valid refresh token to obtain a new access token and re-establish the connection without manual intervention.

Actual Behavior:
The connection fails when the access token expires, even with a valid refresh token available. Manual intervention using msal is required to fix the connection.

Workaround:
Using msal directly, I am able to get a valid token with auth_app.acquire_token_by_refresh_token(), allowing me to reconnect successfully.

Environment:

• O365 library version: 2.0.27
• MSAL library version: 1.23.0
• Python version: 3.10
• Operating System: masOS Ventura 13.4.1

[kwilsonmg]

@pungys97 I've run into this as well. Would you be able to provide an example of how you can use msal directly to use auth_app.acquire_token_by_refresh_token()? Where are you making this call?

[alejcas]

Unless I'm missing something, this de default behavior. So by default the token only lasts 60 minutes unless you add the offline permission to the app. Once this permission is added, the retrieved token gets a "refresh token", that is used after those 60 minutes to get another new access token along with a new refresh token.
This refresh token can be used within 90 days.

This is all explained in the readme

[kwilsonmg]

I meant to come back here and update that I ended up realizing the issue by directly calling the refresh function in Account.py and facepalming. I retract what I said there and suspect that that was the issue @pungys97 ran into. This can probably be closed @janscas as I can confirm it does work.

[pungys97]

Well I checked and we have the scope applied so this should not be the issue. We have refresh_token and I am able to retrieve new access token from msal. But I probably found the root cause:
ERROR Client Error: 401 Client Error: Unauthorized for url: https://graph.microsoft.com/v1.0/me/messages?%24top=1 | Error Message: IDX14100: JWT is not well formed: '[PII of type 'Microsoft.IdentityModel.Logging.SecurityArtifact' is hidden. For more details, see https://aka.ms/IdentityModel/PII.]', there are no dots (.). The token needs to be in JWS or JWE Compact Serialization Format. (JWS): 'EncodedHeader.EndcodedPayload.EncodedSignature'. (JWE): 'EncodedProtectedHeader.EncodedEncryptedKey.EncodedInitializationVector.EncodedCiphertext.EncodedAuthenticationTag'. | Error Code:

I believe I did not provide correct description of the whole problem. We are using msal for oauth to login user to the app. From the msal we get the access token and the refresh token. When I then try to use the o365 Account within 60 minutes it works just fine. However, when I try to use it after 60 minutes I am not able to. So the access token from msal works but the refreshing does not. When I call authenticate method, then I am prompted to login again. I have a custom token backend that loads and saves the credentials.

[pungys97]

Ok so I figured out what my problem was. I misunderstood the logic of refreshing token in the background. I thought that when I provide custom token backend it would try to refresh the token when it is expired on initialisation. Explicitly calling account.connection.refresh_token() solves my problem.

[alejcas]

Hi, default token backends will automatically force a call to refresh_token when the token is expired.

See here.

When building a custom TokenBackend make sure that if you overwrite should_refresh_token, you end up returning TRUE.

If you don't overwrite that function the default behavior is to always return TRUE as you can see here.

[pungys97]

Ok. So so in order for the token to refresh you need to get the TokenExpiredError, however I seem to get HttpError instead and therefore the token never gets refreshed (see below the log). When I call refresh_token manually it works just fine.

DEBUG https://graph.microsoft.com:443 "GET /v1.0/me/messages?%24top=1 HTTP/1.1" 401 None
ERROR Client Error: 401 Client Error: Unauthorized for url: https://graph.microsoft.com/v1.0/me/messages?%24top=1 | Error Message: CompactToken validation failed with reason code: 80049228. | Error Code: 

[alejcas]

Interesting! Seems like microsoft changed something. I’ll try to fix this

[alejcas]

Can you please confirm that the error raised is: InvalidTokenError?

I can't check what oauth2 error is being raised when you get the error:

Error Message: CompactToken validation failed with reason code: 80049228.

[alejcas]

Sorry you already answered. So it's a requests simple HttpError.

Can you provide the whole raw error text?

This error should be raised by oauthlib and it is not raising it.

[pungys97]

To be honest, I'm not able to consistently reproduce this. Sometimes the refreshing works just as it should and sometimes I get this error. I really keep going back and forth between trying to find bug on my side and believing my code is fine. The reason why I believe the underlying issue is not in my code is that when I manually call the refresh_token() or use the msal with the same credentials I manage to re-authenticate.

The entire error message is this:
{'error': {'code': 'InvalidAuthenticationToken', 'message': 'CompactToken validation failed with reason code: 80049228.', 'innerError': {'date': '2023-11-09T23:53:27', 'request-id': 'uuid', 'client-request-id': 'uuid'}}}

[pungys97]

Anyways, I think the easiest solution for me is to wrap the whole thing in try/catch and refresh there explicitly.

[alejcas]

Anyways, I think the easiest solution for me is to wrap the whole thing in try/catch and refresh there explicitly.

Sure but O365 should do it automatically.

I'll investigate further into this.

Thanks

[alejcas]

Hi seems like this error is returned by MS Graph when you use the refresh token BEFORE the access token.
Are you somehow calling refresh token elsewhere?

[alejcas]

Another possibility is that somehow your environment is changing time on the machine executing the code?

O365 uses time to see if the token is expired. Maybe the first time it gets the token and before using it, it notices the token is expired and then call for a refresh token...

https://github.com/O365/python-o365/blob/master/O365/utils/token.py#L25