Fixing the 403 Forbidden Error From Keycloak's UserInfo Endpoint
During my internship I was fixing a bug in an API endpoint. The method calls a service provided by another system (A).
- The method requests Keycloak’s
/auth/realms/momenta-prod/protocol/openid-connect/tokento obtain an Access Token - It puts the Access Token into the request headers and calls the service provided by system A
The HTTP status code that came back was 500. Looking at the response, the immediate cause was an unhandled exception, but fortunately the downstream service’s logs were fairly complete. From them I could tell that an HTTPError was thrown inside the service while the mozilla_django_oidc library was handling OpenID Connect authentication. More specifically, the request to Keycloak’s UserInfo endpoint returned a 403 Forbidden error.HTTPError at /api/v1/xxx403 Client Error: Forbidden for url: https://{keycloak-server}/auth/realms/momenta-prod/protocol/openid-connect/userinfoRequest Method: GETRequest URL: https://{a-server}/api/v1/xxx?Vehiclename=L7-PMV304Django Version: 4.2.6Python Executable: /usr/local/bin/python3Python Version: 3.11.4Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view return view_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/viewsets.py", line 125, in view return self.dispatch(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 515, in dispatch response = self.handle_exception(exc) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 475, in handle_exception self.raise_uncaught_exception(exc) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 486, in raise_uncaught_exception raise exc ^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 503, in dispatch self.initial(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 420, in initial self.perform_authentication(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 330, in perform_authentication request.user ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/request.py", line 232, in user self._authenticate() ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/rest_framework/request.py", line 385, in _authenticate user_auth_tuple = authenticator.authenticate(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mozilla_django_oidc/contrib/drf.py", line 80, in authenticate user = self.backend.get_or_create_user(access_token, None, None) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mozilla_django_oidc/auth.py", line 347, in get_or_create_user user_info = self.get_userinfo(access_token, id_token, payload) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/mozilla_django_oidc/auth.py", line 281, in get_userinfo user_response.raise_for_status() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 1024, in raise_for_status raise HTTPError(http_error_msg, response=self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Exception Type: HTTPError at /api/v1/VersionLasterException Value: 403 Client Error: Forbidden for url: https://{keycloak-server}/auth/realms/momenta-prod/protocol/openid-connect/userinfo
In theory this should have been an easy bug to fix. But since it involved integrating with another service, debugging was awkward, all I had was the service’s incomplete logs to work from. OIDC apparently does return some hints in the response headers, telling you the 403 Forbidden is caused by a missing openid scope, but the response was so long that I honestly never noticed ()
And on top of that, OIDC was a blind spot for me, I had no battle scars here yet
Solution
Keycloak’s UserInfo endpoint requires the Access Token to contain the openid scope; otherwise it returns a 403 Forbidden error. This is mandated by the OpenID Connect specification, to ensure only properly authenticated tokens can access user information. For details see Securing applications and services with OpenID Connect - Keycloak & Final: OpenID Connect Core 1.0 incorporating errata set 2
Under some Keycloak configurations, the Access Token’s JWT payload may not directly include a scope field, but as long as you request the token with the openid scope, the UserInfo endpoint works fine. Here is some Python code for referenceimport requestsdef get_access_token(): client_id = 'your client id' client_secret = 'your client secret' url = 'https://{your-keycloak-server}/auth/realms/{your-realm}/protocol/openid-connect/token' headers = {"Content-Type": "application/x-www-form-urlencoded"} payload = { "grant_type": "client_credentials", "client_id": client_id, "client_secret": client_secret, "scope": "openid" # ❗️ make sure the openid scope is included } resp = requests.request("POST", url, headers=headers, data=payload) content = resp.json() return content['access_token']def request(): access_token = get_access_token() url = 'your request url' headers = {'Authorization': f'Bearer {access_token}'} resp = requests.request(method, url, headers=headers)
Adding scope=openid to the request body ensures the Access Token contains the openid scope. After decoding the Access Token at JSON Web Tokens - jwt.io, you can see the newly issued Access Token now includes openid in the scope claim
Left/right: tokens without/with the openid scope
