Support DigestAuth with "x-www-authenticate" in first response header

[kiryph]

The Fronius inverter for photovoltaic systems uses for its local REST API digest authentication.

It uses in its initial response header X-WWW-Authenticate instead of www-authenticate and hence make the httpx request fail.

Here an initial curl command with the response header:

❯ curl -I "http://192.168.1.141/config/emrs"
HTTP/1.1 401 Unauthorized
X-WWW-Authenticate: Digest realm="Webinterface area", charset="UTF-8", algorithm=MD5, nonce="66ec2f8d:36a65537553cf788606762854336a0b3", qop="auth"
Content-Type: text/html
Content-Length: 347
Date: Thu, 19 Sep 2024 14:05:01 GMT
Server: webserver

httpx 0.27.2 fails to extract X-WWW-Authenticate with nonce, qop, algorithm, realm in

httpx/httpx/_auth.py

Lines 201 to 212 in 87713d2

	if response.status_code != 401 or "www-authenticate" not in response.headers:
	# If the response is not a 401 then we don't
	# need to build an authenticated request.
	return
	for auth_header in response.headers.get_list("www-authenticate"):
	if auth_header.lower().startswith("digest "):
	break
	else:
	# If the response does not include a 'WWW-Authenticate: Digest ...'
	# header, then we don't need to build an authenticated request.
	return

My script

import httpx

url = "http://192.168.1.141/config/emrs" # adjust IP to that one of your inverter
user = "technician"
pw = <INSERT PW>

auth = httpx.DigestAuth(user,pw)

with httpx.Client(auth=auth) as client:
    r = client.get(url)
    print(r.status_code)
    print(r.json()['priorities'])

    r = client.get(url)
    print(r.status_code)
    print(r.json()['priorities'])

runs only with following quick diff

❯ git diff
diff --git a/httpx/_auth.py b/httpx/_auth.py
index b03971a..e4f1600 100644
--- a/httpx/_auth.py
+++ b/httpx/_auth.py
@@ -198,17 +198,27 @@ class DigestAuth(Auth):

         response = yield request

-        if response.status_code != 401 or "www-authenticate" not in response.headers:
+        auth_header_key = False
+        for key in [ "www-authenticate", "x-www-authenticate" ]:
+            if key in response.headers:
+                if auth_header_key == False:
+                    auth_header_key = key
+                else:
+                    message = "Malformed Digest WWW-Authenticate response header"
+                    raise ProtocolError(message, request=request)
+
+        if response.status_code != 401 or auth_header_key == False:
             # If the response is not a 401 then we don't
             # need to build an authenticated request.
             return

-        for auth_header in response.headers.get_list("www-authenticate"):
+        for auth_header in response.headers.get_list(auth_header_key):
             if auth_header.lower().startswith("digest "):
                 break
         else:
             # If the response does not include a 'WWW-Authenticate: Digest ...'
-            # header, then we don't need to build an authenticated request.
+            # or 'X-WWW-Authenticate: Digest ...' header, then we don't need to
+            # build an authenticated request.
             return

         self._last_challenge = self._parse_challenge(request, response, auth_header)

Note, this diff is only to show that it works. I did not follow any coding guidelines of this project.

For reference, here a couple of projects which have written their custom digest authorization to handle the Fronius response header:

• https://github.com/wiggal/GEN24_Ladesteuerung/blob/main/FUNCTIONS/httprequest.py
• https://github.com/muexxl/batcontrol/blob/main/fronius/fronius.py#L326-L361
• https://github.com/jakkaj/ha_fronius_set_service/blob/main/fronius_settings_service/script.py#L28-L112
• https://github.com/sergioperez/fronius-auth-proxy/blob/master/makeRequest.js
• https://github.com/montekarlos/homebridge-fronius-inverter-output-pin-switch/blob/master/src/fronius.ts#L29-L39
• https://github.com/thingsplex/fronius/blob/c1ba745d93eac4a9d6b4f38aedf1aff58bb4fb6b/src/handler/from-fimp.go#L319-L353
• https://github.com/christoh/FroniusMonitor/blob/3466b197a4f3c93831eca43e8f3dc85362335952/Fronius/Services/DigestAuthHttp.cs#L98-L100

Software versions of the inverter:

Device name 	Symo GEN24 6.0
WebUI 	1.23.1-3

Software-Revisions
CoyoteCore 	1.23.1-6
DEVICEGROUP 	1.28.7-1
GEN24 	1.28.7-1
GEN24ROW 	1.28.7-1
GEN24SYMO 	1.28.7-1
Kronos 	2.35.3-21598
Rhea 	2.15.1-2
S10RW-pilot 	1.23.1-6
Zeus 	2.27.2-13805
imx6sx-pilot 	1.23.1-6 

I am not sure how common it is to use X-WWW-Authenticate. Google did not return many results:

https://community.smartbear.com/discussions/readyapi-questions/digest-authentication-with-x-www-authenticate/252961

However, I still think supporting this deviation might be acceptable. If Fronius screwed this up more heavily than I think, I understand that httpx cannot support all quirks of all digest implementations.

[tomchristie]

It uses in its initial response header "X-WWW-Authenticate" instead of "www-authenticate" and hence make the httpx request fail.

fun

However, I still think supporting this deviation might be acceptable.

I doubt it.

Guidelines for these types of case...

• Do web browsers handle this case?
• Do other well used http clients handle this case?
• What's the spec say?

[kiryph]

@tomchristie

thanks for your quick response.

fun

IMHO it is not fun. But I can imagine how you mean it.

Apparently the prefix X- should not be used for a while. But is something which existed and probably will exist. Devices like this inverter will be easily around for more than 10 years since this device is still sold by Fronius and will run for more than 10 years. I have found three further projects for Fronius inverter which had to special case this digest authentication.

Do other well used http clients handle this case?

1. Node.js module urllib:

https://github.com/node-modules/urllib/blob/7c8aff0378cadbc88061fa4bf05c448e839ad59e/src/HttpClient.ts#L549

2. go-http-digest with the in-source comment to fall back to non-standard headers that some servers use to avoid browser popups:

	wwwAuth := resp.Header.Get("WWW-Authenticate")
	if wwwAuth == "" {
		// fall back to non-standard header that some servers use to avoid browser popups
		wwwAuth = resp.Header.Get("X-WWW-Authenticate")
	}

https://github.com/jpfielding/go-http-digest/blob/cffc47d5d6d859dc39e3e0bece5224f83ffbb27b/pkg/digest/transport.go#L187

Other occurrences:

• AngularJS Digest Authentication Interceptor: https://github.com/PatrickHeneise/angular-digest-interceptor

which also handles this. It might be the case that old AngularJS apps might have introduced it and the Fronius backend was developed in AngularJS.

• Nokia Data Gathering: https://github.com/nokiadatagathering/ndg-ng/blob/21d6a5d92c6b671f5ff503066ace331a86b81914/app/controllers/logic/AuthorizationUtils.java#L85

• Fork of werkzeug: MaxBear/werkzeug@b6cfc68

Since the probably well used node.js module urllib supports it, is there any chance that this might be more convincing than the Fronius inverter on its own?

[tomchristie]

Thanks, that's comprehensive.

Do web browsers handle this case?

[kiryph]

I am not sure what you mean. Can you help me how to check this?

If I understand the in-source comment of go-digest-auth correctly, AngularJS or whatever Fronius uses for its backend went for the non-standard header to avoid browser popups. This might be an indication that Browser do not support this.

[tomchristie]

Yes I'm re-reading that, and it's implicit that browsers don't support this.

Okay, thanks for the detailed investigation.
Yes there are some packages supporting this, no the usage isn't widespread enough that we'd make a change here.

We'll continue to require a custom auth scheme for users wishing to handle this.

[kiryph]

I understand.

For others here an example for the requests library which also struggles with the initial digest response:

import requests
from requests.auth import HTTPDigestAuth

url = "http://192.168.1.141/config/emrs" # endpoint of Fronius inverter
user = "technician"
pw = <pw>

r = requests.get(url, auth=HTTPDigestAuth(user, pw))

print(r.json()['priorities']) # priorities of the load management

and the corresponding diff in the requests lib to handle it:

❯ git diff
diff --git a/src/requests/auth.py b/src/requests/auth.py
index 4a7ce6dc..54a1578b 100644
--- a/src/requests/auth.py
+++ b/src/requests/auth.py
@@ -255,7 +255,7 @@ class HTTPDigestAuth(AuthBase):
             # Rewind the file position indicator of the body to where
             # it was to resend the request.
             r.request.body.seek(self._thread_local.pos)
-        s_auth = r.headers.get("www-authenticate", "")
+        s_auth = r.headers.get("www-authenticate", r.headers.get("x-www-authenticate", ""))

         if "digest" in s_auth.lower() and self._thread_local.num_401_calls < 2:
             self._thread_local.num_401_calls += 1

Note, the custom dictionary type in the requests library is case insensitive.