Error 401

[jeeberhardt]

I have an error 401 when I try to connect to https://xxxxx.fr/ews/exchange.asmx. It works with curl (or pycurl) but not with requests-ntlm. Thank you for your answer!

$ pip freeze | grep requests
requests==2.8.1
requests-ntlm==0.2.0

Curl:

#$DATA contains the xml in the python code
curl -s -u USERNAME:XXXX:XXXX -L https://xxxxx.fr/ews/exchange.asmx -d "$DATA" -H "Content-Type:text/xml" --ntlm

Python:

import requests
from requests_ntlm import HttpNtlmAuth

url = 'https://xxxxx.fr/ews/exchange.asmx'
username = 'DOMAIN.fr\\USERNAME'
password = 'XXXX:XXXX'

headers = {'Content-Type': 'text/xml'}

data = '''<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:xsd="http://www.w3.org/2001/XMLSchema"
               xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
               xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
  <soap:Body>
    <ResolveNames xmlns="http://schemas.microsoft.com/exchange/services/2006/messages"
                  xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"
                  ReturnFullContactData="true">
      <UnresolvedEntry>USERNAME</UnresolvedEntry>
    </ResolveNames>
  </soap:Body>
</soap:Envelope>
'''

response = requests.get(url, data=data, headers=headers, auth=HttpNtlmAuth(username, password))
print response

Python output:

<Response [401]>

TCPdump output:

sudo tcpdump -q
18:04:46.885156 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:46.933064 IP xxxxx.fr.https > home.54023: tcp 0
18:04:46.933192 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:46.946482 IP home.54023 > xxxxx.fr.https: tcp 517
18:04:46.982548 IP xxxxx.fr.https > home.54023: tcp 0
18:04:46.987380 IP xxxxx.fr.https > home.54023: tcp 1440
18:04:46.989147 IP xxxxx.fr.https > home.54023: tcp 1440
18:04:46.989217 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:46.989381 IP ixxxxx.fr.https > home.54023: tcp 1151
18:04:46.989442 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:46.993091 IP home.54023 > xxxxx.fr.https: tcp 214
18:04:47.035220 IP xxxxx.fr.https > home.54023: tcp 107
18:04:47.035350 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:47.035999 IP home.54023 > xxxxx.fr.https: tcp 949
18:04:47.075558 IP xxxxx.fr.https > home.54023: tcp 533
18:04:47.075653 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:47.077615 IP home.54023 > xxxxx.fr.https: tcp 1045
18:04:47.117239 IP xxxxx.fr.https > home.54023: tcp 789
18:04:47.117317 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:47.180575 IP home.54023 > xxxxx.fr.https: tcp 1301
18:04:47.221738 IP xxxxx.fr.https > home.54023: tcp 341
18:04:47.221816 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:47.223726 IP home.54023 > xxxxx.fr.https: tcp 0
18:04:47.260520 IP xxxxx.fr.https > home.54023: tcp 0
18:04:47.260641 IP home.54023 > xxxxx.fr.https: tcp 0

[Lukasa]

There doesn't appear to be any auth traffic there at all. Can you print response.headers for me please? response.history as well, if possible.

[jeeberhardt]

Here it is !

# print response.headers
{'Content-Length': '0', 'X-Powered-By': 'ASP.NET', 'request-id': 'a281ceb9-383f-44ae-9252-4c81a29a3e93', 'Server': 'Microsoft-IIS/8.5', 'X-FEServer': 'SERV-CAS2', 'Date': 'Sun, 15 Nov 2015 20:54:15 GMT', 'WWW-Authenticate': 'Negotiate, NTLM'}
# print response.history
[<Response [401]>, <Response [401]>]

[Lukasa]

Hmm, you seem to be struggling to auth with NTLM. NTLM is a fairly tricky auth method. Can you also run:

for r in response.history:
    print r.headers

[jeeberhardt]

Classic Microsoft ...

{'Content-Length': '0', 'X-FEServer': 'SERV-CAS2', 'X-Powered-By': 'ASP.NET', 'request-id': '20621211-3b20-4cfa-98c6-35ac87302446', 'Server': 'Microsoft-IIS/8.5', 'X-WSSecurity-Enabled': 'True', 'Set-Cookie': 'ClientId=XXWZQNUJE0MAOQEFUOW; expires=Mon, 14-Nov-2016 21:05:45 GMT; path=/; HttpOnly', 'Date': 'Sun, 15 Nov 2015 21:05:45 GMT', 'X-OAuth-Enabled': 'True', 'WWW-Authenticate': 'Negotiate, NTLM', 'X-WSSecurity-For': 'None'}

{'Content-Length': '0', 'X-Powered-By': 'ASP.NET', 'request-id': 'd1d21ed6-4d5c-44b1-a8a7-2e78c6eaec1b', 'Server': 'Microsoft-IIS/8.5', 'X-FEServer': 'SERV-CAS2', 'Set-Cookie': 'ClientId=LSAGTGZ0SICJQVQWSQ; expires=Mon, 14-Nov-2016 21:05:46 GMT; path=/; HttpOnly', 'Date': 'Sun, 15 Nov 2015 21:05:45 GMT', 'WWW-Authenticate': 'NTLM TlRMTVNTUAACAAAACgAKADgAAAAFgomiOITAEhsmFgsAAAAAAAAAAMAAwABCAAAABgOAJQAAAA9JAEcAQgBNAEMAAgAKAEkARwBCAE0AQwABABIAUwBFAFIAVgAtAEMAQQBTADIABAAkAGkAZwBiAG0AYwAuAHUALQBzAHQAcgBhAHMAYgBnAC4AZgByAAMAOABzAGUAcgB2AC0AYwBhAHMAMgAuAGkAZwBiAG0AYwAuAHUALQBzAHQAcgBhAHMAYgBnAC4AZgByAAUAJABpAGcAYgBtAGMALgB1AC0AcwB0AHIAYQBzAGIAZwAuAGYAcgAHAAgAkjtzZekf0QEAAAAA, Negotiate'}

[Lukasa]

So that's a real NTLM auth challenge. Working out why this isn't working is really going to be quite tricky, I'm afraid. =(

Out of interest, does curl work out your NTLM domain? Do you want to try removing it from the requests case, and having your username just be USERNAME?

[jeeberhardt]

yeah ... I think so, too. ^^
But thank you for your time, it is not your fault if NTLM is a little bit shitty. I tried with urllib2, and it doesn't eitheir. At the end, I will keep my (ugly) subprocess functions with curl.

Yes, it does. If I am just using my USERNAME, curl is still working. (Is it what you asked?)

[Lukasa]

@jeeberhardt Does it work with requests?

[jeeberhardt]

Nope, if remove the DOMAIN\\ from DOMAIN\\USERNAME and just keep USERNAME, I have this error:

ValueError: username should be in 'domain\\username' format.

But if I keep \\USERNAME, I have this:

<Response [501]>

Is it what you wanted ?

[Lukasa]

Nope, 501 suggests that the server doesn't like that at all. Hmm, I'm currently out of ideas for how best to debug this I'm afraid. =(

[jeeberhardt]

Argh ... or maybe because of a special character in the password ?

[Lukasa]

That's certainly possible!

[jeeberhardt]

Apparently not, I just changed it and ... nothing ! So thank you again, I will stay with subprocess + curl.

[jhoxx]

Hi Cory,
I am having exactly same problem and I believe that it is cased by forcing authentication to NTLM_v2 only. Is there anything we can do with it ?

thanks

J.

[Lukasa]

@jhoxx I'm not sure I fully understand that question.

[jhoxx]

I tried to authenticate to IIS service via requests-ntlm, but it gave me a 401 repsponse. I can see the request on server, with error message pointing to "wrong passwords". This usually happens when authentication is made by NTLM version 1. Thus I would like to know if there is any way how to enforce requests-ntlm to use NTLM v2 protocol only.

J.

[Lukasa]

Currently no support exists in this library, as far as I can see. You'll need to see what code is required from our dependencies and then check whether an appropriate patch fixes your problem.

[jhoxx]

Ok, thank you Cory.

[vmuriart]

@jhoxx, @jeeberhardt, are you guys using Windows machines? We can try to see if the problem is coming from ntlm3 library, by using pywin32 to perform the negotiations

[jhoxx]

@vmuriart, Hi Vik, yes, I do use Windows and yes I can try if you help me with the API call.

J.

[vmuriart]

@jhoxx give the pr a try and import HttpNtlmSspiAuth a try.

[Loknar]

I'm experiencing what seems to be the same issue as @jeeberhardt described, except it only happens when using a proxy setup (by pointing http_proxy/HTTP_PROXY and https_proxy/HTTPS_PROXY environment variables to a proxy service, on Ubuntu), without the proxy settings things are working fine ..

Edit: Oh wow, the auth failure was caused by http vs https and how the proxy handled that, problem solved now, the fact that http was working without proxy but failing with proxy was confusing me, seems that Upgrade-Insecure-Requests: 1 was added to the header and that somehow caused the NTLM Authendication to fail, switching to https when accessing the endpoint through proxy solved my problem.

[mikeyg123]

I have the same problem - however I'm using a java jespa based server which I can debug.

The server works with curl, chrome, ie etc. Debugging the server I can see the ntResponse is a 24 byte array.

Looking at the response from requests-ntlm I see the ntResponse is a 158 byte array. The authentication server then rejects this with error -1073741715 (0xc000006d)
All the documentation I've found tells me that the ntResponse should be 24bytes - e.g. https://www.innovation.ch/personal/ronald/ntlm.html

I realise that it's been a while but could anyone take another look please

Thanks
MikeG

[mikeyg123]

Update:
I got this working by downgrading to requsts-ntlm 0.3.0
pip install requests-ntlm==0.3.0

[chenming1986]

I was facing the same problem and @mikeyg123 's comment fixed it. Thanks!

[douglasquintanilha]

Same problem, @mikeyg123 solution worked for me, thanks man.

[mikeyg123]

The problem has been clearly shown to be an issue with the ntResponse
This should be pretty simple for someone who knows the code to fix
yet years go by
and nothing happens

[fzzylogic]

Ran into this problem too and downgrading fixed it for me too. It works on 0.3.0, 1.0.0 and 1.1.0 but not 1.2.0. System is Windows 10 Pro, authenticating against locally installed SSRS.

[doomwomble]

Same here, authenticating to SharePoint 2019 on-prem. Updating to 1.2.0 breaks the authentication code, giving me an HTTP 401 and reverting to 1.1.0 fixes the issue.

[yongdiedie]

I am NET WEB API project, configured with Windows authentication, version 1.2.0 will cause my interface to error 401. Even if there are no issues with my account and project configuration, the problem will automatically disappear when downgraded to version 1.1.0

[simonlee-hello]

I encountered the same issue when using an NTLM hash for authentication and received a 401 response. Later, I solved the problem by processing the hash before passing it in. For example, you can handle it like this:

def get_password_or_hash(args):
    if args.password:
        return args.password
    elif args.hash:
        return f"00000000000000000000000000000000:{args.hash.upper()}"