mechanize crash

[jorissen]

tried script with mixed result -- downloaded about 120 workouts, then crashed with

2015-01-31_700755201.txt is downloading...
Traceback (most recent call last):
File "download.py", line 207, in
download_files_for_user(username, password, output)
File "download.py", line 162, in download_files_for_user
activities(agent, download_folder)
File "download.py", line 131, in activities
datafile = agent.open(url).get_data()
File "/Library/Python/2.7/site-packages/mechanize/_mechanize.py", line 203, in open
return self._mech_open(url, data, timeout=timeout)
File "/Library/Python/2.7/site-packages/mechanize/_mechanize.py", line 255, in _mech_open
raise response
mechanize._response.httperror_seek_wrapper: HTTP Error 500: Internal Server Error

[jorissen]

Also, suggestion -- .txt file extension is confusing, maybe better to choose .tcx or .gpx

[jorissen]

This is for the download.py script -- sorry, seeing only now that you have a lot of other stuff in this repository, too.

[magsol]

Is it a specific workout that it crashes on (e.g. 2015-01-31_700755201) or after a specific number of downloads (e.g. ~120) or just completely random?

The 500 error referenced is reminiscent of what was plaguing even the manual TCX download on Garmin Connect for months.

Agree on the download extension, too; feel free to submit a PR for that!

[jorissen]

I've tried running it several more times, and it always times out before downloading all of the workouts in my account, usually after a handful of workouts. Since it doesn't re-download files, I've collected most of what I need, step by step. I'm not sure if the mechanize library is just overly sensitive to something; or garmin's servers are flaky as hell. The first time (reported above, when I was able to grab 120 files in one go) was on a very good internet connection - maybe that's also a factor.

I have only the one garmin account, and not much insight into these processes, so I probably don't have much more to contribute.

Merry Christmas --

[magsol]

Thank you for the descriptive feedback; that's very helpful.

I'm currently traveling so I don't have the best bandwidth right now for testing, but I've committed a refactoring in a separate branch that no longer relies on mechanize. If you're able, feel free to check out that branch (requests) and test it out; from a black-box perspective it should be functionally identical.

Merry Christmas to you as well, and Happy New Year!

[jorissen]

~/Downloads/garmin-requests% sudo pip install mechanicalsoup
[...]
~/Downloads/garmin-requests% /usr/bin/python download.py -u ***********
Garmin account password (NOT saved):
Attempting to login to Garmin Connect...
Traceback (most recent call last):
File "download.py", line 200, in
download_files_for_user(username, password, output)
File "download.py", line 145, in download_files_for_user
login(agent, username, password)
File "download.py", line 72, in login
login_url = 'https://sso.garmin.com/sso/login?%s' % urllib.parse.urlencode(data)
AttributeError: 'module' object has no attribute 'parse'
~/Downloads/garmin-requests%

[jorissen]

Any ideas?

[magsol]

Ooh. What version of Python are you using? I think that error goes away in Python 3.

[EjeWGU]

Some alternative for Python 3? Mechanize only works on 2.x
I tried to do it myself with RoboBrowser but my python level is not enough for it