Hidden Dynamics -- TA feedback

[ollimuh]

Bugs:

(report potential bugs / errors here (ideally along with a solution))

• Tutorial 1, section 1.1 equation (5) -- mu_1 to be changed to mu_R

• tutorial 2, equation 1. sum should be over i, not over j

• Captions in the videos are very off (tutorial 3 video 2 captions are borderline funny : ‘gaussian term eta’ != ‘gardener’s term either’)

Clarifications / Inconsistencies:

(report when something needs clarification (+suggest a clarification))

• Tutorial 1, Exercise 2: It would be nice to have a definition of ‘accuracy’ somewhere, given that this needs to be calculated in the exercise.

• Tutorial 1, Exercise 3 : there is a inconsistency btw the help notes on the returned values decision_lenght_list and the actual return variable decision_speed_list in the student notebook (solution is right though)

• Tutorial 1, Exercise 4 : Is there a reason why the average accuracy is represented wrt the decision speed while it was represented wrt to the decision time in exercise 2? I find it quite confusing. Moreover, I found that the way the decision speed is computed (as being the inverse
  of decision time) makes it very hard to interpret.

• Tutorial 2, Exercise 1: Confusing that it is called std, students then unneededly squared the noise

• noise_level (float): standard deviation of observation models. Same for two components

• Tutorial 2, Exercise 3 slider: should be specified that we are in a constant state 0, if not the variations of the graph when sliding across switch_prob can be misleading

• Tutorial 3, exercise 1.
  Defining state[0] in the solutions is set to mu_0, on the equations it sampled from a multivariate gaussian N(mu_0,sigma_0). Consistent solution:
  State[0] = stats.multivariate_normal(mean=params['mu_0'],c ov=params['sigma_0']).rvs(1)

• Tutorial 3, exercise 3.
  Sigma and other variables are inconsistent with the equations. It would be easier to follow with consistent notation. E.g Sigma_hat = sigma_filt

Questions:

(report here when something is unclear and you don’t have a fix)

• Tutorial 1, Exercise 4 requires too much debugging in the currents state. Can this be simplified/ reduced to one line of code? I won’t have enough time to go through this.

• Tutorial 2, Exercise 1: without looking at the solution, I'm completely lost what should be done there. Please add an instruction on what we want to achieve here!
  This is especially true for the covariance matrix. It is never mentioned above, and it has an unintuitive shape. There was no instruction/explanation on what the covariance matrix is or why it has the (2, 1, 1) shape.

• Tutorial 2, Exercise 1, model.covars_ = noise_level**2 maybe? Instead of just noise_level. It does not really change the result, though.

• For T2, I am extremely confused by the situation that we sometimes use A.T sometimes not for the transition matrix. What is the reason for such inconsistency? (NOTE: I understand the construct of matrix for A_ij to represent switch from j to i, or vice versa, but I wonder why they are not fixed on one representation)

• In T1, section 1.1, it would be nice to show how equation (5) and (6) were derived and why you had to do that. (Also applies to other equations, such as those in tutorial 3)

[mwaskom]

@ollimuh thanks much for creating the google doc and passing information back to us. For small problems that don't require emergency fixes, this is much better than individual bug reports. Can you please help to coordinate with the TAs so that people don't duplicate issues that will end up in the daily summary? There are a number of small comments about W2D4 rolling in and it's getting hard to stay on top of.

[ollimuh]

I added a section to the feedback document for each day for minor issues/typos.

[mwaskom]

Thanks ... it would also be helpful if the TA document said something like, add your issue here or post on github but don't do both.

[ollimuh]

You can edit the document according to your needs. This is what it currently says:

Scope:

As with any software, there may be bugs / inconsistencies / confusion in the tutorial notebooks, which have been missed during the prepods (especially towards the later Sections on each day). Fixing bugs or making changes on a short notice is not always a practicable option, as each change needs to be propagated to repositories accessible in China (and students there get to work on the tutorials first).

This document is supposed to collect potential issues as lead TAs prepare for tutorials on days of their expertise, to make them available as an Erratum on Neurostars (google docs are not accessible from China) and in the discussions for each day.

Content will be moved to Neurostars at midnight UTC (before the tutorials start) on each day.

Bugs and Clarifications / Inconsistencies will go to an Erratum post, the daily discussion and into a daily Github issue. Questions will go to the daily discussion thread.

Please use this document for collecting minor issues / typos instead of creating issues on Github. This will help content developers to focus on urgent issues during NMA (all sections for each day are being forwarded to a github issue after the tutorials end).

[mwaskom]

OK thanks, I see it's more a problem of getting people to actually follow directions :)