Implement online user overrides

[bruno-f-cruz]

For now we will scope them to:

• Bump the spout side-ways by ∆X on each trial. This can be done by adding a numeric input + a "bump" button.
• Enable AutoWater for the next trial. This can be done by having 3 buttons: Left, Right, None. These will override the auto-water in the trial spec. It is not clear to me what happens under non-deterministic scenarios. E.g: if P_left = P_right =0.5. What should happen? scientists turn on water-water (which one? left or right?), and we sample the p_reward from left/right?

[bruno-f-cruz]

@micahwoodard The spout bias is pretty easy to implement. I still do not understand the business logic of the auto water tho. Any chance you (or someone else) can add more details here?

From the dynamic foraging repository I am finding conflicting information. In the implementation it sounds like auto water is implemented by side:

https://github.com/AllenNeuralDynamics/dynamic-foraging-task/blob/da8d2e9594f47fcb6630dedc1290b966e802a4d1/src/foraging_gui/rigcontrol.py#L172

However, in the readme:

On (Auto water): If the auto water button is clicked, auto water will be given based on type, multiplier, and conditions (unrewarded and ignored).

[micahwoodard]

@bruno-f-cruz here is the proposal from @hanhou about what the autowater and lickspout movement logic should be.

For simplicity, I think we could use only |bias| as the single metric for both autowater and autolickport.

I agree with Xinxin Yin that being able to turn auto water and lick spout on and off separately may be desirable, but I don't think their logic should be independent or we should only rely on lick spout even in early sessions. To me, using water as an incentive to shape behavior is better than moving the lickspout, since it does not change the nature of the task (i.e., tracking water reward) and thus doesn't make the mice mistakenly think it's a different task (i.e. tracking the lick spouts).

In summary, we may want:

• (Bowen) try water first, and if it doesn't work, move the lick spout
• (Han) move the lick spouts to the biased side if |bias| > threshold_bad and bring them back to center if |bias| < threshold_good
• (Xinxin) being able to control the relative strength of autolickspout and autowater over the training

I can think of a simple algorithm that implements all these points. It essentially uses a parameter antibias_water_first_max to control, once |bias| crosses threshold_bad, the number of auto-water attempts before we start to move the lick spouts:

antibias_water_count = 0
if bias_check_interval <= trial_number - last_bias_check:
    last_bias_check = trial_number
    if |bias| > threshold_bad:   # red state
        # do intervention
        if antibias_water_count < antibias_water_first_max:
             give_water_correction()
             antibias_water_count += 1
        else:
             move_lickspout_to_the_biased_side()
        return
    # Reset antibias_water_count once not in red state
    antibias_water_count = 0
    
    # If in green state, always bring lick spouts back to center
    if |bias| < threshold_good: 
        move_lickspouts_back_to_center()
       
    # If in yellow state, do nothing

In this implementation, antibias_water_first_max=0 gives us pure lickspout movement (my original proposal)

antibias_water_first_max=inf gives us pure autowater

and antibias_water_first_max=1 gives us Bowen's proposal

we can gradually increase the parameter antibias_water_first_max over the curriculum (from 1 to inf?).

[bruno-f-cruz]

This sounds good, but this is something to be calculated from the trial generator side. My initial question was not so much the algorithm used to calculate automatically whether or not to use auto-water, but instead how this algorithm interacts (or should interact) with manual control by experimenters at the rig. I am happy to remove manual control altogether, but it has to be understood to be the case.

[micahwoodard]

I am not sure on autowater or users interacting with it. Would be a good question for @alexpiet