Advanced Inputs and Fighter State Transitions

[zicklag]

Continuing discussion from #203.
Relevant wiki: Fighter-State-Machine

I'm still a little unsure about what the best practice will be as far as where to make decisions about which state to transition to. For instance, right now we have the collect_player_actions system making a decision, based on the fighter's inventory, what state to transition to when you press the Throw button.

This seems good in this case, but it get's a little more confusing when consider things like mid-air attacks where the same attack button might trigger a different state. Should collect_player_actions be responsible for making that decision? Or does it just depend on a case-by-case basis?

I'll post my current stream of thoughts in a comment below.

[zicklag]

@odecay @edgarssilva I think we might need to overhaul the way we currently handle inputs.

For simply controlled games where one button maps to one action, the leafwing_input_manager model of putting things in nifty ActionState wrappers is really great, but I think our needs are much more advanced. Here's what I'm thinking.

Input Pipeline

We handle the input as an input pipeline where each step in the pipeline hands off it's output to the next step:

1. Raw Input

We have the raw user input step. This is just Bevy's giving us access to inputs, so nothing to do here.

2. "Virtual Gamepad"

We use the controller settings to map that raw inputs to a virtual "gamepad". This is different than mapping inputs to actions like we do now.

For instance, we may have an "attack" button, just like Smash Brothers. And pressing the attack button alone does one kind of attack. But simultaneously press the attack button and pushing the movement right will trigger a different attack.

So we would do the same thing.

Our virtual gamepad will have buttons such as "attack", "jump" and a directional input for "move", and whatever other buttons we need.

This allows users to re-map their controls, and it abstracts the movement away, we don't care whether you use a DPad or a Control stick, etc., but it doesn't actually say anything about what kind of button combinations does what. Just what kind of buttons there are.

This step will also handle determining which player the events are related to.

3. Input Buffer

Each player will have an input buffer that the Virtual Gamepad step will write to. This buffer will garbage collect old events by some logic whether it be number of events or how long ago the event happened. Each event will record what the event was and what time it happened.

Note: The VecDeque might come in handy here.

4. State Transition Collectors & State Handlers

The current collect_player_actions() function would go away, because it would be replaced by the input buffer. The transition_from_[state]() functions would then be responsible for deciding what states to change to based on the input buffer.

This makes it possible for us to have an "in the air" state, and have it decide to do an air attack of you press "attack" while you're in the air. It answers some of our questions about logic that we put into collect_player_actions() that seemed like maybe shouldn't go in there.

I think we would still want the ability for other systems to push prioritized state transition intents, because we might want that for status effects that have the ability to do a "forced takeover" of the players state, as long as it has a higher priority.

What we wouldn't have is the model we have currently where the moving state just takes over for the idling state because it's higher priority. Priority alone just isn't flexible enough for us to determine which state we should move to in the advanced cases.

Instead the idling state would choose that it goes to moving when it finds a move event in the input buffer.

Note: @odecay mentioned that we will also want to have a way to take into account when we are in recovery frames, and that input may trigger a new action, cancelling the recovery state.

Input Buffer Query API

On an implementation note, I think we're going to want to be kind of clever about how we design the input buffer query API.

I'm thinking we're going to want to be able to do a sort of "pattern matching" on the buffer. For example we can say, if this button was pressed, and there was a pause where it wasn't pressed that was shorter than x and longer than y, and then this button was pressed again, then trigger an attack".

This should be able to be structured as a kind of query on the input buffer, and the input buffer will return whether or not the query matches the buffer's current state.

We also might want these queries to be representable in YAML/JSON, so that our fighter YAMLs can specify the triggers for different attacks.

Then we could have our YAML say something roughly like:

"When in this state, if this query matches, transition to this other state"

[zicklag]

So we could do combos like ["punch", "punch", "up", "punch"] in the YAML files that can trigger different attacks?

Yeah, I think so. We might also need a bit more expressiveness to represent things like double-tapping buttons within a short time frame to trigger a run, etc.

[odecay]

I do think we will want an actual Input buffer(I guess in LWIM terminoligy this is going to be an ActionLike buffer) rather than what in this game would be an action intent/state transition intent buffer. This would let us do things like double tap or dpad motion inputs, or deciding the window in which to differentiate between a "hold button" input and a "hit two buttons at the same time" input which have overlapping buttons. I think it's different from what you describe in "arpeggios" but there might be some overlap.

[zicklag]

I do think we will want an actual Input buffer rather than just an action intent buffer

I'm pretty sure that's what I meant to describe above, if I understand what you're saying right.

The input buffer holds the buffer of input events mapped to the "virtual gamepad". So we are storing inputs, not action intents in the buffer. They are just mapped from whatever the controller mappings are, to a universal representation that we can use to map on gamepad actions without having to know whether they are pressing the "A" button or the "B" button to jump.

[odecay]

3. Input Buffer

Each player will have an input buffer that the Virtual Gamepad step will write to. This buffer will garbage collect old events by some logic whether it be number of events or how long ago the event happened. Each event will record what the event was and what time it happened.

Note: The VecDeque might come in handy here.

4. State Transition Collectors & State Handlers

The current collect_player_actions() function would go away, because it would be replaced by the input buffer. The transition_from_[state]() functions would then be responsible for deciding what states to change to based on the input buffer.

This makes it possible for us to have an "in the air" state, and have it decide to do an air attack of you press "attack" while you're in the air. It answers some of our questions about logic that we put into collect_player_actions() that seemed like maybe shouldn't go in there.

I think we would still want the ability for other systems to push prioritized state transition intents, because we might want that for status effects that have the ability to do a "forced takeover" of the players state, as long as it has a higher priority.

What we wouldn't have is the model we have currently where the moving state just takes over for the idling state because it's higher priority. Priority alone just isn't flexible enough for us to determine which state we should move to in the advanced cases.

Instead the idling state would choose that it goes to moving when it finds a move event in the input buffer.

I think this is almost complete spec of what we will want, additionally we will want to determine where in an animations frames we are, eg if we are in recovery frames of an action, we may cancel the remainder of the animation and state with a new input. This also could be implemented by adding state component for "recovery window" if we didnt want to be checking against animation frames directly in state transition systems.

[zicklag]

additionally we will want to determine where in an animations frames we are, eg if we are in recovery frames of an action, we may cancel the remainder of the animation and state with a new input.

Oh, good idea! I'll add a note about that.

[zicklag]

Building on the idea above I think we might be able to essentially migrate to having a set of rules for when to transition out of any state, and these could be specified in the YAML. This would replace the transition_from_[state]() systems.

It could be something like this:

# When in this state
falling:
  # Any externally triggered state with a higher priority will forcibly
  # take over this state.
  #
  # TODO: We may not need priority if we have `immune_to`?
  priority: 20

  # Regardless of priority, this state will never yield to these state
  # transitions.
  immune_to:
    # We're in the air, so ground pounds don't effect us
    - ground_quaking

  # What to transition to when we naturally exit the "falling" state
  exit_to: idling
  
  # Define ways to transition out of the state by user input
  inputs:
    # When this happens
    - query:
        - !Punch
        - !Delay
          duration: 0.1
        - !Punch
      
      # Transition to this state
      to: double_air_attacking
    
    # And another one
    - query:
        - !Move
          direction: [0, -1]
      to: quick_dropping

This allows us to map the fighter state machine out pretty nicely, and make it different for each fighter with their YAMLs.

[odecay]

I was thinking that it would be nice to have some more generic way of defining transitions from states, I was noticing so far I was ending up with multiple state transition systems from different attacking states which look almost identical except for the state they operate on.
Although I was also thinking of solving that with something like this https://bevy-cheatbook.github.io/patterns/generic-systems.html?highlight=generic#generic-systems

[odecay]

I do think I like something like defining possible state transition via assets seems nice though.

[zicklag]

I was thinking that it would be nice to have some more generic way of defining transitions from states, I was noticing so far I was ending up with multiple state transition systems from different attacking states which look almost identical except for the state they operate on.

Yeah, I was noticing that too. I think we can probably reduce this to one state transition system that uses the same rule system for all of the states. And that works out great for the fighter asset because if its a rule instead of a dynamic system, it fits into YAML and gives you this nice "mind map" of the fighter's state machine without having to poke around the code.

[odecay]

I also want to mention this crate and this video about tree based animation state transition.
I think the crate may be a bit overcomplicated/not fit for us but I think we could do something similar here and it seems like a really nice way to deal with possible state transitions.
https://github.com/PhaestusFox/bevy_sprite_animation
https://www.youtube.com/watch?v=5aHhmRiVpZI

[odecay]

The tree data structure for state transitions in particular is the part that I think seems nice.

[zicklag]

I'll watch the video if I get the chance, but I read the readme and it looks kind of nifty.

I liked the way you could add metadata to frames. We might be able to take inspiration from that.

[odecay]

I really like the metadata on frames aspect as well, it would be a more generalized way of doing what we are already using the AttackFrames component for.

It's not really necessary to watch the video I linked, but its the unity version this model that the bevy plugin was inspired by.

[zicklag]

Maybe when you specify the animation frames in the fighter asset you could add extra metadata instead of just the frame index.

frames:
  - idx: 0
  - idx: 1
  - idx: 2
    play_sound: sound.ogg

[odecay]

Yeah that seems like it could be nice, I wonder if we are thinking about animation frames we also want to consider #191