Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempting to lex kotlin style interpolated strings #423

Open
CreggEgg opened this issue Sep 21, 2024 · 3 comments
Open

Attempting to lex kotlin style interpolated strings #423

CreggEgg opened this issue Sep 21, 2024 · 3 comments
Labels
question Further information is requested

Comments

@CreggEgg
Copy link

I'm looking to lex a string like "${..}" where the ..'s represent other valid language tokens inside and am wondering what the best way to approach this is. I've tried to make a start and end token for this type of string but the end one is prioritized over this token: } on its own. My current attempt to circumvent this is

#[token("}")]
RBrace,
#[regex(r#""(?:[^(\$\{)]|\\")*(\$\{)"#)]
StartInterpolatedString(&'a str),
#[regex(r#"\}(?:[^"(\$\{)]|\\")*""#, interpolated_string_callback)]
EndInterpolatedString(&'a str),

and then a callback that looks like this

fn interpolated_string_callback<'a>(lex: &mut Lexer<'a, Token<'a>>) -> Option<Skip> {
    if lex.extras.string_state == StringState::Started {
        None
    } else {
        Some(Skip)
    }
}

But this is closer to pseudo code then anything actually useful and won't even compile and it seems to me like this is not the right approach for this issue. I'm new to logos and this seems atypically cumbersome based on my previous logos experiences so its very possible I'm just doing something really silly but I would really appreciate any help at all!

@jeertmans
Copy link
Collaborator

Hello, I think the best is to have a token InterpolationString(...) matching the start of an interpolation string, and have a callback handle everything:

fn callback(lex: ...) -> Vec<Token> {
    /* accumulate tokens until you match a right brace */
    /* but make sure to handle specific cases like escaped right brace, nested interpolation string, etc. */
}

#[derive(Logos)
enum Token {
    #[token("${", callback)]
    InterpolationString(Vec<Token>),
    #[token("}")]
    RightBrace,
}

@jeertmans jeertmans added the question Further information is requested label Sep 21, 2024
@CreggEgg
Copy link
Author

Thanks for the response! That makes sense to me. One question though, how would I get content at the start of the string for example: "User's name: ${name}" using this approach?

@jeertmans
Copy link
Collaborator

Apply the same logic, but for string, you match the start of a string and use a callback to parse its content.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants