TwigPreLexer: improve performance

[gharlan]

Q	A
Bug fix?	no
New feature?	no
Tickets
License	MIT

When using the cache:warmup command in my symfony project, 25% of the time is used by the TwigPreLexer:

This PR reduces the time of the cache:warmup command by 28% in my project (which does not use the html component syntax so far):

[gharlan]

This change alone increases the cache:warmup time by 11% (in my project):

Together with the other change, this change is now only relevant for templates using the html component syntax.

[weaverryan]

Both of these changes make a lot of sense - and love seeing the Blackfire profile to back them up. Thanks Gregor!

[Kocal]

Wow, just being curious, but how the pre-lexing takes you ~15 seconds?
How many Twig templates do you have?

In our project, we use ~105 times <twig:...> syntax across ~44 Twig templates (we have ~550 Twig templates in total), and it took only ~4.40 seconds (in APP_ENV=dev APP_DEBUG=1):

[weaverryan]

I am wondering if, in some projects, people have custom Twig namespaces that may point to directories with other things in them (like images). But indeed, I hadn't looked at HOW long the time was in those screenshots - 40 seconds is still a crazy amount of time. After these changes, what percentage of your cache:warmup is from the pre-lexer? Is it not almost nothing or still very high (i.e. it still requires several seconds)?

[Kocal]

Btw we use symfony/ux-twig-component at version 2.12.0 and we didn't configure anything.

Our components are located under App\Website\Twig\Components and their Twig templates in templates/components.

When applying the patch locally, Symfony\UX\TwigComponent\Twig\TwigPreLexer::preLexComponents goes from ~4.40s to ~0.9s (still using APP_ENV=dev APP_DEBUG=1), which is really nice! :)

[gharlan]

How many Twig templates do you have?

Only 236.

I am wondering if, in some projects, people have custom Twig namespaces that may point to directories with other things in them (like images).

In my case there is nothing special (afaik). Default directory structure with templates directory (containing only templates).

But indeed, I hadn't looked at HOW long the time was in those screenshots - 40 seconds is still a crazy amount of time.

Yes and it is really annoying while developing. Usually I don't use cache:warmup, but the automatic cache renewal of symfony in dev mode is also very slow.
It is on a M1 Pro Macbook. PHP 8.2 and apache via brew.

After these changes, what percentage of your cache:warmup is from the pre-lexer?

With the change in consume() it went down from 15s to 9s.
With the str_contains check, the pre-lexer is eliminated from blackfire profile. (I don't use the twig html syntax so far, so all of my templates use the fast abort via str_contains check).

This is the profile with both improvements: https://blackfire.io/profiles/50cca928-b5ca-4b0d-9064-7f698727a932/graph

I really don't know why it is so slow on my machine. But I think it is not a problem of symfony-ux. I need to analyze this further.

[smnandre]

Do you have many anonymous components ? Or are they in majority class-based ?