Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

readme: add comparision table #9

Merged
merged 8 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 26 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,32 @@

`shipmonk/doctrine-entity-preloader` is a PHP library designed to tackle the n+1 query problem in Doctrine ORM by efficiently preloading related entities. This library offers a flexible and powerful way to optimize database access patterns, especially in cases with complex entity relationships.

- 🚀 **Performance Boost:** Minimizes n+1 issues by preloading related entities with **constant number of queries**.
- 🔄 **Flexible:** Supports all associations: `#[OneToOne]`, `#[OneToMany]`, `#[ManyToOne]`, and `#[ManyToMany]`.
- 💡 **Easy Integration:** Simple to integrate with your existing Doctrine setup.
- :rocket: **Performance Boost:** Minimizes n+1 issues by preloading related entities with **constant number of queries**.
- :arrows_counterclockwise: **Flexible:** Supports all associations: `#[OneToOne]`, `#[OneToMany]`, `#[ManyToOne]`, and `#[ManyToMany]`.
- :bulb: **Easy Integration:** Simple to integrate with your existing Doctrine setup.


## Comparison

| | [Default](https://docs.google.com/presentation/d/1sSlZOxmEUVKt0l8zhimex-6lR0ilC001GXh8colaXxg/edit#slide=id.g30998e74a82_0_0) | [Manual Preload](https://docs.google.com/presentation/d/1sSlZOxmEUVKt0l8zhimex-6lR0ilC001GXh8colaXxg/edit#slide=id.g309b68062f4_0_0) | [Fetch Join](https://docs.google.com/presentation/d/1sSlZOxmEUVKt0l8zhimex-6lR0ilC001GXh8colaXxg/edit#slide=id.g309b68062f4_0_15) | [setFetchMode](https://docs.google.com/presentation/d/1sSlZOxmEUVKt0l8zhimex-6lR0ilC001GXh8colaXxg/edit#slide=id.g309b68062f4_0_35) | [**EntityPreloader**](https://docs.google.com/presentation/d/1sSlZOxmEUVKt0l8zhimex-6lR0ilC001GXh8colaXxg/edit#slide=id.g309b68062f4_0_265) |
|------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| [OneToMany](tests/EntityPreloadBlogOneHasManyTest.php) | :red_circle: 1 + n | :green_circle: 1 + 1 | :orange_circle: 1, but duplicates rows | :green_circle: 1 + 1 | :green_circle: 1 + 1 |
| [OneToManyDeep](tests/EntityPreloadBlogOneHasManyDeepTest.php) | :red_circle: 1 + n + n² | :green_circle: 1 + 1 + 1 | :orange_circle: 1, but duplicates rows | :red_circle: 1 + 1 + n² | :green_circle: 1 + 1 + 1 |
| [OneToManyAbstract](tests/EntityPreloadBlogOneHasManyAbstractTest.php) | :red_circle: 1 + n + n² | :orange_circle: 1 + 1 + 1, but duplicates rows | :orange_circle: 1, but duplicates rows | :red_circle: 1 + 1 + n² | :orange_circle: 1 + 1 + 1, but duplicates rows |
| [ManyToOne](tests/EntityPreloadBlogManyHasOneTest.php) | :red_circle: 1 + n | :green_circle: 1 + 1 | :orange_circle: 1, but duplicates rows | :green_circle: 1 + 1 | :green_circle: 1 + 1 |
| [ManyToOneDeep](tests/EntityPreloadBlogManyHasOneDeepTest.php) | :red_circle: 1 + n + n | :green_circle: 1 + 1 + 1 | :orange_circle: 1, but duplicates rows | :red_circle: 1 + 1 + n | :green_circle: 1 + 1 + 1 |
| [ManyToMany](tests/EntityPreloadBlogManyHasManyTest.php) | :red_circle: 1 + n | :green_circle: 1 + 1 | :orange_circle: 1, but duplicates rows | :red_circle: 1 + n | :green_circle: 1 + 1 |

Unlike manual preload does not require writing custom queries for each association.

Unlike fetch joins, the EntityPreloader does not fetches duplicate data, which slows down both the query and the hydration process, except when necessary to prevent additional queries fired by Doctrine during hydration process.

Unlike `Doctrine\ORM\AbstractQuery::setFetchMode` it can

* preload nested associations
* preload `#[ManyToMany]` association
* avoid additional queries fired by Doctrine during hydration process


## Installation

Expand Down Expand Up @@ -44,18 +67,6 @@ foreach ($categories as $category) {
}
```

## Comparison vs. Fetch Joins

Unlike fetch joins, the EntityPreloader does not fetches duplicate data, which slows down both the query and the hydration process, except when necessary to prevent additional queries fired by Doctrine during hydration process.

## Comparison vs. `Doctrine\ORM\AbstractQuery::setFetchMode`

Unlike `setFetchMode` it can

* preload nested associations
* preload `#[ManyToMany]` association
* avoid additional queries fired by Doctrine during hydration process

## Configuration

`EntityPreloader` allows you to adjust batch sizes and fetch join limits to fit your application's performance needs:
Expand Down
24 changes: 24 additions & 0 deletions tests/EntityPreloadBlogManyHasManyTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,30 @@ public function testManyHasManyUnoptimized(): void
]);
}

public function testOneHasManyWithWithManualPreloadUsingPartial(): void
{
$this->skipIfPartialEntitiesAreNotSupported();
$this->createDummyBlogData(articleInEachCategoryCount: 5, tagForEachArticleCount: 5);

$articles = $this->getEntityManager()->getRepository(Article::class)->findAll();

$this->getEntityManager()->createQueryBuilder()
->select('PARTIAL article.{id}', 'tag')
->from(Article::class, 'article')
->leftJoin('article.tags', 'tag')
->where('article IN (:articles)')
->setParameter('articles', $articles)
->getQuery()
->getResult();

$this->readTagLabels($articles);

self::assertAggregatedQueries([
['count' => 1, 'query' => 'SELECT * FROM article t0'],
['count' => 1, 'query' => 'SELECT * FROM article a0_ LEFT JOIN article_tag a2_ ON a0_.id = a2_.article_id LEFT JOIN tag t1_ ON t1_.id = a2_.tag_id WHERE a0_.id IN (?, ?, ?, ?, ?)'],
]);
}

public function testManyHasManyWithFetchJoin(): void
{
$this->createDummyBlogData(articleInEachCategoryCount: 5, tagForEachArticleCount: 5);
Expand Down
45 changes: 44 additions & 1 deletion tests/EntityPreloadBlogOneHasManyDeepTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
use Doctrine\ORM\Mapping\ClassMetadata;
use ShipMonkTests\DoctrineEntityPreloader\Fixtures\Blog\Category;
use ShipMonkTests\DoctrineEntityPreloader\Lib\TestCase;
use function array_map;
use function array_merge;

class EntityPreloadBlogOneHasManyDeepTest extends TestCase
{
Expand All @@ -28,6 +30,46 @@ public function testOneHasManyDeepUnoptimized(): void
]);
}

public function testOneHasManyDeepWithWithManualPreloadUsingPartial(): void
{
$this->skipIfPartialEntitiesAreNotSupported();
$this->createCategoryTree(depth: 5, branchingFactor: 5);

$rootCategories = $this->getEntityManager()->createQueryBuilder()
->select('category')
->from(Category::class, 'category')
->where('category.parent IS NULL')
->getQuery()
->getResult();

$this->getEntityManager()->createQueryBuilder()
->select('PARTIAL category.{id}', 'subCategory')
->from(Category::class, 'category')
->leftJoin('category.children', 'subCategory')
->where('category IN (:categories)')
->setParameter('categories', $rootCategories)
->getQuery()
->getResult();

$subCategories = array_merge(...array_map(static fn(Category $category) => $category->getChildren()->toArray(), $rootCategories));
$this->getEntityManager()->createQueryBuilder()
->select('PARTIAL subCategory.{id}', 'subSubCategory')
->from(Category::class, 'subCategory')
->leftJoin('subCategory.children', 'subSubCategory')
->where('subCategory IN (:subCategories)')
->setParameter('subCategories', $subCategories)
->getQuery()
->getResult();

$this->readSubSubCategoriesNames($rootCategories);

self::assertAggregatedQueries([
['count' => 1, 'query' => 'SELECT * FROM category c0_ WHERE c0_.parent_id IS NULL'],
['count' => 1, 'query' => 'SELECT * FROM category c0_ LEFT JOIN category c1_ ON c0_.id = c1_.parent_id WHERE c0_.id IN (?, ?, ?, ?, ?)'],
['count' => 1, 'query' => 'SELECT * FROM category c0_ LEFT JOIN category c1_ ON c0_.id = c1_.parent_id WHERE c0_.id IN (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'],
]);
}

public function testOneHasManyDeepWithFetchJoin(): void
{
$this->createCategoryTree(depth: 5, branchingFactor: 5);
Expand All @@ -37,13 +79,14 @@ public function testOneHasManyDeepWithFetchJoin(): void
->from(Category::class, 'category')
->leftJoin('category.children', 'subCategories')
->leftJoin('subCategories.children', 'subSubCategories')
->where('category.parent IS NULL')
->getQuery()
->getResult();

$this->readSubSubCategoriesNames($rootCategories);

self::assertAggregatedQueries([
['count' => 1, 'query' => 'SELECT * FROM category c0_ LEFT JOIN category c1_ ON c0_.id = c1_.parent_id LEFT JOIN category c2_ ON c1_.id = c2_.parent_id'],
['count' => 1, 'query' => 'SELECT * FROM category c0_ LEFT JOIN category c1_ ON c0_.id = c1_.parent_id LEFT JOIN category c2_ ON c1_.id = c2_.parent_id WHERE c0_.parent_id IS NULL'],
]);
}

Expand Down
28 changes: 11 additions & 17 deletions tests/EntityPreloadBlogOneHasManyTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,10 @@

namespace ShipMonkTests\DoctrineEntityPreloader;

use Composer\InstalledVersions;
use Doctrine\ORM\Mapping\ClassMetadata;
use Doctrine\ORM\Query\QueryException;
use ShipMonkTests\DoctrineEntityPreloader\Fixtures\Blog\Article;
use ShipMonkTests\DoctrineEntityPreloader\Fixtures\Blog\Category;
use ShipMonkTests\DoctrineEntityPreloader\Lib\TestCase;
use function str_starts_with;

class EntityPreloadBlogOneHasManyTest extends TestCase
{
Expand Down Expand Up @@ -52,31 +49,26 @@ public function testOneHasManyWithWithManualPreload(): void

public function testOneHasManyWithWithManualPreloadUsingPartial(): void
{
$this->skipIfPartialEntitiesAreNotSupported();
$this->createDummyBlogData(categoryCount: 5, articleInEachCategoryCount: 5);

$categories = $this->getEntityManager()->getRepository(Category::class)->findAll();

$query = $this->getEntityManager()->createQueryBuilder()
$this->getEntityManager()->createQueryBuilder()
->select('PARTIAL category.{id}', 'article')
->from(Category::class, 'category')
->leftJoin('category.articles', 'article')
->where('category IN (:categories)')
->setParameter('categories', $categories)
->getQuery();

if (str_starts_with(InstalledVersions::getVersion('doctrine/orm') ?? 'unknown', '3.')) {
self::assertException(QueryException::class, null, static fn() => $query->getResult());

} else {
$query->getResult();
->getQuery()
->getResult();

$this->readArticleTitles($categories);
$this->readArticleTitles($categories);

self::assertAggregatedQueries([
['count' => 1, 'query' => 'SELECT * FROM category t0'],
['count' => 1, 'query' => 'SELECT * FROM category c0_ LEFT JOIN article a1_ ON c0_.id = a1_.category_id WHERE c0_.id IN (?, ?, ?, ?, ?)'],
]);
}
self::assertAggregatedQueries([
['count' => 1, 'query' => 'SELECT * FROM category t0'],
['count' => 1, 'query' => 'SELECT * FROM category c0_ LEFT JOIN article a1_ ON c0_.id = a1_.category_id WHERE c0_.id IN (?, ?, ?, ?, ?)'],
]);
}

public function testOneHasManyWithFetchJoin(): void
Expand Down Expand Up @@ -137,6 +129,8 @@ public function testOneHasManyWithPreload(): void
private function readArticleTitles(array $categories): void
{
foreach ($categories as $category) {
$category->getName();

foreach ($category->getArticles() as $article) {
$article->getTitle();
}
Expand Down
11 changes: 11 additions & 0 deletions tests/Lib/TestCase.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

namespace ShipMonkTests\DoctrineEntityPreloader\Lib;

use Composer\InstalledVersions;
use Doctrine\DBAL\DriverManager;
use Doctrine\DBAL\Logging\Middleware;
use Doctrine\ORM\EntityManager;
Expand All @@ -23,6 +24,7 @@
use ShipMonkTests\DoctrineEntityPreloader\Fixtures\Blog\User;
use Throwable;
use function unlink;
use function version_compare;

abstract class TestCase extends PhpUnitTestCase
{
Expand Down Expand Up @@ -175,6 +177,15 @@ protected function refreshExistingEntity(object $entity): object
return $freshEntity;
}

protected function skipIfPartialEntitiesAreNotSupported(): void
{
$ormVersion = InstalledVersions::getVersion('doctrine/orm') ?? '0.0.0';

if (version_compare($ormVersion, '3.0.0', '>=') && version_compare($ormVersion, '3.3.0', '<')) {
self::markTestSkipped('Partial entities are not supported in Doctrine ORM versions 3.0 to 3.2');
}
}

protected function getQueryLogger(): QueryLogger
{
return $this->queryLogger ??= $this->createQueryLogger();
Expand Down