diff --git a/docs/README.md b/docs/README.md index 8840454b644b3..655eae801030b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -216,6 +216,14 @@ You can have a more dynamic visualization of a terminal output by using the foll {% terminal-video src="/documentation/shared/images/caching/cache-terminal-animation.mp4" /%} ``` +#### Table of Contents + +You can add a table of contents to your document by using the following component. This is mostly useful for blog posts. + +```markdown +{% toc /%} +``` + #### Custom iframes We can display a special iframe and setting its width inside the document. diff --git a/docs/blog/2024-12-22-nx-highlights-2024.md b/docs/blog/2024-12-22-nx-highlights-2024.md new file mode 100644 index 0000000000000..a66d9db401f57 --- /dev/null +++ b/docs/blog/2024-12-22-nx-highlights-2024.md @@ -0,0 +1,280 @@ +--- +title: 'Nx highlights 2024' +slug: nx-highlights-2024 +authors: ['Juri Strumpflohner', 'Victor Savkin'] +tags: ['nx'] +cover_image: +--- + +Thank you for being part of the Nx community. Your support and contributions make Nx what it is today—driving it forward as the go-to solution for delivering the best developer experience in monorepos. Nx and Nx Cloud integrate to provide a **complete solution for managing monorepos** at every stage of the development cycle, both locally and in CI. + +In 2024, we focused on making Nx **more adaptable**—seamlessly fitting into OSS projects and large enterprise codebases. Our goal was to balance **flexibility and power**, ensuring Nx scales with your needs while remaining easy to adopt in **existing setups**. This progress is the result of **working with the community, collaborating with other tooling teams, and engaging directly with enterprises**. + +{% github-repository title="Star Nx on GitHub" url="https://github.com/nrwl/nx" /%} + +As we wrap up the year, we’re excited to share the highlights and give you a look at what’s coming in 2025. + +{% toc /%} + +## Top 10 Highlights + +Here are our picks for the top 10 highlights for 2024. There's no particular ordering here in terms of more or less relevance. + +### Project Crystal + +Nx has been solving monorepos long before package managers introduced features like workspaces. Early on, we used **TypeScript path mappings** to manage project linking, resulting in **integrated monorepos**—powered by [Nx plugins](/concepts/nx-plugins#what-are-nx-plugins) to automate setup and tool integration. + +When **npm**, **Yarn**, and **PNPM** introduced "workspaces," another option emerged: **package-based monorepos**, where Nx primarily acted as a task runner with caching and parallelization. + +Integrated monorepos offer automation but are opinionated. Package-based monorepos provide flexibility but require more effort to manage. Combining the two wasn’t easy, largely because of how Nx plugins were initially designed. + +We wanted to make sure you didn’t have to choose. This led to **Project Crystal**—an effort to make Nx plugins more adaptable. These "crystalized" plugins: + +- **Rely on your existing config files** (e.g., `@nx/vite` enhances `vite.config.ts` rather than replacing it). +- **Enhance without interfering**, allowing for a better experience without locking you into the plugin. + +Over the past year, most of our plugins have been "crystalized," making it easier to mix and match both monorepo styles. + +![project-crystal-plugins.avif](/blog/images/articles/project-crystal-plugins.avif) + +This approach gives you flexibility while keeping the automation and developer experience that make Nx essential for scaling monorepos. + +Want to dive deeper? Check out [our announcement blog post](/blog/what-if-nx-plugins-were-more-like-vscode-extensions) and [documentation](/concepts/inferred-tasks). + +### Nx Agents, Atomizer, and Flakiness Detection + +![agents-atomizer-flaky-viz.avif](/blog/images/articles/agents-atomizer-flaky-viz.avif) + +Nx isn’t just about managing monorepos locally. A big part of the workflow happens on CI. If your monorepo doesn’t run reliably in CI, it’s not working. That’s why Nx Cloud has been a major focus—especially for scaling monorepos in CI environments. This year, we introduced some of the most impactful features yet: + +- [Nx Agents](/ci/features/distribute-task-execution) +- [Atomizer](/ci/features/split-e2e-tasks) +- [Flakiness detection](/ci/features/flaky-tasks) + +With remote caching as the foundation, these features take CI performance to the next level. At scale, **parallelizing tasks across machines becomes necessary**. We've done this manually for clients in the past, scripting complex CI setups. A lot of that experience shaped Nx Agents, making distribution seamless and efficient. + +Manually configuring parallelization is time-consuming and fragile. It’s a static optimization, while your monorepo keeps changing. We wrote a [blog post](/blog/reliable-ci-a-new-execution-model-fixing-both-flakiness-and-slowness) that explains the difference between the traditional **push-based model** and Nx’s **pull-based approach** to CI. + +![nx-agents-run-viz.avif](/blog/images/articles/nx-agents-run-viz.avif) + +If you haven’t tried Nx Agents yet, enabling them takes a single line in your CI setup: + +```yaml +npx nx-cloud start-ci-run --distribute-on="3 linux-medium-js" +``` + +Check out [the docs](/ci/features/distribute-task-execution) or watch our [free video course](/courses/pnpm-nx-next/lessons-11-nx-agents) where we update an existing CI pipeline to use Nx Agents. + +Even with distribution in place, **long-running tasks like e2e tests** can still slow things down. **Atomizer automatically breaks these into smaller pieces** that can run in parallel, making task distribution much more efficient. + +![](/courses/pnpm-nx-next/images/e2e-splitting-anim.gif) + +Finally, we introduced **flakiness detection** across all CI task runs. Flaky tasks (usually tests) waste a lot of time. Instead of forcing you to re-run CI manually, Nx Cloud retries flaky tasks automatically. If the task passes on the second run, the pipeline continues without interruption. + +![flaky-task-retry.avif](/blog/images/articles/flaky-task-retry.avif) +Finally, we introduced flakiness detection across all CI task runs. Flaky tasks (usually tests) waste a lot of time. Instead of forcing you to re-run CI manually, Nx Cloud retries flaky tasks automatically. If the task passes on the second run, the pipeline continues without interruption. + +![flaky-task-retry.avif](/blog/images/articles/flaky-task-retry.avif) + +### Nx Release + +A big use case for Nx isn’t just managing application monorepos but also developing and publishing packages to NPM. One missing piece was a built-in way to handle versioning, changelog generation, and publishing. While tools like [Lerna](https://lerna.js.org) or [Changesets](https://github.com/changesets/changesets) could be integrated, many wanted something more tightly connected to Nx itself. + +That’s why we released [**Nx Release**](/features/manage-releases). It handles the entire process: + +- Bumping versions +- Generating changelogs +- Publishing to NPM or other registries + +Nx Release is designed to be flexible. It comes with a [programmatic API](/features/manage-releases#using-the-programmatic-api-for-nx-release) and is implemented in a tech-agnostic way. Nx provides the foundation, and plugins like `@nx/js` extend it to support JavaScript and TypeScript packages. The same system can be extended for Go, Rust, Java, and more. + +In the future, Nx Release could grow to handle other release processes, like Docker containers or application deployments. + +Take a look at [our docs](/features/manage-releases) or browse [Nx Release recipes](/recipes/nx-release) to see how you can adapt it to fit your workflow. + +### Gradle Support + +2024 was a big step forward with the release of the **@nx/gradle plugin**. This plugin lets you manage Gradle projects within an Nx workspace, bringing features like task caching, parallel execution, and dependency-aware task orchestration to Java and Kotlin projects. + +Gradle projects were traditionally tricky to manage alongside JavaScript libraries in monorepos. Few tools could handle mixed tech stacks smoothly. With **@nx/gradle**, that process is much simpler. Gradle projects now work seamlessly inside Nx, allowing you to: + +- **Run Gradle tasks directly with Nx commands** (e.g., `nx build myproject`) +- **Visualize dependencies across projects**, including composite builds, with the Nx Graph +- **Only execute tasks that are affected by code changes**, speeding up builds and CI pipelines + +**Composite Build Support** takes this further by pulling in dependencies from composite builds and reflecting them in the task graph. This release reflects our larger goal of breaking down barriers between ecosystems. Nx aims to enable efficient workflows across different languages and technologies, moving closer to a seamless polyglot monorepo experience. + +For more details, check out the blog post ["Manage Your Gradle Project using Nx"](/blog/manage-your-gradle-project-using-nx) and our [Gradle tutorial](/getting-started/tutorials/gradle-tutorial). + +### Nx Import + +Migrating projects into a monorepo is often seen as complicated and error-prone. With **`nx import`**, introduced in 2024, we've simplified the process—preserving **Git history** and minimizing disruptions along the way. + +`nx import` builds on the work done in **Project Crystal**, making migrations smoother. For example, when importing a Gradle-based app, Nx detects the Gradle configuration, suggests installing the `@nx/gradle` plugin, and sets everything up for you. It reads your Gradle files, identifies runnable tasks, configures caching, and integrates the project into the workspace. + +The experience is the same whether you’re importing a JavaScript app, a Gradle project, or another tech stack. + +With `nx import`, you can: + +- Map existing repositories into an Nx workspace (while preserving Git history) +- Retain existing configurations and workflows without losing functionality +- Get consistent orchestration, caching, and dependency graphing across projects + +This significantly reduces the effort involved in **polyrepo-to-monorepo migrations**, allowing teams to consolidate projects without downtime or loss of productivity. What used to be a hassle is now a streamlined process that works across different technologies. + +Check out the [docs](/recipes/adopting-nx/import-project) for more details. + +### GitHub Managed Orgs in Nx Cloud + +![github-org-settings.avif](/blog/images/articles/github-org-settings.avif) + +One of our goals with Nx Cloud is to make onboarding as simple as possible. We want to improve CI performance without adding unnecessary configuration or complexity. This year, we introduced **GitHub-powered organizations** to streamline team management and onboarding. + +By connecting your GitHub organization, you can select repositories directly. Nx Cloud will automatically submit a PR to configure your workspace and handle GitHub integration. + +![nx-cloud-connect-gh-repo.avif](/blog/images/articles/nx-cloud-connect-gh-repo.avif) + +A major benefit is that user management syncs directly with GitHub. When someone links their GitHub account to Nx Cloud, they’re automatically added to the correct organization. Membership changes in GitHub are reflected in Nx Cloud without manual intervention. + +Learn more about how it works [here](/ci/intro/connect-to-nx-cloud). + +### Module Federation + +In 2024, we continued expanding Nx's **Module Federation** support, improving functionality and making it easier to use based on feedback from the community. + +One major update was the new pattern for the **module-federation-dev-server**. It improves compatibility, reduces resource usage, and provides a smoother local development experience. This approach makes working with Module Federation in Nx easier to set up and scale across larger workspaces. + +We also adopted **Module Federation 2.0**, adding runtime plugins and key enhancements. On top of that, we introduced support for **Rspack Module Federation**, allowing Nx users to take advantage of newer tools in the ecosystem. + +To simplify setup, we launched **@nx/module-federation**—a package that streamlines configuration and management of federated apps in Nx workspaces. Nx’s approach to Module Federation has been highlighted by the community and is now featured on [module-federation.io](https://module-federation.io/practice/monorepos/nx-for-module-federation.html). + +For more details on how Module Federation works in Nx, check out the [technical overview](/concepts/module-federation/nx-module-federation-technical-overview) or catch one of our [live streams on Rspack and Module Federation](https://www.youtube.com/watch?v=_c4zjYm0pYE). + +### Nx Console + +Nx Console has become a key part of the "Nx experience." In 2024, we introduced several updates to make it even more useful. + +One standout feature is the **Project Detail View**. With Project Crystal, Nx can dynamically infer targets from your configuration. You can see these [inferred tasks](/concepts/inferred-tasks) by running `nx show project my-project --web`, but now they’re directly available inside Nx Console. This gives you quick access to project details, command options, caching configurations, and more—right in your editor. + +{% video-player src="/documentation/blog/media/nxconsole-project-view.mp4" alt="Nx Console Project Detail View" /%} + +Another big addition is **CI Pipeline Visualization** for Nx Cloud. + +{% video-player src="/documentation/blog/media/nx-console-pipeline-running.mp4" alt="Nx Console CI Pipeline Execution" /%} + +Nx Console notifies you inside your IDE when pipelines succeed or fail. This removes the need to monitor CI logs or constantly switch between tools. + +{% video-player src="/documentation/blog/media/nxconsole-ci-completion.mp4" alt="Nx Console CI Completion" /%} + +These updates make CI more accessible and help you stay focused without leaving your development environment. + +If you haven’t tried it yet, grab Nx Console for [VSCode](https://marketplace.visualstudio.com/items?itemName=nrwl.angular-console) or [JetBrains IDEs](https://plugins.jetbrains.com/plugin/21060-nx-console). + +### Affected Project Graph in Nx Cloud + +Understanding how changes impact your workspace is key to optimizing and debugging CI pipelines. This year, we introduced the **Affected Project Graph** in Nx Cloud to make this easier. + +![affected-project-graph-nxcloud.avif](/blog/images/articles/affected-project-graph-nxcloud.avif) + +The Affected Project Graph builds on existing tools like [cache miss troubleshooting](/troubleshooting/troubleshoot-cache-misses#check-3-use-the-nx-cloud-troubleshooting-tools), helping you track down what triggered a cache reset. Now, you can visually map which projects are affected by changes, providing clearer insights into your pipeline. + +This feature lets you: + +- **Debug pipelines faster** by seeing why tasks were triggered +- **Spot bottlenecks** in dependency chains +- Use insights to optimize your workspace structure + +For a deeper look at this feature, check out our blog post: [Understand the CI Affected Graph](/blog/ci-affected-graph). Combined with other Nx Cloud tools, the Affected Project Graph makes it easier to fine-tune CI performance and manage complex workspaces. + +### Local Caching Moves to DB Storage + +Speed is always a priority at Nx. This year, we transitioned **local caching** from a file-based system to a **database-backed approach**. This shift reduces I/O overhead, improving performance and laying the groundwork for smarter cache management. + +With database-backed caching, we can introduce features like: + +- Retaining only the most relevant cache entries based on usage +- Setting cache size limits +- Prioritizing failed tasks earlier in the queue + +This change not only boosts performance but also opens up opportunities for more efficient task orchestration. The move to database storage is part of a broader effort to rewrite critical parts of Nx in **Rust**, pushing for better performance and scalability. + +## Coming up in 2025 + +We have already a lot of exciting features that we want to ship next year. Here are some of them we're looking forward to. + +### Local Agents + +Nx Cloud already helps balance **speed and cost** by dynamically scaling agents based on PR size and workload. This ensures you only use the resources you need, keeping CI fast without unnecessary expense. + +But there’s still untapped potential. **Local Agents** will let you connect your developer machine to Nx Cloud, allowing it to handle tasks when it’s underutilized. This reduces the need for extra CI agents while making better use of the machines you already have—without disrupting your workflow. + +We have a proof-of-concept in place, and we're working to turn this into a full feature. + +### Polygraph + +Many teams manage **multiple monorepos or polyrepos** instead of consolidating everything into a single large repository. This often happens for practical reasons or as part of a gradual transition. + +**Polygraph** gives teams some of the benefits of a larger monorepo—like consistency and visibility—**without needing to merge everything at once**. Even if a monorepo is the end goal, the goal of Polygraph is to help teams make progress early, rather than waiting until everything is in one place. + +This is especially useful for **platform teams** responsible for maintaining consistency, security, and best practices across many repositories. Polygraph provides better control by allowing you to: + +- Run **conformance rules** across all repositories in your Nx Cloud organization +- Visualize **dependencies** between projects in different repositories using a multi-repo graph + +Stay tuned as we keep working on this and releasing it in 2025. + +### Rewriting Nx Core in Rust + +We’ve already rewritten performance-critical parts of Nx in **Rust**, and in 2025 we plan to extend this to more of the core. One of the main goals is to ensure that Nx adds as little overhead as possible to your monorepo, keeping things fast and efficient at scale. + +This rewrite aligns with Nx’s goal of being **tech agnostic**, supporting polyglot monorepos and workflows beyond JavaScript. By reducing reliance on Node.js for the core, Nx can fit into a wider range of environments, while plugins will continue to be written in **TypeScript** to keep development familiar and extensible. + +By moving core parts to Rust, we’re focusing on performance where it matters most, while maintaining the flexibility that makes Nx useful across different tech stacks. + +### Support for Long-Running Tasks + +Nx is built around tasks that start and finish, but that doesn’t always fit workflows involving **dev servers, watch-mode builds, or background services**. These tasks run indefinitely, which can be tricky to manage—especially when other processes depend on them, like e2e tests that require a server to stay up. + +We’re working on adding support for **long-running tasks** that can run alongside regular build and test processes. This will make workflows smoother and help avoid the need for manual orchestration. The upcoming **Nx terminal UI rewrite** will reflect this, making it easier to track and visualize long-running processes alongside other tasks (more about that later in the article). + +If you’re curious about the direction we’re heading, take a look at the ongoing [RFC](https://github.com/nrwl/nx/discussions/29025). + +### Unified Monorepo Support + +Nx already works well with both **integrated monorepos** driven by Nx plugins and **package-based monorepos** using npm, Yarn, or PNPM workspaces. Where things get trickier is mixing the two approaches—integrated plugins didn’t always align smoothly with workspace-based setups. + +With **Nx Crystal Plugins**, we’re making it easier to blend these approaches. You get the automation and guardrails of Nx plugins while retaining the flexibility of workspace-based monorepos. This lets you enhance a PNPM, npm, or Yarn workspace with Nx’s powerful task orchestration, caching, and dependency management—without fully committing to an integrated setup. + +A great example of this is our [Next.js + Nx + PNPM course](https://nx.dev/courses/pnpm-nx-next), where we show how to enhance a PNPM workspace with Nx. But this is just the beginning. Our goal is to make the experience even better, reducing friction and making it seamless to mix and match both models. + +Also note: existing workspaces won’t be affected. Integrated monorepos will continue to work as they always have, but new projects will default to this more flexible model. + +You can read more about this direction in the [RFC](https://github.com/nrwl/nx/discussions/29099). + +### Nx Terminal Redesign + +The terminal is one of the main ways developers interact with Nx, and we’re working on a complete redesign to make it nicer, clearer, and easier to use. + +The goal is to reduce noise and surface the most important information for each task. Regardless whether you're running multiple dev servers, long-running tasks, or parallel builds. + +This redesign also ties into our work on **long-running tasks**. Processes like dev servers or background services will be easier to manage directly from the terminal, letting you see their status alongside regular build and test processes. + +We want the terminal to feel more intuitive and stay out of your way—while giving you the details you need when you need them. + +### Improved Node.js Experience + +The goal is to bring the same level of polish to Node development that Nx already provides for frontend frameworks. This includes **better tools for managing microservices**, **streamlining Docker** builds, and improving backend workflows in monorepos. + +We want Nx to feel just as natural for scaling Node applications as it does for frontend projects. + +### Expanding Polyglot Support + +Nx has grown beyond JavaScript, with support for tools across different ecosystems. In 2024, we introduced **Gradle support** to simplify Java app management in monorepos. + +Next, we’re expanding to **Maven** and **.NET**, continuing to break down barriers for teams managing projects in multiple languages. The goal is to make Nx a natural fit for polyglot monorepos, keeping workflows fast and consistent across tech stacks. + +## Excited for 2025? + +We are too. As we wrap up 2024, we want to thank you for being part of the Nx community. Your feedback and contributions continue to shape Nx and push it forward. + +We hope you enjoy the holidays and, if you're celebrating, Merry Christmas! Looking forward to seeing what we build in 2025. diff --git a/docs/blog/images/articles/Pasted image 20241222221313.avif b/docs/blog/images/articles/Pasted image 20241222221313.avif new file mode 100644 index 0000000000000..28e87866ae641 Binary files /dev/null and b/docs/blog/images/articles/Pasted image 20241222221313.avif differ diff --git a/docs/blog/images/articles/Pasted image 20241222222141.avif b/docs/blog/images/articles/Pasted image 20241222222141.avif new file mode 100644 index 0000000000000..de42a76f8a0f3 Binary files /dev/null and b/docs/blog/images/articles/Pasted image 20241222222141.avif differ diff --git a/docs/blog/images/articles/affected-project-graph-nxcloud.avif b/docs/blog/images/articles/affected-project-graph-nxcloud.avif new file mode 100644 index 0000000000000..edf025e211071 Binary files /dev/null and b/docs/blog/images/articles/affected-project-graph-nxcloud.avif differ diff --git a/docs/blog/images/articles/agents-atomizer-flaky-viz.avif b/docs/blog/images/articles/agents-atomizer-flaky-viz.avif new file mode 100644 index 0000000000000..e6050125fc28d Binary files /dev/null and b/docs/blog/images/articles/agents-atomizer-flaky-viz.avif differ diff --git a/docs/blog/images/articles/crystal-plugins-npm-workspace.avif b/docs/blog/images/articles/crystal-plugins-npm-workspace.avif new file mode 100644 index 0000000000000..db5dfa4511454 Binary files /dev/null and b/docs/blog/images/articles/crystal-plugins-npm-workspace.avif differ diff --git a/docs/blog/images/articles/flaky-task-retry.avif b/docs/blog/images/articles/flaky-task-retry.avif new file mode 100644 index 0000000000000..f0caf30984dbd Binary files /dev/null and b/docs/blog/images/articles/flaky-task-retry.avif differ diff --git a/docs/blog/images/articles/github-org-settings.avif b/docs/blog/images/articles/github-org-settings.avif new file mode 100644 index 0000000000000..28e87866ae641 Binary files /dev/null and b/docs/blog/images/articles/github-org-settings.avif differ diff --git a/docs/blog/images/articles/nx-agents-run-viz.avif b/docs/blog/images/articles/nx-agents-run-viz.avif new file mode 100644 index 0000000000000..306114af5dfcb Binary files /dev/null and b/docs/blog/images/articles/nx-agents-run-viz.avif differ diff --git a/docs/blog/images/articles/nx-cloud-connect-gh-repo.avif b/docs/blog/images/articles/nx-cloud-connect-gh-repo.avif new file mode 100644 index 0000000000000..de42a76f8a0f3 Binary files /dev/null and b/docs/blog/images/articles/nx-cloud-connect-gh-repo.avif differ diff --git a/docs/blog/images/articles/project-crystal-plugins.avif b/docs/blog/images/articles/project-crystal-plugins.avif new file mode 100644 index 0000000000000..3cb5161c7b0e1 Binary files /dev/null and b/docs/blog/images/articles/project-crystal-plugins.avif differ diff --git a/docs/blog/media/nx-console-pipeline-running.mp4 b/docs/blog/media/nx-console-pipeline-running.mp4 new file mode 100644 index 0000000000000..030aaa96069fc Binary files /dev/null and b/docs/blog/media/nx-console-pipeline-running.mp4 differ diff --git a/docs/blog/media/nxconsole-ci-completion.mp4 b/docs/blog/media/nxconsole-ci-completion.mp4 new file mode 100644 index 0000000000000..9a40de49312a4 Binary files /dev/null and b/docs/blog/media/nxconsole-ci-completion.mp4 differ diff --git a/docs/blog/media/nxconsole-project-view.mp4 b/docs/blog/media/nxconsole-project-view.mp4 new file mode 100644 index 0000000000000..7accc5c6cf485 Binary files /dev/null and b/docs/blog/media/nxconsole-project-view.mp4 differ