Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: split some policies by the implicated method #14211

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This is also a good place to share with all of us **why you are submitting this

---

- [ ] I've read the [guidelines](https://github.com/conan-io/conan-center-index/blob/master/docs/adding_packages/README.md) for contributing.
- [ ] I've read the [contributing guidelines](https://github.com/conan-io/conan-center-index/blob/master/CONTRIBUTING.md).
- [ ] I've followed the [PEP8](https://www.python.org/dev/peps/pep-0008/) style guides for Python code in the recipes.
- [ ] I've used the [latest](https://github.com/conan-io/conan/releases/latest) Conan client version.
- [ ] I've tried at least one configuration locally with the [conan-center hook](https://github.com/conan-io/hooks.git) activated.
36 changes: 36 additions & 0 deletions docs/adding_packages/build_and_package.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Build and Package

This document gathers all the relevant information regarding the general lines to follow while writing either the `build()` or the `package()` methods.
Both methods often use build helpers to build binaries and install them into the `package_folder`.

<!-- toc -->
## Contents

* [Build Method](#build-method)
* [Package](#package)<!-- endToc -->

## Build Method

* `build()` method should focus on build only, not installation. During the build, nothing should be written in `package` folder. Installation step should only occur in `package()` method.

* The build itself should only rely on local files. Nothing should be downloaded from the internet during this step. If external files are required, they should come from `requirements` or `build_requirements` in addition to source files downloaded in `source()` or coming from the recipe itself.

* Except for CMake and a working build toolchain (compiler, linker, archiver, etc.), the recipe should not assume that any other build tool is installed on the user-build machine (like Meson, autotools, or pkg-config). On Windows, the recipe should not assume that a shell is available (like MSYS2). Therefore, if the build method requires additional tools, they should be added to `build_requirements()`.

* It is forbidden to run other conan client commands during build. In other words, if upstream build files call conan under the hood (through `cmake-conan` for example or any other logic), this logic must be removed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do I check for that?


* Settings from profile should be honored (`build_type`, `compiler.libcxx`, `compiler.cppstd`, `compiler.runtime` etc).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Settings from profile should be honored (`build_type`, `compiler.libcxx`, `compiler.cppstd`, `compiler.runtime` etc).
* Settings from profile should be honored (`build_type`, 'os', 'arch', `compiler.libcxx`, `compiler.cppstd`, `compiler.runtime` etc).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was intended to call out problematic settings not all of them 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure which are more problematic 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, it deserves an example. for newcomers, it's kinda unclear, how to to it exactly. it would be nice if you illustrate with a small code block, e.g. for CMake.


* These env vars from profile should be honored and properly propagated to underlying build system during the build: `CC`, `CXX`, `CFLAGS`, `CXXFLAGS`, `LDFLAGS`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it deserves an example. for newcomers, it's kinda unclear. can you illustrate how to honor these variables, e.g. for CMake?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the docs are bad - I am not trying to fix them in this PR 😄 I got many more too come!


## Package

* CMake config files must be removed (they will be generated for consumers by `cmake_find_package`, `cmake_find_package_multi`, or `CMakeDeps` generators). Use `rmdir(self, os.path.join(self.package_folder, "lib", "cmake"))` or `rm(self, "*.cmake", os.path.join(self.package_folder, "lib"))`.

* pkg-config files must be removed (they will be generated for consumers by `pkg_config` or `PkgConfigDeps` generators). Use `rmdir(self, os.path.join(self.package_folder, "lib", "pkgconfig"))` or `rm(self, "*.pc", os.path.join(self.package_folder, "lib"))`.

* On *nix systems, executables and shared libraries should have empty `RPATH`/`RUNPATH`/`LC_RPATH`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do I achieve that? how do I ensure that? examples are needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we also need to put some links to the relevant documentation for:

  • RPATH
  • RUNPATH
  • LC_RPATH
  • LC_ID_DYLIB
  • @rpath

maybe with some brief examples on how to inspect them (commands like objdump, readelf, otool, etc.).
these are very technical and low-level things, we cannot expect all the readers to know about them.


* On macOS, install name in `LC_ID_DYLIB` section of shared libs must be `@rpath/<libfilename>`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do I achieve that? how do I ensure that? examples are needed.


* Installed files must not contain absolute paths specific to build machine. Relative paths to other packages is also forbidden since relative paths of dependencies during build may not be the same for consumers. Hardcoded relative paths pointing to a location in the package itself are allowed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do I check for that? how do I prevent/fix that? examples are needed.

Original file line number Diff line number Diff line change
@@ -1,84 +1,17 @@
# Packaging Policy
# `ConanFile` Attributes

This document gathers all the relevant information regarding the general lines to follow while creating new recipes that will eventually be part of this repository.
The `ConanFile` class has a lot of different properties that can help consumers search for projects, help the client build packages for different configurations
or are known by ConanCenter's build service and have special meaning.

<!-- toc -->
## Contents

* [Sources](#sources)
* [Dependencies](#dependencies)
* [Build](#build)
* [Package](#package)
* [Settings](#settings)
* [Options](#options)
* [Recommended Names](#recommended-names)
* [Predefined Options and Known Defaults](#predefined-options-and-known-defaults)
* [Options to Avoid](#options-to-avoid)<!-- endToc -->

## Sources

**Origin of sources:**

* Library sources should come from an official origin like the library source code repository or the official
release/download webpage.

* If an official source archive is available, it should be preferred over an auto-generated archive.

**Source immutability:** Downloaded source code stored under `source` folder should not be modified. Any patch should be applied to the copy of this source code when a build is executed (basically in `build()` method).

**Building from sources:** Recipes should always build packages from library sources.

**Sources not accessible:**

* Library sources that are not publicly available will not be allowed in this repository even if the license allows their redistribution.

* If library sources cannot be downloaded from their official origin or cannot be consumed directly due to their
format, the recommendation is to contact the publisher and ask them to provide the sources in a way/format that can be consumed
programmatically.

* In case of needing those binaries to use them as a "build require" for some library, we will consider following the approach of adding it
as a system recipe (`<build_require>/system`) and making those binaries available in the CI machines (if the license allows it).

## Dependencies

* Version range is not allowed.

* Specify explicit RREV (recipe revision) of dependencies is not allowed.

* Vendoring in library source code should be removed (best effort) to avoid potential ODR violations. If upstream takes care to rename symbols, it may be acceptable.

* Only other conan-center recipes are allowed in `requires`/`requirements()` and `build_requires`/`build_requirements()` of a conan-center recipe.

* If a requirement is conditional, this condition must not depend on build context. Build requirements don't have this constraint.

* Forcing options of dependencies inside a conan-center recipe should be avoided, except if it is mandatory for the library.

## Build

* `build()` method should focus on build only, not installation. During the build, nothing should be written in `package` folder. Installation step should only occur in `package()` method.

* The build itself should only rely on local files. Nothing should be downloaded from internet during this step. If external files are required, they should come from `requirements` or `build_requirements`, in addition to source files downloaded in `source()` or coming from recipe itself.

* Except CMake and a working build toolchain (compiler, linker, archiver etc), the recipe should not assume that any other build tool is installed on user build machine (like Meson, autotools or pkg-config). On Windows, recipe should not assume that a shell is available (like MSYS2). Therefore, if the buid requires additional build tools, they should be added to `build_requirements()`.

* It is forbidden to run other conan client commands during build. In other words, if upstream build files call conan under the hood (through `cmake-conan` for example or any other logic), this logic must be neutralized.

* Settings from profile should be honored (`build_type`, `compiler.libcxx`, `compiler.cppstd`, `compiler.runtime` etc).

* These env vars from profile should be honored and properly propagated to underlying build system during the build: `CC`, `CXX`, `CFLAGS`, `CXXFLAGS`, `LDFLAGS`.

## Package

* CMake config files must be removed (they will be generated for consumers by `cmake_find_package`, `cmake_find_package_multi` or `CMakeDeps` generators).

* pkg-config files must be removed (they will be generated for consumers by `pkg_config` or `PkgConfigDeps` generators).

* On *nix systems, executables and shared libraries should have empty `RPATH`/`RUNPATH`/`LC_RPATH`.

* On macOS, install name in `LC_ID_DYLIB` section of shared libs must be `@rpath/<libfilename>`.

* Installed files must not contain absolute paths specific to build machine. Relative paths to other packages is also forbidden since relative paths of dependencies during build may not be the same for consumers. Hardcoded relative paths pointing to a location in the package itself are allowed.

## Settings

All recipes should list the four settings `os`, `arch`, `compiler` and `build_type` so Conan will compute a different package ID
Expand Down
20 changes: 20 additions & 0 deletions docs/adding_packages/dependencies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Dependencies

This section outlines all the practices and guidelines for the `requirements()` and `build_requirements()` methods. This includes everything from "vendored" dependencies to
when and how the versions could be changed.

<!-- toc -->
## Contents

* [Rules](#rules)<!-- endToc -->

## Rules

* [Version range](https://docs.conan.io/en/latest/versioning/version_ranges.html) is not allowed.
* Specify explicit [RREV](https://docs.conan.io/en/latest/versioning/revisions.html) (recipe revision) of dependencies is not allowed.
* Vendoring in library source code should be removed (best effort) to avoid potential ODR violations. If upstream takes care to rename
symbols, it may be acceptable.
* Only ConanCenter recipes are allowed in `requires`/`requirements()` and `build_requires`/`build_requirements()`.
* If a requirement is conditional, this condition must not depend on [build context](https://docs.conan.io/en/1.35/devtools/build_requires.html#build-and-host-contexts). Build requirements don't have this constraint.
* Forcing options of dependencies inside a recipe should be avoided, except if it is mandatory for the library - in which case it must
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

example would be nice

be enforced through the `validate()` methods.
33 changes: 33 additions & 0 deletions docs/adding_packages/sources_and_patches.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Sources and Patches

This documents contains everything related to the `source()`. This includes picking sources, where they should come from and goes into when and how to modify sources.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This documents contains everything related to the `source()`. This includes picking sources, where they should come from and goes into when and how to modify sources.
This documents contains everything related to the `source()`. This includes picking sources, where they should come from.

in another section, we already stated sources are immutable

These are a very important aspects and it helps us to establish the quality of the packages offered by ConanCenter.

<!-- toc -->
## Contents

* [Sources](#sources)<!-- endToc -->

## Sources

**Origin of sources:**

* Library sources should come from an official origin like the library source code repository or the official
release/download webpage.

* If an official source archive is available, it should be preferred over an auto-generated archive.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should explain it a little bit more, on how to distinguish these two.
e.g., in GitHub projects, prefer archives from the releases tab, rather code - download zip


**Source immutability:** Downloaded source code stored under `source` folder should not be modified. Any patch or `replace_in_file` statement should be applied to the copy of this source code when a build is executed (basically in `build()` method).

**Building from sources:** Recipes should always build packages from library sources.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have some known exceptions to this rule (MSYS2, Android NDK, etc.)


**Sources not accessible:**

* Library sources that are not publicly available will not be allowed in this repository even if the license allows their redistribution.

* If library sources cannot be downloaded from their official origin or cannot be consumed directly due to their
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we had some special cases for that, e.g. for gSOAP. maybe explain it here.

format, the recommendation is to contact the publisher and ask them to provide the sources in a way/format that can be consumed
programmatically.

* In case of needing those binaries to use them as a "build require" for some library, we will consider following the approach of adding it
as a system recipe (`<build_require>/system`) and making those binaries available in the CI machines (if the license allows it).
2 changes: 1 addition & 1 deletion docs/error_knowledge_base.md
Original file line number Diff line number Diff line change
Expand Up @@ -447,7 +447,7 @@ class SomeRecipe(ConanFile):

There is the case when the package is header-only, but the options affects the generated artifact, (e.g. kanguru, pagmo2 ...), so you need to use `self.info.settings.clear()` instead.

- For "tool" recipes ([example](https://github.com/conan-io/conan-center-index/blob/e604534bbe0ef56bdb1f8513b83404eff02aebc8/recipes/cmake/3.x.x/conanfile.py#L104-L105)) which only provide binaries, see [our packing policy](adding_packages/packaging_policy.md#settings) for more, should do as follows:
- For "tool" recipes ([example](https://github.com/conan-io/conan-center-index/blob/e604534bbe0ef56bdb1f8513b83404eff02aebc8/recipes/cmake/3.x.x/conanfile.py#L104-L105)) which only provide binaries, see [our packing policy](adding_packages/conanfile_attributes.md#settings) for more, should do as follows:

```python
def package_id(self):
Expand Down
9 changes: 5 additions & 4 deletions docs/faqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ For example, `GSL` is the name of `Guidelines Support Library` from Microsoft an

## What is the policy on creating packages from pre-compiled binaries?

The policy is that in the general case [recipes should build packages from sources](adding_packages/packaging_policy.md), because of reproducibility and security concerns. The implication is that the sources must be publicly available, and in a format that can be consumed programmatically.
The policy is that in the general case [recipes should build packages from sources](adding_packages/sources_and_patches.md), because of reproducibility and security concerns. The implication is that the sources must be publicly available, and in a format that can be consumed programmatically.

Check the link for further details.

Expand Down Expand Up @@ -132,7 +132,7 @@ We often receive new fixes and improvements to the recipes already available for

## Do static libraries tend to be compiled as PIC by default?

Yes! You can learn more about default options in [Packaging Policy](adding_packages/packaging_policy.md#options).
Yes! You can learn more about default options in [Packaging Policy](adding_packages/conanfile_attributes.md#predefined-options-and-known-defaults).

## Why PDB files are not allowed?

Expand Down Expand Up @@ -254,14 +254,15 @@ in an incompatible Conan package. To deal with those cases, you are allowed to p

<!-- ref https://github.com/conan-io/conan-center-index/pull/5405#issuecomment-854618305 -->

There are different motivations
There are different motivations:

- time and resources: adding the build time required by the test suite plus execution time can increase our building times significantly across the 100+ configurations.
- ConanCenter is a service that builds binaries for the community for existing library versions, this is not an integration system to test the libraries.

## Why not add an option to build unit tests

- Adding a testing option will change the package ID, but will not provide different packaged binaries
- Use the configuration [skip_test](adding_packages/packaging_policy.md#options-to-avoid) to define the testing behavior.
- Use the configuration [skip_test](adding_packages/conanfile_attributes.md#options-to-avoid) to define the testing behavior.

## What is the policy for supported python versions?

Expand Down