-
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
2 changed files
with
182 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
# Contributor Covenant Code of Conduct | ||
|
||
## Our Pledge | ||
|
||
In the interest of fostering an open and welcoming environment, we as | ||
contributors and maintainers pledge to making participation in our project and | ||
our community a harassment-free experience for everyone, regardless of age, body | ||
size, disability, ethnicity, sex characteristics, gender identity and expression, | ||
level of experience, education, socio-economic status, nationality, personal | ||
appearance, race, religion, or sexual identity and orientation. | ||
|
||
## Our Standards | ||
|
||
Examples of behavior that contributes to creating a positive environment | ||
include: | ||
|
||
* Using welcoming and inclusive language | ||
* Being respectful of differing viewpoints and experiences | ||
* Gracefully accepting constructive criticism | ||
* Focusing on what is best for the community | ||
* Showing empathy towards other community members | ||
|
||
Examples of unacceptable behavior by participants include: | ||
|
||
* The use of sexualized language or imagery and unwelcome sexual attention or | ||
advances | ||
* Trolling, insulting/derogatory comments, and personal or political attacks | ||
* Public or private harassment | ||
* Publishing others' private information, such as a physical or electronic | ||
address, without explicit permission | ||
* Other conduct which could reasonably be considered inappropriate in a | ||
professional setting | ||
|
||
## Our Responsibilities | ||
|
||
Project maintainers are responsible for clarifying the standards of acceptable | ||
behavior and are expected to take appropriate and fair corrective action in | ||
response to any instances of unacceptable behavior. | ||
|
||
Project maintainers have the right and responsibility to remove, edit, or | ||
reject comments, commits, code, wiki edits, issues, and other contributions | ||
that are not aligned to this Code of Conduct, or to ban temporarily or | ||
permanently any contributor for other behaviors that they deem inappropriate, | ||
threatening, offensive, or harmful. | ||
|
||
## Scope | ||
|
||
This Code of Conduct applies both within project spaces and in public spaces | ||
when an individual is representing the project or its community. Examples of | ||
representing a project or community include using an official project e-mail | ||
address, posting via an official social media account, or acting as an appointed | ||
representative at an online or offline event. Representation of a project may be | ||
further defined and clarified by project maintainers. | ||
|
||
## Enforcement | ||
|
||
Instances of abusive, harassing, or otherwise unacceptable behavior may be | ||
reported by contacting the project team at [email protected]. All | ||
complaints will be reviewed and investigated and will result in a response that | ||
is deemed necessary and appropriate to the circumstances. The project team is | ||
obligated to maintain confidentiality with regard to the reporter of an incident. | ||
Further details of specific enforcement policies may be posted separately. | ||
|
||
Project maintainers who do not follow or enforce the Code of Conduct in good | ||
faith may face temporary or permanent repercussions as determined by other | ||
members of the project's leadership. | ||
|
||
## Attribution | ||
|
||
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, | ||
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html | ||
|
||
[homepage]: https://www.contributor-covenant.org | ||
|
||
For answers to common questions about this code of conduct, see | ||
https://www.contributor-covenant.org/faq |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,108 @@ | ||
# prom2parquet | ||
|
||
Remote write target for Prometheus that saves metrics to parquet files | ||
|
||
**This should be considered an alpha project** | ||
|
||
## Overview | ||
|
||
The prom2parquet remote write endpoint for Prometheus listens for incoming datapoints from Prometheus and saves them to | ||
[parquet files](https://parquet.apache.org) in a user-configurable location. This can either (currently) be pod-local | ||
storage or to an AWS S3 bucket. Metrics are saved in the following directory structure: | ||
|
||
``` | ||
/data/<prefix>/<metric name>/2024022021.parquet | ||
``` | ||
|
||
Each file for a particular metric will have the same schema, but different metrics may have different schemas. At a | ||
minimum, each file has a `timestamp` and a `value` column, and a variety of other extracted columns corresponding to the | ||
labels on the Prometheus timeseries. They also have a "catch-all" `labels` column to contain other unextracted columns. | ||
|
||
## Usage | ||
|
||
``` | ||
Usage: | ||
prom2parquet [flags] | ||
Flags: | ||
--clean-local-storage delete pod-local parquet files upon flush | ||
-h, --help help for prom2parquet | ||
--prefix string directory prefix for saving parquet files | ||
--remote remote supported remote endpoints for saving parquet files | ||
(valid options: none, s3/aws) (default none) | ||
-p, --server-port int port for the remote write endpoint to listen on (default 1234) | ||
-v, --verbosity verbosity log level (valid options: debug, error, fatal, info, panic, trace, warning/warn) | ||
(default info) | ||
``` | ||
|
||
Here is a brief overview of the options: | ||
|
||
### clean-local-storage | ||
|
||
To reduce pod-local storage, you can configure prom2parquet to remove all parquet files after they've been written | ||
(currently once per hour). This is generally not very useful unless you've also configured a remote storage option. | ||
|
||
### prefix | ||
|
||
This option provides a prefix that can be used to differentiate between metrics collections. | ||
|
||
### remote | ||
|
||
Whether to save the parquet files to some remote storage; currently the only supported remote storage option is AWS S3. | ||
|
||
### server-port | ||
|
||
What port prom2parquet should listen on for timeseries data from Prometheus. | ||
|
||
## Configuring Prometheus | ||
|
||
Prometheus needs to know where to send timeseries data. You can include this block in your Prometheus's `config.yml`: | ||
|
||
```yaml | ||
remote_write: | ||
- url: http://prom2parquet-svc.monitoring:1234/receive | ||
remote_timeout: 30s | ||
``` | ||
Alternately, if you're using the [Prometheus operator](https://prometheus-operator.dev), you can add this configuration | ||
to your Prometheus custom resource: | ||
```yaml | ||
spec: | ||
remoteWrite: | ||
- url: http://prom2parquet-svc.monitoring:1234/receive | ||
``` | ||
## Contributing | ||
We welcome any and all contributions to prom2parquet project! Please open a pull request. | ||
### Development | ||
To set up your development environment, run `git submodule init && git submodule update` and `make setup`. To build | ||
`prom2parquet`, run `make build`. | ||
|
||
This project uses [🔥Config](https://github.com/acrlabs/fireconfig) to generate Kubernetes manifests from definitions | ||
located in `./k8s/`. If you want to use this mechanism for deploying prom2parquet, you can just type `make` to build | ||
the executable, create and push the Docker images, and deploy to the configured Kubernetes cluster. | ||
|
||
All build artifacts are placed in the `.build/` subdirectory. You can remove this directory or run `make clean` to | ||
clean up. | ||
|
||
### Code of Conduct | ||
|
||
Applied Computing Research Labs has a strict code of conduct we expect all contributors to adhere to. Please read the | ||
[full text](https://github.com/acrlabs/simkube/blob/master/CODE_OF_CONDUCT.md) so that you understand the expectations | ||
upon you as a contributor. | ||
|
||
### Copyright and Licensing | ||
|
||
SimKube is licensed under the [MIT License](https://github.com/acrlabs/simkube/blob/master/LICENSE). Contributors to | ||
this project agree that they own the copyrights to all contributed material, and agree to license your contributions | ||
under the same terms. This is "inbound=outbound", and is the [GitHub | ||
default](https://docs.github.com/en/site-policy/github-terms/github-terms-of-service#6-contributions-under-repository-license). | ||
|
||
> [!WARNING] | ||
> Due to the uncertain nature of copyright and IP law, this repository does not accept contributions that have been all | ||
> or partially generated with GitHub Copilot or other LLM-based code generation tools. Please disable any such tools | ||
> before authoring changes to this project. |