-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update README #528
Merged
Update README #528
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,9 @@ | ||
* [ ] ❗ I have followed the [Contributing to DVCLive](https://github.com/iterative/dvclive/blob/master/CONTRIBUTING.md) guide. | ||
- \[ \] ❗ I have followed the | ||
[Contributing to DVCLive](https://github.com/iterative/dvclive/blob/master/CONTRIBUTING.md) | ||
guide. | ||
|
||
* [ ] 📖 If this PR requires [documentation](https://dvc.org/doc) updates, I have created a separate PR (or issue, at least) in [dvc.org](https://github.com/iterative/dvc.org) and linked it here. | ||
- \[ \] 📖 If this PR requires [documentation](https://dvc.org/doc) updates, I | ||
have created a separate PR (or issue, at least) in | ||
[dvc.org](https://github.com/iterative/dvc.org) and linked it here. | ||
|
||
Thank you for the contribution - we'll try to review it as soon as possible. 🙏 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,164 @@ | ||
# DVCLive | ||
|
||
[![PyPI](https://img.shields.io/pypi/v/dvclive.svg)](https://pypi.org/project/dvclive/) | ||
[![Status](https://img.shields.io/pypi/status/dvclive.svg)](https://pypi.org/project/dvclive/) | ||
[![Python Version](https://img.shields.io/pypi/pyversions/dvclive)](https://pypi.org/project/dvclive) | ||
[![License](https://img.shields.io/pypi/l/dvclive)](https://opensource.org/licenses/Apache-2.0) | ||
|
||
[![Tests](https://github.com/iterative/dvclive/workflows/Tests/badge.svg?branch=main)](https://github.com/iterative/dvclive/actions?workflow=Tests) | ||
[![Codecov](https://codecov.io/gh/iterative/dvclive/branch/main/graph/badge.svg)](https://app.codecov.io/gh/iterative/dvclive) | ||
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit) | ||
[![Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) | ||
|
||
DVCLive is a Python library for logging machine learning metrics and other | ||
metadata in simple file formats, which is fully compatible with DVC. | ||
|
||
# [Documentation](https://dvc.org/doc/dvclive) | ||
|
||
- [Get Started](https://dvc.org/doc/start/experiments) | ||
- [How it Works](https://dvc.org/doc/dvclive/how-it-works) | ||
- [API Reference](https://dvc.org/doc/dvclive/live) | ||
- [Integrations](https://dvc.org/doc/dvclive/ml-frameworks) | ||
|
||
______________________________________________________________________ | ||
|
||
# Quickstart | ||
|
||
## Install *dvclive* | ||
|
||
```console | ||
$ pip install dvclive | ||
``` | ||
|
||
## Initialize DVC Repository | ||
|
||
```console | ||
$ git init | ||
$ dvc init | ||
$ git commit -m "DVC init" | ||
``` | ||
|
||
## Example code | ||
|
||
Copy the snippet below as a basic example of the API usage: | ||
|
||
```python | ||
# train.py | ||
import random | ||
import sys | ||
from dvclive import Live | ||
|
||
with Live(save_dvc_exp=True) as live: | ||
epochs = int(sys.argv[1]) | ||
live.log_param("epochs", epochs) | ||
for epoch in range(epochs): | ||
live.log_metric("train/accuracy", epoch + random.random()) | ||
live.log_metric("train/loss", epochs - epoch - random.random()) | ||
live.log_metric("val/accuracy",epoch + random.random() ) | ||
live.log_metric("val/loss", epochs - epoch - random.random()) | ||
live.next_step() | ||
``` | ||
|
||
See [Integrations](https://dvc.org/doc/dvclive/ml-frameworks) for examples using | ||
DVCLive alongside different ML Frameworks. | ||
|
||
## Running | ||
|
||
Run couple of times passing different values: | ||
|
||
```console | ||
$ python train.py 5 | ||
$ python train.py 5 | ||
$ python train.py 7 | ||
``` | ||
|
||
## Comparing | ||
|
||
DVCLive outputs can be rendered in different ways: | ||
|
||
### DVC CLI | ||
|
||
You can use [dvc exp show](https://dvc.org/doc/command-reference/exp/show) and | ||
[dvc plots](https://dvc.org/doc/command-reference/plots) to compare and | ||
visualize metrics, parameters and plots across experiments: | ||
|
||
```console | ||
$ dvc exp show | ||
``` | ||
|
||
``` | ||
───────────────────────────────────────────────────────────────────────────────────────────────────────────── | ||
Experiment Created train.accuracy train.loss val.accuracy val.loss step epochs | ||
───────────────────────────────────────────────────────────────────────────────────────────────────────────── | ||
workspace - 6.0109 0.23311 6.062 0.24321 6 7 | ||
master 08:50 PM - - - - - - | ||
├── 4475845 [aulic-chiv] 08:56 PM 6.0109 0.23311 6.062 0.24321 6 7 | ||
├── 7d4cef7 [yarer-tods] 08:56 PM 4.8551 0.82012 4.5555 0.033533 4 5 | ||
└── d503f8e [curst-chad] 08:56 PM 4.9768 0.070585 4.0773 0.46639 4 5 | ||
───────────────────────────────────────────────────────────────────────────────────────────────────────────── | ||
``` | ||
|
||
```console | ||
$ dvc plots diff $(dvc exp list --names-only) --open | ||
``` | ||
|
||
![dvc plots diff](./docs/dvc_plots_diff.png) | ||
|
||
### DVC Extension for VS Code | ||
|
||
Inside the | ||
[DVC Extension for VS Code](https://marketplace.visualstudio.com/items?itemName=Iterative.dvc), | ||
you can compare and visualize results using the | ||
[Experiments](https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/experiments-table.md) | ||
and | ||
[Plots](https://github.com/iterative/vscode-dvc/blob/main/extension/resources/walkthrough/plots.md) | ||
views: | ||
|
||
![VSCode Experiments](./docs/vscode_experiments.png) | ||
|
||
![VSCode Plots](./docs/vscode_plots.png) | ||
|
||
While experiments are running, live updates will be displayed in both views. | ||
|
||
### DVC Studio | ||
|
||
If you push the results to [DVC Studio](https://dvc.org/doc/studio), you can | ||
compare experiments against the entire repo history: | ||
|
||
![Studio Compare](./docs/studio_compare.png) | ||
|
||
You can enable | ||
[Studio Live Experiments](https://dvc.org/doc/studio/user-guide/projects-and-experiments/live-metrics-and-plots) | ||
to see live updates while experiments are running. | ||
|
||
______________________________________________________________________ | ||
|
||
# Comparison to related technologies | ||
|
||
**DVCLive** is an *ML Logger*, similar to: | ||
|
||
- [MLFlow](https://mlflow.org/) | ||
- [Weights & Biases](https://wandb.ai/site) | ||
- [Neptune](https://neptune.ai/) | ||
|
||
The main difference with those *ML Loggers* is that **DVCLive** does not | ||
**require** any additional services or servers to run. | ||
|
||
Logged metrics, parameters, and plots are stored as plain text files that can be | ||
shcheklein marked this conversation as resolved.
Show resolved
Hide resolved
|
||
versioned by tools like Git or tracked as pointers to files in DVC storage. | ||
|
||
You can then use different [options](#comparing) to visualize the metrics, | ||
parameters, and plots across experiments. | ||
|
||
______________________________________________________________________ | ||
|
||
# Contributing | ||
|
||
Contributions are very welcome. To learn more, see the | ||
[Contributor Guide](CONTRIBUTING.rst). | ||
|
||
# License | ||
|
||
Distributed under the terms of the | ||
[Apache 2.0 license](https://opensource.org/licenses/Apache-2.0), *dvclive* is | ||
free and open source software. |
This file was deleted.
Oops, something went wrong.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think those could be excluded btw. As far as I remember formatting them breaks the rendering in some subtle way. We are excluding them in all the websites.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hukkin/mdformat#359 😅