Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import extraction with Rust #581

Merged
merged 79 commits into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
aa9b468
use rust to fetch imports; wip
fpgmaas Mar 9, 2024
0f8366d
Now, we print all imports to console
fpgmaas Mar 9, 2024
815a9cf
working draft that returns ranges
fpgmaas Mar 9, 2024
d17607a
working draft that returns ranges
fpgmaas Mar 9, 2024
0158164
remove the PythonImportExtractor
fpgmaas Mar 9, 2024
04aed39
Remove the Location class from Python
fpgmaas Mar 9, 2024
da10266
working on the data/some_imports.py test
fpgmaas Mar 10, 2024
db1cfa1
Added functionality to read files using file encoding
fpgmaas Mar 10, 2024
3aecccc
now deptry instead of deptryrs
fpgmaas Mar 10, 2024
fa19226
use PDM
fpgmaas Mar 10, 2024
7150ab7
parallel processing attempt
fpgmaas Mar 10, 2024
fbc658d
improved parallel running
fpgmaas Mar 11, 2024
a0998a9
improved parallel running
fpgmaas Mar 11, 2024
9fe6dda
docstrings and formatting
fpgmaas Mar 11, 2024
a732b67
fix CI/CD pipeline
fpgmaas Mar 11, 2024
59985cb
fix merge conflicts with main
fpgmaas Mar 11, 2024
f272882
lock file version
fpgmaas Mar 11, 2024
6c4d83d
new dev cicd pipeline
fpgmaas Mar 11, 2024
0c16f5c
new dev cicd pipeline
fpgmaas Mar 11, 2024
ade10a0
new dev cicd pipeline
fpgmaas Mar 11, 2024
ee37b73
new dev cicd pipeline
fpgmaas Mar 11, 2024
25465e7
remove pdm install
fpgmaas Mar 11, 2024
4ab1d99
add venv
fpgmaas Mar 11, 2024
1d1d117
add rust toolchain
fpgmaas Mar 11, 2024
22bfb36
1.75.0 rust
fpgmaas Mar 11, 2024
a8c9edd
1.75.0 rust
fpgmaas Mar 11, 2024
37cdaf5
create venv manually
fpgmaas Mar 11, 2024
7147fbd
pdm run
fpgmaas Mar 11, 2024
d8d8b50
pdm run
fpgmaas Mar 11, 2024
d4e11dd
try to make modular
fpgmaas Mar 11, 2024
8a09897
try to make modular
fpgmaas Mar 11, 2024
d37c8a1
new attempt
fpgmaas Mar 11, 2024
407cd25
new attempt
fpgmaas Mar 11, 2024
6a7f1ad
new attempt
fpgmaas Mar 11, 2024
f869340
change order
fpgmaas Mar 11, 2024
6a8651f
change order
fpgmaas Mar 11, 2024
33d8dbb
x64 and x86 for windows
fpgmaas Mar 11, 2024
323cf87
add quality and docs check
fpgmaas Mar 11, 2024
8be7d93
add quality and docs check
fpgmaas Mar 11, 2024
bb24bbe
fix mypy
fpgmaas Mar 11, 2024
f778aa2
try to release
fpgmaas Mar 11, 2024
7c078cf
try to release
fpgmaas Mar 11, 2024
1cdfa2b
try to release
fpgmaas Mar 11, 2024
6a08ebe
try to release
fpgmaas Mar 11, 2024
1e8a95b
try to release
fpgmaas Mar 11, 2024
340608b
try to release
fpgmaas Mar 11, 2024
eb700ea
try to release
fpgmaas Mar 12, 2024
1d363ba
try to release
fpgmaas Mar 12, 2024
f6d5db6
try to release
fpgmaas Mar 12, 2024
0b63d5a
try to release
fpgmaas Mar 12, 2024
e434f4b
try to release
fpgmaas Mar 12, 2024
3e85760
build for more python versions
fpgmaas Mar 12, 2024
3ce59f1
try abi3
fpgmaas Mar 12, 2024
945891b
try abi3
fpgmaas Mar 12, 2024
995b227
try abi3
fpgmaas Mar 12, 2024
bff16de
try abi3
fpgmaas Mar 12, 2024
c824ade
try abi3
fpgmaas Mar 12, 2024
87a1a2c
try abi3
fpgmaas Mar 12, 2024
68cb9aa
try abi3
fpgmaas Mar 12, 2024
ef5ab9e
fix code quality
fpgmaas Mar 12, 2024
4585129
fix the workflows
fpgmaas Mar 12, 2024
7a637f3
remove obsolete version line
fpgmaas Mar 12, 2024
628cf9f
move the rust functionality to deptry.rust
fpgmaas Mar 12, 2024
9073132
fix some ruff in tests
fpgmaas Mar 12, 2024
f0cc9e0
remove warning
fpgmaas Mar 12, 2024
d208b3d
improve release pipeline
fpgmaas Mar 12, 2024
4520f0c
split import detection into its own module
fpgmaas Mar 12, 2024
d05ff29
Merge branch 'main' into feature/rust
fpgmaas Mar 12, 2024
d77c4c0
chore: misc tweaks
mkniewallner Mar 12, 2024
06b2adb
chore(pre-commit): run `fmt`/`clippy` locally
mkniewallner Mar 12, 2024
9bfb7c7
refactor: misc Rust tweaks
mkniewallner Mar 12, 2024
18961d7
refactor: remove some Poetry references
mkniewallner Mar 12, 2024
437c342
ci: no need to build for 32-bits
mkniewallner Mar 12, 2024
a1b72a9
chore(ruff): add `deptry` as `known-first-party`
mkniewallner Mar 12, 2024
21ff9a6
test: fix isort spacing
mkniewallner Mar 12, 2024
d71e871
ci: add concurrency
mkniewallner Mar 12, 2024
de371b1
ci: remove another Poetry reference
mkniewallner Mar 12, 2024
86bdcaa
chore: add rust toolchain
mkniewallner Mar 12, 2024
7160a4b
ci: use `rustup` and add cache
mkniewallner Mar 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .github/actions/setup-env/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: 'Setup Python and Rust Environment'
description: 'Set up Python and Rust environment for PDM projects with matrix support'

inputs:
python-version:
description: 'Python version for setup-python'
required: true
default: 3.11
target:
description: 'Target architecture for maturin'
required: true
default: 'x86_64'
python-target:
description: 'Target architecture for python installation'
required: true
default: 'x64'

runs:
using: 'composite'
steps:
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
architecture: ${{ matrix.python-target }}

- name: Setup Rust toolchain
run: rustup component add clippy rustfmt
shell: bash

- uses: Swatinem/rust-cache@v2

- name: Install PDM and dependencies
run: |
python -m pip install pdm
python -m venv .venv
if [ "${{ runner.os }}" = "Windows" ]; then
source .venv/Scripts/Activate
else
source .venv/bin/activate
fi
python -m pip install --upgrade pip
pdm config python.use_venv true
pdm install
shell: bash

- name: Build and install the project using Maturin
uses: PyO3/maturin-action@v1
with:
target: ${{ inputs.target }}
command: develop
sccache: 'true'
49 changes: 0 additions & 49 deletions .github/actions/setup-poetry-env/action.yml

This file was deleted.

107 changes: 72 additions & 35 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,70 +1,104 @@
name: Main
name: CI

on:
pull_request:
types: [opened, synchronize, reopened]
push:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
quality:
runs-on: ubuntu-latest
steps:
- name: Check out
uses: actions/checkout@v4

- uses: actions/cache@v4
- name: Set up the environment
uses: ./.github/actions/setup-env
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
python-version: '3.11'
target: 'x86_64'
python-target: 'x64'

- name: Run pre-commit
run: |
rustup component add rustfmt
pdm run pre-commit run -a --show-diff-on-failure

- name: Inspect dependencies with deptry
run: |
pdm run deptry python

linux:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
target: [x86_64, aarch64]
steps:
- name: Check out
uses: actions/checkout@v4

- name: Set up the environment
uses: ./.github/actions/setup-poetry-env
uses: ./.github/actions/setup-env
with:
python-version: ${{ matrix.python-version }}
target: ${{ matrix.target }}
python-target: 'x64'

- name: Run pre-commit
run: poetry run pre-commit run -a --show-diff-on-failure
- name: Check typing
run: pdm run mypy

- name: Inspect dependencies
run: poetry run deptry .
- name: Run tests
run: pdm run pytest tests --cov --cov-config=pyproject.toml --cov-report=xml

- name: Check Poetry lock file consistency
run: poetry lock --check
- name: Upload coverage reports to Codecov with GitHub Action on Python 3.11 and x86_64
uses: codecov/codecov-action@v3
if: ${{ matrix.python-version == '3.11' && matrix.target == 'x86_64' }}

tests-and-type-check:
runs-on: ${{ matrix.image }}
windows:
runs-on: windows-latest
strategy:
matrix:
os: [macOS, Ubuntu, Windows]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
include:
- os: macOS
image: macos-12
- os: Ubuntu
image: ubuntu-22.04
- os: Windows
image: windows-2022
fail-fast: false
defaults:
run:
shell: bash
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
target: [x64]
steps:
- name: Check out
uses: actions/checkout@v4

- name: Set up the environment
uses: ./.github/actions/setup-poetry-env
uses: ./.github/actions/setup-env
with:
python-version: ${{ matrix.python-version }}
target: ${{ matrix.target }}
python-target: ${{ matrix.target }}

- name: Run tests
run: poetry run pytest tests --cov --cov-config=pyproject.toml --cov-report=xml
run: pdm run pytest tests --cov --cov-config=pyproject.toml --cov-report=xml

- name: Check typing
run: poetry run mypy
macos:
runs-on: macos-latest
strategy:
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
target: [x86_64, aarch64]
steps:
- name: Check out
uses: actions/checkout@v4

- name: Upload coverage reports to Codecov with GitHub Action on Python 3.11 for Ubuntu
uses: codecov/codecov-action@v3
if: ${{ matrix.python-version == '3.11' && matrix.os == 'Ubuntu' }}
- name: Set up the environment
uses: ./.github/actions/setup-env
with:
python-version: ${{ matrix.python-version }}
target: ${{ matrix.target }}
python-target: ${{ matrix.target == 'aarch64' && 'arm64' || 'x64' }}

- name: Run tests
run: pdm run pytest tests --cov --cov-config=pyproject.toml --cov-report=xml

check-docs:
runs-on: ubuntu-latest
Expand All @@ -73,7 +107,10 @@ jobs:
uses: actions/checkout@v4

- name: Set up the environment
uses: ./.github/actions/setup-poetry-env
uses: ./.github/actions/setup-env
with:
python-version: '3.11'
target: 'x86_64'

- name: Check if documentation can be built
run: poetry run mkdocs build -s
run: pdm run mkdocs build -s
41 changes: 0 additions & 41 deletions .github/workflows/on-release-main.yml

This file was deleted.

Loading
Loading