Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The merge complete #4

Closed
wants to merge 55 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
c599739
Merge machine-learning-api into hsml subdirectory
aversey Jul 13, 2024
d8f3979
Merge feature-store-api into hsfs subdirectory
aversey Jul 13, 2024
0d753cb
Remove redundant java-related files
aversey Jul 13, 2024
02d3529
Move java code of hsfs
aversey Jul 13, 2024
1e8828c
Move locust_benchmark of hsfs
aversey Jul 13, 2024
11530a2
Move python code of hsfs
aversey Jul 13, 2024
d4b3ba5
Move python code of hsml
aversey Jul 13, 2024
fc822fb
Move hsml tests
aversey Jul 13, 2024
edd37e5
Remove redundant __init__.py in hsml/tests
aversey Jul 13, 2024
fa3d841
Move hsfs tests
aversey Jul 13, 2024
2c2e2d2
Merge hsfs tests
aversey Jul 13, 2024
a6f70d3
Fix the problem with impure tests changing backend_fixtures
aversey Jul 13, 2024
09f5da7
Fix ruff check of test_base_client
aversey Jul 13, 2024
49e3e30
Move hsfs utils
aversey Jul 13, 2024
a4e2ce2
Merge hsfs docs
aversey Jul 14, 2024
69f8771
Fix pyproject dependencies for hsfs docs
aversey Jul 14, 2024
b937e71
Merge the rest of hsfs docs
aversey Jul 14, 2024
496714b
Merge hsfs mkdocs
aversey Jul 14, 2024
95b3812
Merge hsml docs
aversey Jul 14, 2024
31bc652
Add files generated by auto_doc to gitignore
aversey Jul 16, 2024
5372ae8
Merge hsfs pyproject.toml
aversey Jul 3, 2024
5a3e833
Merge hsml pyproject.toml
aversey Jul 3, 2024
d217788
Remove hsfs and hsml pyproject.toml
aversey Jul 3, 2024
12001ff
Remove hive from pyproject dependencies
aversey Jul 14, 2024
fbd932e
Fix mistypes in pyproject
aversey Jul 14, 2024
65a939a
Fix documentation generation
aversey Jul 8, 2024
41e8e55
Fix circular dependencies
aversey Jul 8, 2024
78f77ee
Fix circular dependency in hsml (CONNECTION_SAAS_HOSTNAME)
aversey Jul 14, 2024
97c28d4
Fix circular dependency in hsml (model/util)
aversey Jul 14, 2024
f8e8b90
Ruff fix hsml/util
aversey Jul 14, 2024
608a92b
Fix docgen
aversey Jul 15, 2024
d91a209
Fix hsml.python import errors
aversey Jul 15, 2024
f7b1f36
Fix import problem in docgen
aversey Jul 15, 2024
66f266b
Ruff fix auto_doc
aversey Jul 15, 2024
d18d394
Revert the move of requirements-docs to an extra
aversey Jul 16, 2024
8f6bb0d
Move workflows
aversey Jul 15, 2024
bf290a0
Merge workflows
aversey Jul 15, 2024
1b44efd
Remove redundant github files
aversey Jul 15, 2024
e4ee584
Skip test_login until Robin is back
aversey Jul 11, 2024
f4eac5a
Fix test_connection
aversey Jul 15, 2024
e25429a
Fix test_hopsworks_udf
aversey Jul 15, 2024
e2e746c
Fix import problem in docgen
aversey Jul 15, 2024
05340c0
Fix pyproject optional docgen dependency
aversey Jul 15, 2024
864e4d2
Revert the move of requirements-docs to an extra
aversey Jul 16, 2024
f1fba26
Rename unit_tests_pandas to unit_tests_pandas1 for clarity
aversey Jul 16, 2024
8131c4b
Rename unit_tests_typechecked
aversey Jul 16, 2024
d92d572
Merge workflows
aversey Jul 15, 2024
c3456e6
Fix import problem in docgen
aversey Jul 15, 2024
9c573da
Merge and remove the rest of python files
aversey Jul 15, 2024
aec69be
Merge gitignore
aversey Jul 15, 2024
349fed6
Move docker and jenkins files
aversey Jul 15, 2024
8696570
Merge CONTRIBUTING
aversey Jul 15, 2024
49afc85
Merge README and remove hsfs and hsml subdirectories
aversey Jul 15, 2024
6ffa4c0
Revert the move of requirements-docs to an extra
aversey Jul 16, 2024
6c8e8f4
Add hopsworks_common
aversey Jul 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/java-ut.yml → .github/workflows/java.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ name: java
on: pull_request

jobs:
unit_tests_utc:
name: Java Unit Tests
unit_tests:
name: Unit Tests
runs-on: ubuntu-latest

steps:
Expand All @@ -17,8 +17,8 @@ jobs:
- name: Set up JDK 8
uses: actions/setup-java@v3
with:
java-version: '8'
distribution: 'adopt'
java-version: "8"
distribution: "adopt"

- name: Cache local Maven repository
uses: actions/cache@v2
Expand All @@ -32,8 +32,8 @@ jobs:
working-directory: ./java
run: mvn clean test

unit_tests_local:
name: Java Unit Tests (Local TZ)
unit_tests_local_tz:
name: Unit Tests (Local TZ)
runs-on: ubuntu-latest

steps:
Expand All @@ -46,8 +46,8 @@ jobs:
- name: Set up JDK 8
uses: actions/setup-java@v3
with:
java-version: '8'
distribution: 'adopt'
java-version: "8"
distribution: "adopt"

- name: Cache local Maven repository
uses: actions/cache@v2
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
name: mkdocs-master
name: mkdocs-main

on: pull_request

jobs:
publish-master:
publish-main:
runs-on: ubuntu-latest

steps:
Expand All @@ -21,10 +21,10 @@ jobs:

- name: install deps
working-directory: ./python
run: cp ../README.md . && pip3 install -r ../requirements-docs.txt && pip3 install -e .[python,dev]
run: cp ../README.md . && pip3 install -r ../requirements-docs.txt && pip3 install -e .[dev]

- name: generate autodoc
run: python3 auto_doc.py
run: python3 ./python/auto_doc.py

- name: Cache local Maven repository
uses: actions/cache@v2
Expand All @@ -33,7 +33,6 @@ jobs:
key: ${{ runner.os }}-maven-${{ hashFiles('java/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-

- name: Set up JDK 8
uses: actions/setup-java@v3
with:
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/mkdocs-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ jobs:

- name: install deps
working-directory: ./python
run: cp ../README.md . && pip3 install -r ../requirements-docs.txt && pip3 install -e .[python,dev]
run: cp ../README.md . && pip3 install -r ../requirements-docs.txt && pip3 install -e .[dev]

- name: generate autodoc
run: python3 auto_doc.py
run: python3 ./python/auto_doc.py

- name: Cache local Maven repository
uses: actions/cache@v2
Expand All @@ -38,7 +38,6 @@ jobs:
key: ${{ runner.os }}-maven-${{ hashFiles('java/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-

- name: Set up JDK 8
uses: actions/setup-java@v3
with:
Expand All @@ -47,12 +46,14 @@ jobs:

- name: Build java doc documentation
working-directory: ./java
run: mvn clean install javadoc:javadoc javadoc:aggregate -DskipTests && cp -r target/site/apidocs ../docs/javadoc
run:
mvn clean install javadoc:javadoc javadoc:aggregate -DskipTests && cp -r target/site/apidocs ../docs/javadoc

- name: setup git
run: |
git config --global user.name Mike
git config --global user.email [email protected]

- name: mike deploy docs
run: |
mike deploy ${{ env.RELEASE_VERSION }} ${{ env.MAJOR_VERSION }} -u --push
Expand Down
29 changes: 0 additions & 29 deletions .github/workflows/optional-dependency.yml

This file was deleted.

72 changes: 46 additions & 26 deletions .github/workflows/python-lint.yml → .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ name: python

on: pull_request

env:
APP_API_KEY: ${{ secrets.APP_API_KEY }}
ENABLE_HOPSWORKS_USAGE: "false"

jobs:
lint_stylecheck:
name: Lint and Stylecheck
Expand Down Expand Up @@ -31,22 +35,25 @@ jobs:
- name: ruff on python files
if: steps.get-changed-files.outputs.src_any_changed == 'true'
env:
SRC_ALL_CHANGED_FILES: ${{ steps.get-changed-files.outputs.src_all_changed_files }}
SRC_ALL_CHANGED_FILES:
${{ steps.get-changed-files.outputs.src_all_changed_files }}
run: ruff check --output-format=github $SRC_ALL_CHANGED_FILES

- name: ruff on test files
if: steps.get-changed-files.outputs.test_any_changed == 'true'
env:
TEST_ALL_CHANGED_FILES: ${{ steps.get-changed-files.outputs.test_all_changed_files }}
TEST_ALL_CHANGED_FILES:
${{ steps.get-changed-files.outputs.test_all_changed_files }}
run: ruff check --output-format=github $TEST_ALL_CHANGED_FILES

- name: ruff format --check $ALL_CHANGED_FILES
env:
ALL_CHANGED_FILES: ${{ steps.get-changed-files.outputs.all_changed_files }}
ALL_CHANGED_FILES:
${{ steps.get-changed-files.outputs.all_changed_files }}
run: ruff format $ALL_CHANGED_FILES

unit_tests_ubuntu_utc:
name: Unit Testing (Ubuntu)
unit_tests:
name: Unit Tests
needs: lint_stylecheck
runs-on: ubuntu-latest
strategy:
Expand All @@ -73,12 +80,34 @@ jobs:
run: python --version

- name: Run Pytest suite
env:
ENABLE_HOPSWORKS_USAGE: "false"
run: pytest python/tests

unit_tests_ubuntu_pandas:
name: Unit Testing (Ubuntu) (Pandas 1.x)
unit_tests_no_opt:
name: Unit Tests (No Optional Dependencies)
needs: lint_stylecheck
runs-on: ubuntu-latest

steps:
- name: Set Timezone
run: sudo timedatectl set-timezone UTC

- uses: actions/checkout@v4
- name: Copy README
run: cp README.md python/

- uses: actions/setup-python@v5
name: Setup Python
with:
python-version: "3.10"
cache: "pip"
cache-dependency-path: "python/setup.py"
- run: pip install -e python[python,dev-no-opt]

- name: Run Pytest suite
run: pytest python/tests

unit_tests_pandas1:
name: Unit Tests (Pandas 1.x)
needs: lint_stylecheck
runs-on: ubuntu-latest

Expand All @@ -102,12 +131,10 @@ jobs:
run: python --version

- name: Run Pytest suite
env:
ENABLE_HOPSWORKS_USAGE: "false"
run: pytest python/tests

unit_tests_ubuntu_local:
name: Unit Testing (Ubuntu) (Local TZ)
unit_tests_local_tz:
name: Unit Tests (Local TZ)
needs: lint_stylecheck
runs-on: ubuntu-latest

Expand All @@ -131,12 +158,10 @@ jobs:
run: python --version

- name: Run Pytest suite
env:
ENABLE_HOPSWORKS_USAGE: "false"
run: pytest python/tests

unit_tests_ubuntu_typechecked:
name: Typechecked Unit Testing (Ubuntu)
unit_tests_typechecked:
name: Unit Tests (Typechecked)
needs: lint_stylecheck
runs-on: ubuntu-latest

Expand All @@ -155,13 +180,12 @@ jobs:

- name: Run Pytest suite
env:
ENABLE_HOPSWORKS_USAGE: "false"
HOPSWORKS_RUN_WITH_TYPECHECK: "true"
run: pytest python/tests
continue-on-error: true

unit_tests_windows_utc:
name: Unit Testing (Windows)
unit_tests_windows:
name: Unit Tests (Windows)
needs: lint_stylecheck
runs-on: windows-latest

Expand All @@ -185,12 +209,10 @@ jobs:
run: python --version

- name: Run Pytest suite
env:
ENABLE_HOPSWORKS_USAGE: "false"
run: pytest python/tests

unit_tests_windows_local:
name: Unit Testing (Windows) (Local TZ)
unit_tests_windows_local_tz:
name: Unit Tests (Windows) (Local TZ)
needs: lint_stylecheck
runs-on: windows-latest

Expand All @@ -217,6 +239,4 @@ jobs:
run: pip freeze

- name: Run Pytest suite
env:
ENABLE_HOPSWORKS_USAGE: "false"
run: pytest python/tests
10 changes: 3 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -134,12 +134,8 @@ target/
# mkdocs intemediate files
docs/generated

docs/CONTRIBUTING.md
docs/index.md

# Test artifacts
keyFile.json

# delombok dir
delombok

# dev scripts dir
dev_scripts/
dev_tools/
Loading
Loading