Releases: seasketch/geoprocessing
v7.0.0-beta.13
What's Changed
π New Feature / Improvement
- Preprocessor update by @twelch in #393
- Migrate all preprocessing functions away from using the
clipToPolygonFeatures
abstraction and directly callingclip
andclipMultiMerge
. This makes it easier for developer to reason about and test what is happening. - Directly use subdivided flatgeobuf global datasources instead of VectorDataSource bundles. no need to register them in datasources.json just directly use URL.
- Switch from OSM land to Daylight coastline polygons.
- Add
biggestPolygon
helper function used by preprocessing functions to return the largest polygon to SeaSketch when multiple remain after clipping. - Re-export toJsonFile helper for quickly writing JSON to disk.
- Migrate all preprocessing functions away from using the
π Bug Fix
- avoid node-fetch error when fetching VectorDataSource metadata by migrating preprocessing functions to use new subdivided global datasources.
- clipToLand: fix bug where if land feature fetch returned no polygons it would return the original sketch, instead of throwing that the sketch is outside the land boundary.
π Internal
- refine polygonSmokeTest logging
Full Changelog: v7.0.0-beta.12...v7.0.0-beta.13
v7.0.0-beta.12
What's Changed
π Deprecated
- Add
overlapPolygonArea
andoverlapPolygonSum
, deprecatingoverlapFeatures
by @twelch in #389.- The input parameters and algorithms for the two operations offered by overlapFeatures (area and sum) had diverged too far and it made sense to split them into separate functions. overlapFeatures only let you run one operation or the other so this should be a simple migration of code to call one or the other.
- Split out
toMultiPolygon
andremoveOverlap
helper functions used as building blocks for these functions.
π New Feature / Improvement
- callWithRetry updates, add back node-fetch polyfill due to unsolved SocketClosed error by @twelch in #388
callWithRetry
- add optional ifErrorMsgContains string, will now retry only if thrown error message contains that string otherwise. add exponential backoff so that there is a an increasing wait before the next retry.rasterStats
no longer catches and swallows all errors. Improved to catch specific error thrown when a zero polygon is passed to geoblaze, and returns default raster stats, otherwise it rethrows the error.- add back node-fetch polyfill to cog and flatgeobuf to solve SocketError showing up during high volume network call situation (e.g. smoke tests).
- Upgrade to latest CDK lib and Node 22 lambda runtime by @twelch in #392
π Docs
- Update toolbox page with new overlap functions and deep links to API docs.
Full Changelog: v7.0.0-beta.11...v7.0.0-beta.12
v7.0.0-beta.11
What's Changed
π₯ Breaking Changes
Users on upgrade should look at preprocessing functions now installed with new project and borrow their code. See global datasources for datasource listing.
- Drop
genClipLoader
, replaced withclipToPolygonFeatures
/genClipToPolygonFeatures
andclipToPolygonDatasources
/genClipToPolygonDatasources
. Default preprocessing functions have migrated to usingclipToPolygonFeatures
and fetching features in a separate step. - Drop JSON format from data import and precalc. It's no longer required now that precalc reads fgb from disk and fgb is superior for other uses.
π Deprecated
- Deprecate
getFeatures
(in name) and replace with same functionality ingetDatasourceFeatures
. This makes it clear that this is a datasource aware function.
π New Feature / Improvement
getDatasourceFeatures
now accepts optional sketch option. When used it calls optimized getFeaturesForSketchBBoxesgetFeaturesForSketchBBoxes
add support for all geometry types and allow user to extend them with generics support- Add lower-level
getFeaturesForBBoxes
that takes an array of bounding boxes - loadFgb: add retry on error with exponential backoff (3 times). This will hopefully avoid the Closed socket error seeing in testing environment, seemingly in high volume request situation.
π Internal
- Drop old doc build workflow by @twelch in #382
- Drop global datasources from base-project. No longer included with new project.
- Contribute squares.fgb, squares.json with easy to reason about named polygons for testing. Develop getFeaturesForSketchBBoxes e2e tests to use them.
- Update squareSketches fixtures to match squares.fgb/json for testing.
- Add squaresReadme.txt ascii art image to see the layout of the polygons
π Docs
- Drop macos direct install support from docs by @twelch in #383
- Delete chunked cache results on clear:results by @avmey in #387
Full Changelog: v7.0.0-beta.10...v7.0.0-beta.11
v7.0.0-beta.10
What's Changed
Be sure to run the upgrade
command after updating to this latest beta version. On upgrade:
- genRandomSketch script will be removed in favor of the new genRandomPolygon that supports both regular features and sketches. See sample tutorial for examples.
π₯ Breaking Changes
- drop deprecated
loadCogWindow
andbboxToPixelEdge
functions - drop
overlapRaster
function - re-export dataprovider in top-level library instead of having as a separate export (now that esm is here and flatgeobuf issues have been fixed), migrate base project to use it.
- drop
genPreprocessor
function. Replace with clipToPolygonFeatures, genClipToPolygonFeatures, clipToPolygonDatasources and genClipToPolygonDatasources. - drop genRandomSketch, merge into genRandomFeature and refactor with more options
- drop use of node-fetch for the global fetch offered by Node. There is a "Socket closed" error that occurs when a high number of requests in test environment. This should be fixed with an upgrade to Node v22 (to come).
π Deprecated
- deprecated
fgbFetchAll
function for loadFgb - deprecated
genClipLoader
function, no longer needed now that genClipToPolygonFeatures and genClipToPolygonDatasources is available - deprecated use of bbox and planning area properties in basic.json.
- deprecated template-ocean-eez. template-blank-project is now installed by default. It may simply get merged with base-project before 7.0 release.
createReport
has been found to be a better way of letting developer install a variety of report types.
π New Feature / Improvement
- speed up storybook, merge existing project prettierignore with base on upgrade by @twelch in #361
- Add docusaurus website by @twelch in #362
- Fix precalc cleanup script by @avmey in #363
- Fix multiple choice sketch attributes bug by @avmey in #368
- ADA Accessibility by @avmey in #370
- add validatePolygon preprocessor, with ensureValidPolygon helper function
- add roundDecimalFormat and numberFormat helper functions
- add toSketchProperties helper function
- add getFeaturesForSketchBboxes function with split feature, add splitBBoxAntimeridian and cleanBBox helpers,
- make MetricGroup
type
property optional. - export intersection functions - intersectInChunks, intersectInChunksArea, intersectSum
- add getMetricGroupDatasource and getMetricGroupClassKey methods to projectClientBase
- improve metricGroupSchema in schemas.json
project init:
- drop bbox, planning area, and template questions.
- remove template-ocean-eez from workspace so no longer tested (likely to be removed in future)
- clean up simple and blank reports (function + cards)
- drop ViabilityPage from TabReport and simply inline all report pages
- dataImport: conditional ask additional properties question
- sort init language alphabetically
- add vitest-fetch-mock dep to base project
- migrate to childProperties
- add validatePolygon preprocessor
- add minSize and enforceMinSize options to preprocessors
- drop global datasources from basic-project template except for osm-land and eez-land-union
- remove default objective and metricgroup from project. objectives.json and metrics.json start as empty arrays now. This is inline with them being opt-in features rather than forced use.
createReport:
- switch vectorFunction to use getFeaturesForSketchBboxes.
- migrate vectorFunction/rasterFunction/OverlapCard from using NullSketch in result to get sketch properties to using useSketchProperties
- createFunction and createReport now default to async geoprocessing function. The user often doesn't know which they need right away.
π Bug Fix
- fix genRandomPolygon name bug
- fix i18nAsync bug using base english strings instead of letting component render its own
- fix storybook error when sketch name has spaces or any characters not valid in a Javascript variable name
- remove .story-cache folders from dist build
- genRandomPolygon - start id's at 1, was causing a bug downstream with 0 id value
π Internal
- rename split.js to antimeridian.js.
- storybook no longer auto-opens web browser, caused error in geoprocessing WSL.
- migrate to loadFgb internally from deprecated fgbFetchAll
- upgrade to flatgeobuf 3.36.0 and drop use of web streams polyfill and node-fetch now that bug fixed.
- add Vite babel plugins for storybook
- Remove antimeridian tests from getFeatures
- add getFgbPath method to projectClientBase for loading of flatgeobuf from local filesystem
- move all createReport assets over to now unused template-ocean-eez directory. eventually need to simplify this.
π Docs
- Fix 6.1.0 tutorial, referenced features not available in 6.x. Also some issues with ordering of steps.
- Major refactor and updates to Next docs, complete with systemsetup, sampleproject and newproject tutorials, as well as a number of guides.
- remove codespaces as documented tutorial option
Full Changelog: v7.0.0-beta.9...v7.0.0-beta.10
v7.0.0-beta.9
What's Changed
When you npm run upgrade
- you can expect a lot of changes to installed code due to use of the eslint unicorn plugin internally in the geoprocessing library.
- Project languages are now defined in
project/basic.json
and the English language codeEN
will have been added. If there are additional languages that your project offers translation, then manually add their language codes. You'll find the master list insrc/i18n/languages.json
.
This will configure the translationpublish
andimport
commands to work with these languages. Some old i18n files such assrc/i18n/supported.ts
will be deleted, with the addition ofsrc/i18n/languages.ts
.
π₯ Breaking Changes
- Allow user to limit project i18n languages, change watch to build:watch by @twelch in
- Some CLI commands have been renamed.
watch
is nowbuild:watch
,clear-all-results
has been removed, andclear-results
is nowclear:results
and you can enter the option to deleteall
cached results. This has the benefit of limiting accidental deletes of the full cache, you have to specifically state you want to delete all.
π New Feature / Improvement
#356
- Add eslint unicorn plugin with fixes by @twelch in #355
- Add Brazilian Portuguese to supported languages by @avmey in #354
- upgrade: remove old translation scripts by @twelch in #360
π Bug Fix
Full Changelog: v7.0.0-beta.8...v7.0.0-beta.9
v7.0.0-beta.8
What's Changed
Running upgrade
on your project will:
- Install and configure Prettier along with a new
lint-staged
command. If you are using VSCode you should already have the Prettier extension installed but this addition makes formatting rules consistent across developer machines and will auto-format all staged files on commit if not already.
π₯ Breaking Changes
- Separate worker stacks and lock functions to one stack solving thrashing issue causing Duplicate resource error on deploy by @twelch in #353
- Require explicit registration of worker functions, fix CDK circular dependency error by @twelch in #352
To register workers used by a geoprocessing function, simply pass a workers
option to your GeoprocessingHandler with a list of the worker titles.
export default new GeoprocessingHandler(kelpPersist, {
title: "boundaryOverlap",
description: "boundary overlap",
executionMode: "async",
workers: ['boundaryOverlapWorker']
});
The architecture has changed from beta.7 for nested LambdaStacks. Worker functions are now deployed to separate stacks than non-workers. This should allow multiple geoprocessing functions to register and use the same worker without issue.
Upgrade will require destroying your deployed project first. Steps in order include:
npm run upgrade
npm run destroy
npm run build && npm run deploy
npm run publish:data
π New Feature / Improvement
π Bug Fix
npm run url
command now properly outputs all service URLs and bucket names used by the project for easy reference.- Fix
clear-all-results
command. Add retry to dynamodb bulk delete resolving ThroughputError, break out helper functions by @twelch in #344
π Internal
- Disable task cache for worker lambdas by default, improve logging by @twelch in #342
- Add support for ESLint and Prettier by @twelch in #348
Full Changelog: v7.0.0-beta.7...v7.0.0-beta.8
v7.0.0-beta.7
What's Changed
π₯ Breaking Changes
- Switch tasks class to storing dynamodb results as JSON string chunks, drop correlationId and peerDependencies by @twelch in #330 and #336
A new algorithm is used for storing function results in DynamoDB that works regardless of the structure of your JSON. After deploy, you will need to clear your existing cache by running npm run clear-all-results
There are no longer devDependencies
installed for a project whether using init
or upgrade
. On your next upgrade, all dependencies will be added as dependencies
and you will need to manually remove them from devDependencies
because upgrade does not handle deletion.
π New Feature / Improvement
- Migrate to aws-sdk v3 by @twelch in #321
- Add runLambdaWorker helper for invoking sync worker functions by @avmey in #326
π Bug Fix
- Generate clipped bounding boxes when using clipToGeography by @avmey in #325
- Fix task caching and improved logging by @twelch in #336
π Internal
- update dependencies by @twelch in #304 and #324
- Add additional logging to GeoprocessingHandler to be able to assess functions are working as expected.
Full Changelog: v7.0.0-beta.6...v7.0.0-beta.7
v7.0.0-beta.6
What's Changed
π₯ Breaking Changes
- Migrate lambdas to NestedStack to avoid cdk 500 resource limit. Scales to add more stacks as needed automatically by @twelch in #309 and #314. The breaking change is that due to fixed logical ID's being used for each lambda function, a
duplicate ID
error will result when redeploying an existing project and CDK tries to create a function in a nested LambdaStack that already exists in the root stack. It doesn't appear to take care of the deletion first. Until a solution can be found, you will need to completely destroy and redeploy your project.
See Migrating guide for more information.
π New Feature / Improvement
- Overcome dynamoDB 400KB item size limit by splitting metric results across one or more additional metricGroup DB items in a way that auto-scales @twelch in #313
π Bug Fix
- Fix template path bug on new project init by @twelch in #308
- Handle no-overlap for categorical rasters, creating proper zero metrics by @avmey in #315
- bug: task metricGroup results not saved when just one group by @twelch in #319
π Internal
- Add node 22 to CI tests in prep for future use by @twelch in #305
- End-to-end test of project build + CDK synth for a variety of scenarios by @twelch in #314
Full Changelog: v7.0.0-beta.5...v7.0.0-beta.6
v7.0.0-beta.5
π₯ Breaking Changes
- Add new
upgrade
script for migrating to latest version of geoprocessing library. by @twelch in #301, improved in #306. To use it, you must manually add it to the geoprocessing.json scripts section -"upgrade": "geoprocessing upgrade"
. Run the upgrade script and review any code changes. Some upgrade steps will overwrite existing files in.storybook
,src/i18n
,project
,package.json
, etc. so if you have made custom changes you will need to recover them. - Move geoprocessing.json from top-level to
project/geoprocessing.json
. Upgrade script handles move and updating import insrc/project/projectClients.ts
. Any other imports must be manually updated.
π Internal
- Add Node 22 to CI tests
- Drop wiki docs from repo including publish action. For now they will be managed in the wiki directly again.
- Drop mentions of test:smoke
- Drop all group and environment directives from tests
- Extract getTemplatePackages helper function
Full Changelog: v7.0.0-beta.4...v7.0.0-beta.5
v7.0.0-beta.4
π₯ Breaking Changes
- Remove
prepare
script from package.json. No longer used.
π New Feature / Improvement
- CLI enhancements by @avmey in #280
- Upgrade to Turf v7.0 from beta. Upgrade any turf dependencies in your GP projects package.json. via @twelch
π Bug Fix
- Socket hang up error by @twelch in #300
- Fix tooltip z-index issue and font styling. Document pdf printing by @avmey in #276
Full Changelog: v7.0.0-beta.3...v7.0.0-beta.4