Skip to content

Commit

Permalink
lint
Browse files Browse the repository at this point in the history
  • Loading branch information
ibgreen committed Oct 17, 2024
1 parent 6e481c3 commit 65b4a0a
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 13 deletions.
28 changes: 16 additions & 12 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,33 @@ loaders.gl is developed under open governance by multiple contributors working w
loaders.gl v4.4 will focus on cloud-native, binary data.
A number of modules will expose "ArrowLoaders" will return binary data in the Apache Arrow and Apache GeoArrow formats.

While no loader support has been removed, the flavor of the loaders.gl framework is changing.

**Apache Arrow as a core format**
- Many new loaders now return binary data in the Apache Arrow format.
- This aligns with parallel efforts in companion libraries like deck.gl (as well as the ecosystem at large) to work with zero-copy, compact binary data instead of bloated, deserialized javascript data structures.
- Binary columnar data brings in an order of magnitude better memory usage and improved load/processing performance on big datasets.
- The Apache Arrow JS library is now a central dependency of loaders.gl.

While no loader support has been removed, the flavor of the loaders.gl framework is changing.

**Apache Arrow as a core format**

- Many new loaders now return binary data in the Apache Arrow format.
- This aligns with parallel efforts in companion libraries like deck.gl (as well as the ecosystem at large) to work with zero-copy, compact binary data instead of bloated, deserialized javascript data structures.
- Binary columnar data brings in an order of magnitude better memory usage and improved load/processing performance on big datasets.
- The Apache Arrow JS library is now a central dependency of loaders.gl.

**Improved `DataSource` APIs**
- The `Source` and `DataSource` APIs have matured and are now easier to work with.
- Consule the upgrade guide for migration details.

- The `Source` and `DataSource` APIs have matured and are now easier to work with.
- Consule the upgrade guide for migration details.

### Per-module changes

- **`@loaders.gl/csv`**

- `CSVArrowLoader` - New CSV loader that returns Apache Arrow tables.

- **`@loaders.gl/parquet`**

- `ParquetArrowLoader` now returns Apache Arrow tables and leverages the high-performance `parquet-wasm` library.
- The v4 Parquet loader is still available as `ParquetJSONLoader`

- **`@loaders.gl/schema-utils`**

- New module for working with and converting Apache Arrow data.

- **`@loaders.gl/gis`**
Expand All @@ -37,7 +42,7 @@ While no loader support has been removed, the flavor of the loaders.gl framework
### Upgrading to v4.4

- The `Source` and `DataSource` APIs have matured leading to some minor breaking changes.
- TBA...
- TBA...

---

Expand All @@ -49,7 +54,6 @@ While no loader support has been removed, the flavor of the loaders.gl framework
- More comprehensive support for `options.shape` to control the output format of loaders.
- `ffmpeg` WASM integration for `@loaders.gl/video`


**Single output format per loader**

- The `shape` option that was introduced in loaders.gl v3 to allow loaders to return different data formats is now deprecated and will be removed in many places.
Expand Down
1 change: 0 additions & 1 deletion modules/schema/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,6 @@ export type {
// Struct
// } from './lib/table/arrow-api';


// DEPRECATED

// MESH CATEGORY
Expand Down

0 comments on commit 65b4a0a

Please sign in to comment.