Releases: esi-neuroscience/syncopy
Releases · esi-neuroscience/syncopy
Syncopy v2023.09
Syncopy release v2023.09
NEW
- Add 'channelcmb' parameter to connectivityanalysis frontend. This allows users to compute connectivity measures only between a subset of the channels instead of all channels, which can reduce the required computational cost dramatically. See #565 for details.
Changed
- Improvements to serializable dict: better error messages, add _serialize_value helper to try to turn some non-serializable data structures into compatible, serializable ones. #569
Fixed
- Security fix: increase minimal required tornado version to 6.3.3 to address an issue in tornado.
Syncopy v2023.07
Syncopy release 2023.07
This release brings spike plotting and additional data import and export functions to improve interoperability with other neuroscience software packages.
NEW
- Add spike plotting, implements #434
- Conversion to and from mne's RawArray and EpochsArray
- Add export of to NWB format for AnalogData, TimeLockData, and SpikeData. #508
- Add support for reading NWB files containing SpikeData, related to #508
- Add support for concatenating Syncopy data objects (e.g., to add channels) with new
spy.concat
(PR 522)
Changed
- When reading/importing NWB files, support trials (in addition to epochs, which were interpreted as trials). Epochs are still supported.
Fixed
Syncopy 2023.05
Patch release
Considerable performance tweaks and fixes were introduced with #489 and #454, severely improving runtime across the board. Minor additions include NaN handling of FIR filters and the new spy.synthdata
module.
What's Changed
- 478 creating a dummy dataset using snpanalogdata does not create a data set with desired name of the channels by @tensionhead in #479
- CHG: Handle NaNs in preprocessing by @tensionhead in #481
- 475 undo poetry version pin by @dfsp-spirit in #482
- NEW: Standalone synthetic data module by @tensionhead in #477
- Fix GitHub dependabot alert by @dfsp-spirit in #484
- NEW: add contributing file by @dfsp-spirit in #485
- CHG: Fix and extend developer information section of README. [skip ci] by @dfsp-spirit in #487
- 489 redesign time property by @tensionhead in #490
- 454 chan sel by @dfsp-spirit in #455
- FIX: Also delete hdf5 virtual source directory upon save by @tensionhead in #499
- Merge updates from Dev by @tensionhead in #501
- CHG: Improve signature and doc of synthdata routines by @tensionhead in #502
- Add
ar1_noise
synth. data routine by @tensionhead in #503
Full Changelog: v2023.03...v2023.05
Syncopy 2023.03
New Features
- new Pairwise phase consistency (PPC) method, see
connectivityanalysis
function withmethod=ppc
- new frontend function
redefinetrial
to cut trials/move time axes - Jackknifing for coherence and Granger analysis, see new parameter
jackknife
forconnectivityanalysis
function - logging functionality and respective developer documentation, new function
log
#208 - waveform extra dataset to DiscreteData to store raw data around spikes, #238
- create Syncopy data objects from Python generators (yeah!)
- concatenation of Syncopy data objects along trials
Changes
- spectral power for
mtmfft
now independent of padding as originally intended - support unequal trial sizes for
load_ft_raw
- major performance improvements for DiscreteData #403 #418, #424
Fixed
- fix bug #394 'Copying a spy.StructDict returns a dict'.
- serializable
.cfg
#392 - single trial cross-corr bug #446
- fix bug #457, Syncopy does not warn about temp storage dir size exceeding reporting threshold at startup
Detailed contributions
- 394 fix structdict copy by @dfsp-spirit in #400
- Merge Doc sprint2212 by @tensionhead in #402
- Dev Doc Update by @tensionhead in #405
- Update readthedocs.yml and CI Python version by @tensionhead in #406
- FIX: Serialize cfg entries by @tensionhead in #407
- update from Dev by @tensionhead in #411
- Repair doc merge by @tensionhead in #413
- streamline PR template by @tensionhead in #417
- NEW: TrialIndexer by @tensionhead in #422
- 419 rewrite selectdata tests by @tensionhead in #421
- Merge Dev by @tensionhead in #423
- 408 check out contiguous channel property for discrete data by @tensionhead in #415
- FIX: Replace very inefficient discrete _get_trial by @KatharineShapcott in #403
- Another discrete speedup by @KatharineShapcott in #424
- Add csd connectivity by @kajal5888 in #416
- Logging by @dfsp-spirit in #409
- Update from dev by @tensionhead in #425
- Update from dev by @tensionhead in #429
- Add log function to syncopy root namespace by @dfsp-spirit in #430
- FIX: load_nwb creates integer EventData by @KatharineShapcott in #428
- CHG: Avoid list comprehension for discrete select by @KatharineShapcott in #427
- New _register_dataset tests by @dfsp-spirit in #436
- Add jackknifing by @tensionhead in #435
- Waveform by @dfsp-spirit in #437
- 438 check workers available causes execution to hang by @tensionhead in #440
- 404 jackknifing frontend by @tensionhead in #441
- 444 allow unequal trials for load ft raw by @tensionhead in #445
- FIX: Correct trialdefinition for odd sample numbers by @tensionhead in #447
- Poetry install failing on github after poetry 1.4.1 release by @dfsp-spirit in #450
- FIX: fix CI on master, see #451 by @dfsp-spirit in #452
- 201 implement ppc by @tensionhead in #443
- 266 concatenate syncopy data objects by @tensionhead in #448
- Add redefinetrial by @tensionhead in #449
- Add channel to load tdt by @kajal5888 in #453
- CHG: Normalize spectra independent of padding by @tensionhead in #459
- 456 diskspace warn by @dfsp-spirit in #457
- Pre-release Dev merge by @tensionhead in #460
- FIX: Add seed values to the synthetic data tests by @tensionhead in #461
Full Changelog: v2022.12...v2023.03
Syncopy 2022.12
New Features
- descriptive statistics:
spy.mean
,spy.var
,spy.std
andspy.median
- new high-level function
spy.timelockanalysis
- inter trial coherence:
spy.itc
- PSTH method:
spy.spike_psth
- added Welch's method to
spy.freqanalysis
- connectivity analysis from
SpectralData
input - time dependent coherence
.info
entries for Granger and FOOOF analyses with fixed metadata labels
Changes
spy.selectdata
now works withlatency
andfrequency
instead oftoi/toilim
,foi/foilim
likeft_selectdata
- a valid
latency
selection will always return timelocked data
Fixes
- user selections don't get silently wiped by high-level function calls like
spy.freqanalysis
#332 SpikeData
slow loading issue #360- mempory consumption of trial averaging, both sequential and parallel #380
- custom dimord support for plotting #349
- updated dependencies, support for Python >= 3.8 to allow default conda environments w/o Python pinning #363
- nwb reader improved memory footprint
Detailed Changes
- Dev merge by @tensionhead in #322
- re-sync version and CI .yml by @tensionhead in #334
- FIX: Construct trialdefinition of resampled data by @tensionhead in #337
- Update from Dev by @tensionhead in #338
- 140 multiple outputs by @dfsp-spirit in #321
- PSTH by @tensionhead in #339
- 340 interface additional cf returns by @tensionhead in #343
- Psth tests by @tensionhead in #347
- Seq stats premerge by @dfsp-spirit in #350
- 332 inplace sel by @dfsp-spirit in #346
- Fix poetry version by @dfsp-spirit in #352
- CHG: Add versions for all packages listed in pyproject.toml by @dfsp-spirit in #354
- FIX: Equality operator branching by @tensionhead in #362
- 344 rng by @dfsp-spirit in #355
- 349 plotting does not support custom dimords by @tensionhead in #365
- 364 rewire connectivity analysis by @tensionhead in #366
- 341 implement sequential statistics mean stddev for syncopydata by @tensionhead in #372
- Dev by @tensionhead in #374
- WIP: add function to rearrange metadata labels by @dfsp-spirit in #356
- Quick fix: spyvalueerrors by @dfsp-spirit in #377
- Revert ACME to optional dependency, introduce fallback parallel processing by @tensionhead in #373
- 287 wavelet cross spectra by @tensionhead in #375
- Timelockanalysis + PSTH finish by @tensionhead in #381
- 384 deprecate toitoilim and foifoilim for selectdata by @tensionhead in #385
- Bump certifi from 2022.6.15 to 2022.12.7 by @dependabot in #389
- memory-improvements-nwb by @KatharineShapcott in #386
- 367 inter trial coherence itc by @tensionhead in #388
- 368 welchs method by @dfsp-spirit in #383
- Dev merge for 2022.12 release by @tensionhead in #390
- 387 fix spycleanup by @dfsp-spirit in #391
- Fix win test by @dfsp-spirit in #396
- Release prep merge by @tensionhead in #397
- 392 serializable cfg entries by @kajal5888 in #393
- Last minute update by @tensionhead in #398
Full Changelog: v2022.08...v2022.12
Syncopy v2022.08
New Features
- Added down- and resampling algorithms for the new meta-function
resampledata
- Added FOOOF method as a post-processing option for the freqanalysis method mtmfft.
- Added
load_tdt
to import data from the TDT system, thanks to @kajal5888 - Added
.info
attribute for all data classes to store auxiliary meta information - Added
zscore
normalization topreprocessing
- new global
spy.copy()
function which copies entire Syncopy objects on disk
Changes
- the
out.cfg
attached to an analysis result now allows to replay all analysis methods connectivityanalysis
now has FT compliant output support for the coherencespy.cleanup
now has exposedinteractive
parameter- removed keyword
deep
fromcopy()
, all our copies are in fact deep - demeaning after tapering for granger analysis
- detrending is now possible without filtering in
preprocessing
Fixes
out.cfg
global side-effects (sorry again @kajal5888)CrossSpectralData
plotting- mixing of explicit keywords and
cfg
to control analysis - fixed error on initializing SpikeData with empty ndarray (#257)
Detailed contributions and changes
- FIX: nwb event dimord by @KatharineShapcott in #282
- CHG: Capture explicit default settings if also in
cfg
by @tensionhead in #284 - Add Resampling by @tensionhead in #289
- Frontend Plotting tests by @tensionhead in #293
- CHG: FT compatibility for coherence outputs by @tensionhead in #294
- 297 providing the user an option for non interactive cleanup by @tensionhead in #298
- 291 show select by @tensionhead in #301
- 140 sub remove hdr memmap by @dfsp-spirit in #302
- 209 cfg by @tensionhead in #305
- 290 resampling tests by @tensionhead in #309
- 140 multiple outputs by @dfsp-spirit in #311
- CHG: Rework copy by @tensionhead in #312
- Dev by @tensionhead in #314
- 224 fooof implementation by @dfsp-spirit in #307
- 308 fooof docs by @dfsp-spirit in #316
- FIX: prevent error on init of SpikeData with empty ndarray by @dfsp-spirit in #317
- 303 auxiliary meta info container by @tensionhead in #315
- 318 migrate build system to poetry by @tensionhead in #319
- Fix tox win by @tensionhead in #323
- Dev updates for (pre-)release by @tensionhead in #324
- Tdt reader by @kajal5888 in #275
- 325 upgrade preprocessing by @tensionhead in #326
- NEW: Added start_sample as .info key by @KatharineShapcott in #327
- Dev by @tensionhead in #330
- Resampling doc by @tensionhead in #329
- Dev by @tensionhead in #333
New Contributors
- @dfsp-spirit made their first contribution in #302
- @kajal5888 made their first contribution in #275
2022.05
[2022.05] - 2022-05-13
Bugfixes and features additions for EventData
objects.
NEW
- Added support for flexible columns in
EventData
(thanks to @KatharineShapcott)
CHANGED
- Include specific example how to create an "all-to-all"
trialdefinition
array
by invokingdefinetrial
without arguments in the function's docstring. - Modified versioning scheme: use a date-based scheme instead of increasing
version numbers - Aligned padding API to FieldTrip in both
freqanalysis
andconnectivityanalysis
:
usepad
instead ofpad_to_length
with three supported modes ('maxperlen',
float, 'nextpow2').
DEPRECATED
- Removed support for calling
freqanalysis
with atoi
array as well as an
input dataset that has an active in-place time-selection attached
FIXED
- Improved legibility of
spy.__version__
for non-release installations - Correctly process equidistant
toi
arrays with large spacing infreqanalysis
- Corrected
trialtime
forDiscreteData
objects (thanks to @KatharineShapcott)
[v0.21] - 2022-04-13
Feature update and bugfixes.
NEW
- Added preprocessing functionality
- Added experimental loading functionality for NWB 2.0 files
- Added experimental loading functionality for Matlab mat files
- Added support for "scalar" selections, i.e., things like
selectdata(trials=0)
ordata.selectdata(channels='mychannel')
- Added command line argument "--full" for more granular testing: the new default
for running the testing pipeline is to execute a trimmed-down testing suite that
does not probe all possible input permutations but focuses on the core functionality
without sacrificing coverage. - New meta-function
taper_opt
parameter to control arbitrary taper (e.g. kaiser)
parameters
CHANGED
- Renamed
_selection
class property toselection
- Reworked plotting framework and made it matplotlib 3.5 compatible
- The output of
show
is now automatically squeezed (i.e., singleton dimensions
are removed from the returned array). - Enhanced online documentation, now also covering connectivity analysis
- Multi-tapering (
freqanalysis
,connectivityanalysis
) now is switched on by
only specifying thetapsmofrq
parameter, removed the need for the additional
and redundant setting oftaper='dpss'
- Granger-Geweke algorithm now matches the reference implementation (Dhamala 2008)
with machine precision
REMOVED
- Do not parse scalars using
numbers.Number
, usenumpy.number
instead to
catch Boolean values - Do not raise a
SPYTypeError
if an arithmetic operation is performed using
objects of different numerical types (real/complex; closes #199)
DEPRECATED
- Removed loading code for ESI binary format that is no longer supported
- Repaired top-level imports: renamed
connectivity
toconnectivityanalysis
and the "connectivity" module is now called "nwanalysis" - Included
conda clean
in CD pipeline to avoid disk fillup by unused conda
packages/cache - Inverted
selectdata
messaging policy: only actual on-disk copy operations
trigger aSPYInfo
message (closes #197) - Matched selector keywords and class attribute names, i.e., selecting channels
is now done by using aselect
dictionary with key'channel'
(not'channels'
as before). See the documentation ofselectdata
for details. - Retired Travis CI tests since free test runs are exhausted. Migrated to GitHub
actions (and re-included codecov)
FIXED
- The
trialdefinition
arrays constructed by theSelector
class were incorrect
forSpectralData
objects without time-axis, resulting in "empty" trials. This
has been fixed (closes #207) - Repaired
array_parser
to adequately complain about mixed-type arrays (closes #211) - The
show
routine now consistently returns a list of trials if and only if
multiple trials are selected
0.2
[v0.2] - 2022-01-18
Major Release
NEW
- Added Connectivity submodule with
csd
,granger
andcoh
measures - Added new
CrossSpectralData
class for connectivity data - Added Superlet spectral estimation method to
freqanalysis
- Added arithmetic operator overloading for SyNCoPy objects: it is now possible
to perform simple arithmetic operations directly, e.g.,data1 + data2
. - Added equality operator for SyNCoPy objects: two objects can be parsed for
identical contents using the "==" operator - Added full object padding functionality
- Added support for user-controlled in-place selections
- Added
show
class method for easy data access in all SyNCoPy objects - Added de-trending suppport in
freqanalysis
via thepolyremoval
keyword - New interface for synthetic data generation: using a list of NumPy arrays for
instantiation interprets each array asnChannels
xnSamples
trial data
which are combined to generate aAnalogData
object - Made SyNCoPy PEP 517 compliant: added pyproject.toml and modified setup.py
accordingly - Added IBM POWER testing pipeline (via dedicated GitLab Runner)
CHANGED
- Multi-tapering now works with smoothing frequencies in Hz
- Streamlined padding interface
REMOVED
- Retired tox in
slurmtest
CI pipeline in favor of a "simple" pytest testing
session due to file-locking problems of tox environments on NFS mounts
DEPRECATED
- Removed ACME from source repository: the submodule setup proved to be too
unreliable and hard to maintain. ACME is now an optional (but recommended)
dependency of SyNCoPy
FIXED
- Non-standard
dimord
objects are now parsed and processed byComputationalRoutine
- Impromptu padding performed by
freqanalysis
is done in a more robust way - Stream-lined GitLab Runner setup: use cluster-wide conda instead of local
installations (that differ slightly across runners) and leveragetox-conda
to fetch pre-built dependencies
v0.1b1
[v0.1b1] - 2020-10-23
First public pre-release of SyNCoPy on PyPI and GitHub.
NEW
- Included
selectdata
as acomputeFunction
that uses the parallelization
framework inComputationalRoutine
to perform arbitrary data-selection tasks
(including but not limited to unordered lists, repetitions and removals). - Included time-frequency analysis routines
mtmconvol
andwavelet
- Added plotting functionality: functions
singlepanelplot
andmultiplanelplot
allow quick visual inspection ofAnalogData
andSpectralData
objects - Added support to process multiple SyNCoPy objects in a single meta-function
call (all decorators have been modified accordingly) - Introduced standardized warning messages via new class
SPYWarning
- Included (more or less) extensive developer docs
- Added Travis CI and included badges on GitHub landing page
- New convenience scripts to ease developing/testing
- New conda.yml file + script for consolidating conda/pip requirements: all
of SyNCoPy's dependencies are now collected insyncopy.yml
, the respective
pip-specific requirements.txt and requirements-test.txt files are generated
on the fly by a new functionconda2pip
that relies on ruamel.yaml (new
required dependency for building SyNCoPy) - New GitLab CI directive for uploading SyNCoPy to PyPI
- Included GitHub templates for new issues/pull requests
- SyNCoPy docu is now hosted on readthedocs (re-directed from syncopy.org)
- New logo + icon
CHANGED
- Made
cluster_cleanup
more robust (works withLocalCluster
objects now) - Made data-parser more feature-rich: check for emptiness, parse non-data
datasets etc. - Made
generate_artificial_data
more robust: change usage of random number
seed to allow persistent comparisons across testing runs - Updated CI dependencies (SyNCoPy now requires NumPy 1.18 and Matplotlib 3.3.x)
REMOVED
- All *.py-file headers have been removed
- Removed examples sub-module from main package (examples will be part of a
separate repo)
DEPRECATED
- Wiped all hand-crafted array-matching routines; use
best_match
instead - Do not use
pbr
in the build system any more; rely instead on up-to-date
setuptools functionality - Retired memory map support and raw binary data reading routines
FIXED
- Improved temporary storage handling so that dask workers that import the
package do not repeat all temp-dir I/O tests (and potentially run into
dead-locks or race conditions)