NEWS.Rmd

---
output: github_document
header-includes:
   - \usepackage{amsmath}
   - \usepackage{amssymb}
---
<!-- NEWS.md was auto-generated by NEWS.Rmd. Please DO NOT edit by hand!-->

# Changes in version `r RLumBuild::.get_pkg_version()` (`r Sys.Date()`)

## New functions

* `calc_EED_Model()` models incomplete and heterogeneous bleaching of
mobile grains after Guibert et al. (2017). Along with the function, the
new `ExampleData.MortarData` data set was added.

* `fit_DoseResponseCurve()` and `plot_DoseResponseCurve()` are two new
functions derived from `plot_GrowthCurve()`: the first only performs the
fitting of a dose-response curve, while the second plots it. This brings
greater flexibility and increased speed, as plotting is now independent of
fitting (#141, fixed in #318).

* `melt_RLum()`: Creates a new flat `data.frame` from the objects that can be
used for instance in combination with `'ggplot2'`. Works only on `RLum.Data.Curve-class`
and `RLum.Analysis-class` objects and lists of such objects.

* `merge_RLum.Data.Spectrum()`: This new function allows to merge two or
more `RLum.Data.Spectrum` objects in different ways (#368, fixed in #419).

* `add_metadata()`, `rename_metadata()`, `replace_metadata()`:
These function allow to manipulate the metadata of
`Risoe.BINfileData`, `RLum.Analysis` and `RLum.Data` objects (#480, fixed
in #514, #524, #525, #527, #534, #545).

* `view()`: Provides a shortcut to the `utils::View()` spreadsheet-like data
viewer tailored to the objects in the package (#489, fixed in #490).

## Breaking changes
* We have dropped our dependency on the `readxl` package: functions
`analyse_baSAR()` and `use_DRAC()` now do not accept XLS files anymore but
CSV files instead (#237, fixed in #270). CSV files can be easily generated
from XLS files by Excel or similar applications, or by reading them with
`readxl::read_excel()` and saving them with `write.csv()`.

* The `plot.single` option, which was available for several functions,
sometimes under a slightly different name, has now been renamed to
`plot_singlePanels` for overall consistency and clarity. Function
`scale_GammaDose()` used that option but gave it the opposite meaning: now
also this function conforms to the rest of the package. The use of the
older names is now deprecated and will produce a warning (#351, fixed in #408).

* The `NumberIterations.MC` option of `plot_GrowthCurve()` has now been
renamed to `n.MC` for consistency with other functions. Potentially affected
are also `analyse_Al2O3C_ITC()`, `analyse_baSAR()`, `analyse_SAR.CWOSL()`,
`analyse_SAR.TL()`, `calc_Huntley2006()`, `calc_Lamothe2003()`,
as they may pass that option `plot_GrowthCurve()` via their `...` argument.
The use of the older name is now deprecated and will produce a warning (#546,
fixed in #547).

## Removed functions and deprecations
* Function `Analyse_SAR.OSLdata()` is now officially deprecated,
`analyse_SAR.CWOSL()` should be used instead (#216, fixed in #264).

* Functions `CW2pHMi()`, `CW2pLM()`, `CW2pLMi()` and `CW2pPMi()` have been
renamed to `convert_CW2pHMi()`, `convert_CW2pLM()`, `convert_CW2pLMi()`
and `convert_CW2pPMi()`, respectively. The use of the older names is now
deprecated and will produce a warning (#499, fixed in #503).

* Function `Second2Gray()` has been renamed to `convert_Second2Gray()`
(#498, fixed in #500).


## Bugfixes

### `analyse_Al2O3C_CrossTalk()`
* The function doesn't crash anymore if option `recordType` is specified and
none of the records in the data set is of that type (#487, fixed in #488).

### `analyse_baSAR()`
* Argument `XLS_file` has been replaced by `CSV_file` and, as mentioned
above, the function now only accepts CSV files as input (#237, fixed in #270).
* Add support for `recordType` passed to `get_RLum` in the additional arguments.
* Option `plot.single` has been renamed to `plot_singlePanels` (#351, fixed
in #408).
* The function crashed if the number of MCMC iterations was set equal to the
thinning interval; on the other hand, we saw errors reported from JAGS if the
number of MCMC iterations was not at least double the thinning interval. To
address this, we slightly retouched our automatic setting of the thinning
interval (which was problematic only if the number of MCMC iterations was
extremely low); on the other hand, if a user sets a thinning interval that
is too high, we now reset it to a lower value and raise a warning  (#407,
fixed in #409).
* The function crashed if a `Risoe.BINFileData-class` object was provided
with and the argument `irradiation_times` was set; fixed.
* The function was supposed to remove non-OSL curves from the dataset, well
it almost never did; fixed.
* The function is more robust against input that may have been subset
inconsistently (#517, fixed in #518).
* The function doesn't crash but reports more helpful messages in case the
user overrides the default set of monitored variables using `variable.names`
within the `method_control` argument (#521, fixed in #522).

### `analyse_SAR.CWOSL()`
* The function crashed for a list input that led to `NULL` for various
reasons (e.g., unsuitable set of curves). Here the self-call attempted to
extract information from the results that did not exist in the first place
instead of returning `NULL`; fixed.

### `analyse_FadingMeasurement()`
* The function now checks for the version of the BIN-file that originated the
`RLum.Analysis` object given as input, and reports a message if a version
older than 5 was used (#281, fixed in #282).
* The function doesn't crash anymore on some `RLum.Analysis` input files
(#283, fixed in #288).
* Option `plot.single` has been renamed to `plot_singlePanels` (#351, fixed
in #408).
* The function doesn't crash anymore if no record is left after removal of
records with negative time since irradiation (#454, fixed in #456).

### `analyse_IRSAR.RF()`
* The performance of this function has been improved for the "SLIDE" and
"VSLIDE" methods, thanks both to tweaks in the C++ implementation of the
sliding algorithm and in how that is managed on the R side. In particular,
the default number of sliding windows tested in the algorithm has been
reduced from 10 to 3: this value is no longer hardcoded, but can be tuned
through the new `num_slide_windows` setting (part of the `method.control`
option), thus allowing to find a balance between computation time and
quality of fit (#372, fixed in #388, #398 and #399).
* The function is more robust against `sequence_structure` misspecifications
(#393, fixed in #394).
* Some data preparation steps where not correctly applied for `method = "VSLIDE"`
(#396, fixed in #397).
* Option `method.control` has been renamed to `method_control` for consistency
with all other functions in the package (#411, fixed in #412).

### `analyse_pIRIRSequence()`
* Option `plot.single` has been renamed to `plot_singlePanels` (#351, fixed
in #408).

### `analyse_SAR.CWOSL()`
* Option `plot.single` has been renamed to `plot_singlePanels` (#351, fixed
in #408).
* The function shifted the curves correctly in cases where the first channel
is zero and a log-transformation on the x-axis is requested (#580, fixed in
#531).

### `apply_CosmicRayRamoval()`
* Support has been added for the internal function `smooth_RLum` that can 
be passed as new argument for `method`. 

### `calc_AverageDose()`
* The function crashed if the data contained non-positive values: now these
are removed before proceeding (#532, fixed in #533).

### `calc_CentralDose()`
* Argument `na.rm` is now deprecated: the function will now always remove
missing values, as otherwise the presence of any `NA` would propagate and
produce unusable results (all `NA`s) or buggy behaviour (#302, fixed in #304).
* The function stops the fixed-point iteration for the computation of the
profile log-likelihood as soon as `sigma < 1e-16`, as allowing `sigma` to
become zero leads to infinities and buggy behaviour (also fixed in #304).

### `calc_FastRatio()`
* A number of crashes related to input validation have been fixed (#471,
fixed in #472).

### `calc_gSGC()`
* The function crashed if the data contained negative errors: to avoid this,
we now take the absolute values of errors (#479, fixed in #481).

### `calc_Huntley2006()`
* Some crashes in case of model misspecification have been solved (#538,
fixed in #539).
* Some details in the implementation of the function have been optimized, and
now it is much faster than before. As part of this, we have changed the default
setting of the `rprime` vector that is used in the calculation of the natural
dose response and the field saturation, so that more points are concentrated
in the bulk of the distribution: this previously depended incorrectly on the
number of Monte Carlo iterations requested, so this change brings an additional
speed boost. The default setting can be overridden via the `rprime` argument
(#258, fixed in #541 and #542).
* Fitting the "GOK" model on the unfaded data failed when the "EXP" model we
use to find a good starting point failed: in such cases, we try again using
the simulated fit (#549, fixed in #500; thanks to @SalOehl for reporting and
providing data to reproduce the error).
* The natural/simulated curve was not plotted in the negative quadrant for
`mode = "extrapolation"`, which made it harder to see where it extrapolated
to (#551, fixed in #552; thanks to @SalOehl for reporting).


### `calc_IEU()`
* The code of this function has been consolidated to avoid duplication and
make its debugging easier: this has uncovered a small coding error and also
led to some speed up (#429, fixed in #430, #431 and #432).
* Some crashes in the function have been solved (#424, fixed in #433).

### `calc_Lamothe2003()`
* We addressed a long-standing issue regarding the calculation of the `Ln/Tn`
error after fading correction, which led to smaller than expected errors
(#96, fixed in #296).

### `convert_RLum2Risoe.BINFileData()`
* The function could crash with a `incompatible types (from raw to character)` error
under particular circumstances; fixed. 
* The function would also fail in edge cases where different objects are mixed 
so that certain fields for `METADATA` are already available, however, in the wrong 
format; fixed

### `extract_IrradiationTimes()`
* In some circumstances the function failed to update an existing BIN/BINX
file due to an incorrect way of subsetting an intermediate data frame of
results (#445, fixed in #446).
* If the info element `startDate` was missing an in the irradiation curve the function 
crashed unexpectedly; fixed. 

### `fit_CWCurve()`
* The function crashed if the computation of the confidence intervals done
in `confint()` failed. This has now been fixed, and in cases of failures we
report the error message received from `confint()` (#509, fixed in #510).

### `get_RLum()`
* If `get_RLum()` with `subset` was used on info objects of an `RLum.Analysis-class` object it
would return wrong results if the info element was not present in all records; fixed. 

### `import_Data()`
* The function automatically determines the available import functions and ensures
that formats supported by the package are tested and used.
* A minor change in the code enables the support of `character` vector of length > 1 
as input (e.g., different file names). Further, the function got tested against a mixture
of input formats. Means, you can provide many different files formats and they function 
will try to import them all in one go. 

### `plot_AbanicoPlot()`
* If negative values were provided, the function sometimes produced wrong plots
for `z.log = TRUE` (the default); fixed.
* Remove support for `tcl` and `tck` arguments via `...`, they were never used
internally.

### `plot_DetPlot()`
* Option `plot.single` has been renamed to `plot_singlePanels` (#351, fixed
in #408).

### `plot_DRTResults()`
* The function crashed if option `na.rm` was used alongside the `preheat`
option but the data set contained no missing values (#474, fixed in #475).

### `plot_GrowthCurve()`
* The function is now implemented as a wrapper around `fit_DoseResponseCurve()`
and `plot_DoseResponseCurve()`. There should be no visible user-facing
changes (#319, fixed in #322).
* Support new `...` arguments for plotting (implementation in `plot_DoseResponseCurve()`) for
`legend` (turn on/off legend) and `reg_points_pch` for fine grained control over the point shape.
* Add `density_polygon`, `density_pologyon_col`, `density_rug`, `box` as `...` arguments to plot and
add respective plot options
* The function crashed for single grain data that sometimes cause a lot of
`NaN` values during log conversions required for estimating the start
parameters. The result was the error `object 'b.MC' not found`
(#374; fixed)
* For the non-linear choices of `fit.method` (`"EXP+LIN"`, `"EXP+EXP"`,
`"GOK"` and `"LambertW"`), the function could get stuck in an endless
loop when the number of parameters was larger than or equal to the number of
data points. This is now checked, and in those cases the method is changed
to `"LIN"` (#381, fixed in #465).
* We replaced `nls()` with `minpack.lm::nlsLM()` as it's more robust and can
find a fit in cases where `nls()` would fail. Please keep in mind that it
does not mean that fit results in those cases make any sense,
it just helps to avoid uncontained events in the R session (addresses #381).
* Option `output.plotExtended.single` has been renamed to `plot_singlePanels`
(#351, fixed in #408).
* The function now supports a list of data frames as input, in which case it
calls itself on each element of the list and produces a list of `RLum.Results`
as output (#405, fixed in #434).
* The function reported negative dose values in the MC runs plot when using
`fit.method = "QDR"` and `mode = "extrapolation"` (#504, fixed in #505).
* Argument `NumberIterations.MC` has been renamed to `n.MC` (#546, fixed in #547).

### `plot_RLum.Analysis()`
* Option `plot.single` has been renamed to `plot_singlePanels` (#351, fixed
in #408).

### `plot_RLum.Data.Curve()`
* Argument `norm` is now better validated so that specifying an incorrect
value returns an error instead of silently skipping curve normalisation (#250,
fixed in #263).

### `plot_RLum.Data.Spectrum()`
* The gradient between colours when `plot.type = "persp"` is now more
consistent and doesn't produce artefacts in the regions of transition
between colours (#371, fixed in #380).
* The function would crash if the `bin.rows` or `bin.cols` value was set
too high or if `xlim` and `ylim` were set too tight (#415, fixed in #416).

### `read_BIN2R()`
* The function is now much faster: we have recorded an overall improvement
of over 60% on a 46M file, with potentially bigger time savings for larger
files (#298, fixed in #333, #355 and #360).
* Argument `n.records` is now better supported for BIN files v3 and v4 and
doesn't lead to a crash when used in conjunction with the `fastForward`
option (#343, fixed in #344).
* Argument `zero_data.rm` was not correctly propagated if the input object
was provided as a list (#342, fixed in #345).
* The `METADATA` fields that are not actually read from the BINX file in
case of a ROI record are now set to `NA` rather than being assigned the
value from the previous record (#336, fixed in #360).
* The `.RESERVED` slot is now kept in sync with the rest of the object
when records are dropped from the input BIN/BINX file (#337, fixed in #348).
* The function doesn't crash anymore when reading a file that contains a
record with an unrecognised version after other records with a valid version
have been read (#352, fixed in #353).
* The function doesn't crash anymore when reading a file that contains a
ROI record while using `fastForward = TRUE` (#356, fixed in #360).

### `read_HeliosOSL2R()`
* If the function encounters problems in the provided list of files, it shows
a more relaxed handling; throws an error but continues reading. 

### `read_XSYG2R()`
* The new `n_records` argument now enables to control the number of records
to import, which can be useful in case the file is faulty.
* The function sometimes struggled with the import if only a directory was
provided; fixed.
* The function truncated the terminal output under certain circumstances; fixed.

### `scale_GammaDose()`
* Option `plot_single` has been renamed to `plot_singlePanels` and was changed
so that `plot_singlePanels = TRUE` will produce one plot per panel, as it
would happen for other functions (#351, fixed in #408).

### `template_DRAC()`
* The function gained a new argument `file_input` that enables the creating of a DRAC template
based on an existing DRAC CSV file. This way, already available DRAC files
can be conveniently modified without having to call `use_DRAC()` first.
* The template contained a couple of minor mistakes that got fixed along with a
more strict enforcement of the class of each field, which is now set as an attribute;
the change should have no user-visible effect.
* The function now supports the conversion factors by Cresswell et al. (2018)
(`Cresswelletal2018`).
Please ensure that the server supports that option.

### `trim_RLum.Data()`
* The function has been made more robust against misspecifications of its
`trim_range` argument that could lead to crashes (#460, fixed in #461).

### `use_DRAC()`
* Support for DRAC v1.1 XLS/XLSX files has been dropped, users should use
CSV files according to the DRAC v1.2 CSV template.
* The function gained support for three new `...` arguments: `version_ignore`
to ignore the DRAC version check, `user` and `password` to access password
protected websites.
* The identifiers submitted to the DRAC server are now better randomized to
to guarantee more confidentiality of the data transmitted (#435, fixed in #438).

### `verify_SingleGrainData()`
* **Potentially breaking old code!**: if `cleanup = TRUE` the result was not
consistent and produced a mixture of `RLum.Results` and
`RLum.Analysis`/`Risoe.BINFileData` objects in particular when run over a
list, this was not what stated in the manual. Now the function indeed cleans
up the object, hence if a list of `RLum.Analysis` objects is provided, the
result is a cleaned list of that object. If the clean-up means that the
everything was removed, the result is `NULL`. * Improve warning messages; it
was indeed not very intelligible.
* The function doesn't crash anymore when applied to an empty list (#365,
fixed in #366).
* Silence the very talkative terminal output even if nothing was removed
from a record for `cleanup = TRUE`.
* The function now provides better terminal feedback if the selection results
in an empty record.
* Add a new logical argument `use_fft` that will apply an additional Fast
Discrete Fourier Transform (FFT) verification on the data if wanted.

### `write_R2BIN()`
* The function now supports the `verbose` argument, which can be used to
suppress output to the terminal (#442, fixed in #444).

## Internals

* `get_DRAC_references()` now works also on a `DRAC.list` got updated references.