This site is a place to get help or exchange information about the drEEM toolbox - used to support visualisation and PARAFAC analysis of EEM datasets.

The toolbox was published in conjuction with the following reference:

Murphy K.R., Stedmon C.A., Graeber D. and R. Bro, Fluorescence spectroscopy and multi-way techniques. PARAFAC, Anal. Methods, 2013,DOI:10.1039/c3ay41160e. Appendix A contains the drEEM 0.1.0 tutorial.

Get drEEM, the tutorial, and the demo dataset here: http://www.models.life.ku.dk/drEEM. Get the paper here: Link to paper, open access

What is the drEEM toolbox?

The drEEM toolbox (decomposition routines for Excitation-Emission-Matrices) is used to assess composition of any subject measured by 3D-fluorescence measurements (excitation-emission-matrices, EEMs) in combination with parallel factor analysis (PARAFAC). A typical example is chromophoric dissolved organic matter (CDOM) in natural an artificial aquatic environments.

The drEEM toolbox combines two toolboxes: the DOMFluor toolbox which introduced several algorithms to define and test the right number of components of a PARAFAC model1) and the FDOMcorr toolbox which conducts the neccessary pre-processing steps to make the fluorescence data comparable between instruments and studies2). The procedures of these two toolboxes are furthermore extended by additional visualisation functions and other functions to support and simplify modelling of Excitation-Emission-Matrices (EEMs) of natural CDOM with PARAFAC .

Chromophoric dissolved organic matter (CDOM)

CDOM is the part of dissolved organic matter which is analysable by spectroscopic measurements3).

Since nearly all natural dissolved organic matter contains CDOM, fluorescence measurements are unrivaled as a fast assessment tool of dissolved organic matter composition. For example, it was used to assess the effects of different treatments of municipal waste water on dissolved organic matter composition4), the effect of agricultural land use on dissolved organic matter composition in streams5) and the sources of dissolved organic matter in an estuary6). In the current literature, many more examples of the application of fluorescence measurements for assesment of dissolved organic matter composition can be found7).

Parallel factor analysis (PARAFAC)

 EEM dataset arranged in a threeway structure and decomposed into five PARAFAC components. Figure taken from Murphy K.R., Stedmon C.A., Graeber D. and R. Bro, Fluorescence spectroscopy and multi-way techniques. PARAFAC, Anal. Methods, 2013,DOI:10.1039/c3ay41160e

PARAFAC8)9) belongs to a family of so-called multi-way methods applicable to data that are arranged in three- or higher-order arrays. Examples of three-way arrays that can be analysed with PARAFAC include fluorescence EEMs (sample × excitation wavelength × emission wavelength), chromatographic data (GC-MS: sample × elution time × m/z structure), sensory data (sample × attribute × judge) and electroencephalography (space × time × frequency). PARAFAC of a three-way dataset decomposes the data signal into a set of trilinear terms and a residual array:

 where i = 1, ..., I; j = 1, ... , J; k = 1, ..., K. PARAFAC of a three-way dataset decomposes the data signal into a set of trilinear terms and a residual array. Figure taken from Murphy K.R., Stedmon C.A., Graeber D. and R. Bro, Fluorescence spectroscopy and multi-way techniques. PARAFAC, Anal. Methods, 2013,DOI:10.1039/c3ay41160e

, where i = 1, …, I; j = 1, … , J; k = 1, …, K. In this equation, xijk is the data point corresponding to the ith sample at the jth variable on mode 2 and the kth variable on mode 3 and eijk is the residual representing the variability not accounted for by the model. In the case of a fluorescence excitation-emission matrix, the i, j and k correspond to the sample, emission and excitation modes, respectively. Each f corresponds to a PARAFAC component and each such component has I a-values (scores); one for each sample. Each component also has J b-values; one for each emission wavelength as well as K c-values; one for each excitation wavelength. For the special case of a dataset of fluorescence EEMs, this is shown in the following figure:

Please see Murphy K.R., Stedmon C.A., Graeber D. and R. Bro, Fluorescence spectroscopy and multi-way techniques. PARAFAC, Anal. Methods, 2013,DOI:10.1039/c3ay41160e for further details on PARAFAC and the modelling process ( Link to paper, open access).

Version 0.2.0

What is new?

(1) Now compatible with MAC/Linux operating systems:

readineems
readinscans
fdomcorrect
modeloutmac  (*MAC/Linux alternative to modelout, requires xlwrite from MATHWORKS file exchange)

(2) Now compatible with AquaLog fluorometer files:

readineems
readinscans
For example, create a cube of AquaLog EEMs (X) and an EEM dataset (DS):
[X,Emmat,Exmat,fl,outdata]=readineems(3,'*Waterfall Plot Sample.dat','A2..BV126',[0 1],0,2);
DS = assembledataset(X,Exmat(1,:),Emmat(:,1),'AU','filelist',fl,[]);

(3) Other changes/enhancements:

 (a) fdomcorrect

- outputs to .dat by default (Excel is optional)

   - exports sample names if you use DS not X as the input variable (see above example)
   - faster and more accurate calculation of Raman Areas
   
 (b) randinitanal
   - selection of the best LS model excludes models that did not converge
  
 (c) outliertest
   - fixed bug that produces wrong nSample value in the case that nSample< max(nEx,nEm)
   
 (d) splitds
   - fixed bug that causes failure to recognise split types in newer MATLAB versions

(4) New EEM correction demo mfile

drEEM_demo_020
   

(4) New functionality:

openfluor - export model loadings ready for direct upload to OpenFluor (www.openfluor.org)
readlogfile - import a log file (*.csv) into a data structure (Windows/Mac/Linux)
alignds - align samples on the basis of the above log file (Windows/Mac/Linux)
ABAife - correct for inner filter effects by absorbance method in 1 cm cell
CDAife - correct for inner filter effects by controlled dilution approach (Luciani et al. 2009)
IFEtest_demo - comparison of inner filter effect correction methods (Kothawala et al. 2013)

(5) Obselete functions

  matchsamples - Use readlogfile and alignds instead of matchsamples

Bugs and patches

Please report bugs to dreem@openfluor.org (will be received by Kathleen R. Murphy and Daniel Graeber) so that these can be included in this list and be removed as soon as possible. Please report also the version of drEEM you can find the version in the output, when you type help drEEM in Matlab. Then text similar to the following will appear (only first four lines shown):

 
>> help drEEM
  The drEEM Toolbox for MATLAB (ver. 0.2.0)
  Version 0.2.0 June-2014
  Download from http://www.models.life.ku.dk/
  ...

Issues with different operating systems or missing MS Excel

Using a Mac/Linux or don't have MS Excel? Please use drEEM version 0.2.0. In this version the issues should be solved. Send an email to dreem@openfluor.org, if there are still problems.

Errors when using drEEM version 0.1.0:

- Error when running matchsamples and modelout functions for Mac/Linux systems because of incompatibility of xlsread and xlswrite functions which are used within these functions.

- Error when running matchsamples and modelout functions when Microsoft Excel is not installed because of lacking Windows drivers for xlsread and xlswrite functions which are used within these functions.

- readineems and readinscans cannot be run with parameter format = 'xls' or format = 'xlsx', when the above-mentioned conditions are met.

Issues with older Matlab versions

- fdomcorrect does not work with Matlab versions before 7.13 (R2011b). The reason is the missing function narginchk. Please download the function here: https://gist.github.com/hagenw/5642886

- In Matlab versions before R2009B, readineems and other functions will not work, because earlier versions did not use the ”~” for a dummy variable as for example in:

[~, j1]=min(abs(Ex-inpair(i,1)));

A workaround which may work but still needs to be tested is to substitute ”~” by “dummy”, for the example above it would be:

[dummy, j1]=min(abs(Ex-inpair(i,1)));

Issue with outliertest function, drEEM version 0.1.0

Please use drEEM version 0.2.0, if this issue occurs.

- Symptoms: eemview and compare2models produce an error when trying to plot Test models created with outliertest.m

- Cause: bug in outliertest.m on line 119 produces wrong nSample value in the case that nSample< max(nEx,nEm)

- Solution: The code on line 119 of outliertest.m reads as follows:

Test.nSample=length(Test.X);

Replace the code on this line with the following:

Test.nSample=size(Test.X,1);  %Test.nSample=length(Test.X); %BUG FIXED 2013/12/20
1) Stedmon, C. and Bro, R. Characterizing dissolved organic matter fluorescence with parallel factor analysis: a tutorial. Limnol Oceanogr-Meth, 2008, 6, 572-579
2) Murphy, K.; Butler, K.; Spencer, R.; Stedmon, C.; Boehme, J. & Aiken, G. Measurement of dissolved organic matter fluorescence in aquatic environments: an interlaboratory comparison. Environ Sci Technol, 2010, 44, 9405-9412
3) , 7) Murphy K.R., Stedmon C.A., Graeber D. and R. Bro, Fluorescence spectroscopy and multi-way techniques. PARAFAC, Anal. Methods, 2013,DOI:10.1039/c3ay41160e
4) Murphy, K. R.; Hambly, A.; Singh, S.; Henderson, R. K.; Baker, A.; Stuetz, R. & Khan, S. J. Organic matter fluorescence in municipal water recycling schemes: toward a unified PARAFAC model Environmental science & technology, ACS Publications, 2011, 45, 2909-2916
5) Graeber, D.; Gelbrecht, J.; Pusch, M.; Anlanger, C. & von Schiller, D. Agriculture has changed the amount and composition of dissolved organic matter in Central European headwater streams Science of The Total Environment, Elsevier, 2012, 438, 435-446
6) Stedmon, C. & Markager, S. Resolving the variability in dissolved organic matter fluorescence in a temperate estuary and its catchment using PARAFAC analysis Limnol. Oceanogr., 2005, 50, 686-697
8) content of section taken from Murphy K.R., Stedmon C.A., Graeber D. and R. Bro, Fluorescence spectroscopy and multi-way techniques. PARAFAC, Anal. Methods, 2013,DOI:10.1039/c3ay41160e
9) A. R. Bro, Chemom. Intell. Lab. Syst., 1997, 38, 149–171. B. R. A. Harshman, UCLA Working Papers in Phonetics, University Microfilms, Ann Arbor, no. 10,085, 1970, vol. 16, pp. 1–84.
 
start.txt · Last modified: 2014/06/13 17:08 by daniel
 
Except where otherwise noted, content on this wiki is licensed under the following license:CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki