aboutsummaryrefslogtreecommitdiff
path: root/processing/chapter.tex
diff options
context:
space:
mode:
Diffstat (limited to 'processing/chapter.tex')
-rw-r--r--processing/chapter.tex204
1 files changed, 119 insertions, 85 deletions
diff --git a/processing/chapter.tex b/processing/chapter.tex
index 0d89055..1d11788 100644
--- a/processing/chapter.tex
+++ b/processing/chapter.tex
@@ -507,8 +507,8 @@ WrightTools. %
\texttt{wt.data.from\_shimamdzu} & Shimadzu brand spectrometers. \\ \hline
\texttt{wt.data.from\_Tensor27} & Tensor 27 FT-IR. \\ \hline
\end{tabular}
- \caption[CAPTION TODO]{
- CAPTION TODO
+ \caption{
+ WrightTools data import functions.
}
\label{pro:tab:from_functions}
\end{table}
@@ -586,10 +586,11 @@ After importing and manipulating data, one typically wants to create a plot. %
The artists sub-package contains everything users need to plot their data objects. %
This includes both ``quick'' artists, which generate simple plots as quickly as possible, and a
full figure layout toolkit that allows users to generate full publication quality figures. %
-It also includes ``specialty'' artists which are made to perform certain popular plotting
-operations, as I will describe below. %
+It also includes ``specialty'' artists which are made to perform certain common plotting
+operations. %
-Currently the artists sub-package is built on-top of the wonderful matplotlib library. %
+Currently the artists sub-package is built on-top of the wonderful matplotlib library
+\cite{HunterJohnD2007a}. %
In the future, other libraries (e.g. Mayavi \cite{Mayavi}), may be incorporated. %
\subsection{Strategies for 2D visualization} % ---------------------------------------------------
@@ -598,19 +599,36 @@ Representing two-dimensional data is an important capability for WrightTools, so
discussion about how such representations work is warranted. %
WrightTools data is typically very structured, with values recorded at a grid of positions. %
To represent two-dimensional data, then, WrightTools needs to map the values onto a color axis. %
-There are better and worse choices of colormap... % TODO: elaborate
+When doing this mapping, choosing an appropriate colormap and interpolation type is crucial. %
\subsubsection{Colormap}
+For the purposes of this discussion, there are two types of colormaps: 1. qualitative 2.
+perceptual. %
+Qualitative colormaps have random orderings of color. %
+They are best used to represent unordered things, and they typically have high dynamic range. %
+Perceptual colormaps are monotonic in lightness, and are best at representing ordered information
+(like signal levels in MR-CMDS). %
+
+Historically the Wright Group has used a qualitative colormap for all plotting. %
\autoref{pro:fig:cmaps} shows the red, green, and blue components of four different colormaps. %
-The black line is the net brightness of each color (larger value means lighter color). %
+The black line is the net lightness of each color (larger value means lighter color). %
Below each figure is a gray-scale representation of the corresponding colormap. %
The r, g, and b components are scaled according to human perception. % TODO: values, from where
-The traditional Wright Group colormap (derived from jet) is shown first. %
+The traditional Wright Group colormap (derived from jet) is shown first. i%
Following are two perceptual colormaps, cubehelix from \texcite{GreenDA2009a}, and viridis, the new
matplotlib default.
WrightTools uses the algorithm from \textcite{GreenDA2009a} to define a custom cubehelix colormap
with good perceptual properties and familiar Wright Group coloration. %
+The lightness is not perfectly straight as in viridis and cubehelix, but it is monotonic at
+least. %
+
+\autoref{pro:fig:cmap_comparison} compares the new (top row) and old (bottom row) colormaps in full
+color and greyscale. %
+While the old colormap shows a greater dynamic range, the new colormap is more perceptually
+uniform. %
+In my opinion, users should use logarithmic scaling of an appropriate base if more dynamic range is
+required, rather than using a non-perceptual colormap. %
\subsubsection{Interpolation type}
@@ -635,22 +653,27 @@ plots. %
\includegraphics[scale=0.5]{"processing/cubehelix_cmap"}
\includegraphics[scale=0.5]{"processing/viridis_cmap"}
\includegraphics[scale=0.5]{"processing/default_cmap"}
- \caption[CAPTION TODO]{
- CAPTION TODO}
+ \caption[Colormap lightness.]{
+ Comparison of four different colormaps.
+ R, G, and B components are each shown.
+ Black line is human-perceived lightness.
+ }
\label{pro:fig:cmaps}
\end{figure}
\begin{figure}
\includegraphics[width=\textwidth]{"processing/cmap_comparison"}
- \caption[CAPTION TODO]{
- CAPTION TODO}
+ \caption{
+ Colormap comparison.
+ }
\label{pro:fig:cmap_comparison}
\end{figure}
\begin{figure}
\includegraphics[width=\textwidth]{"processing/fill_types"}
- \caption[CAPTION TODO]{
- CAPTION TODO}
+ \caption{
+ Interpolation types in 2D plotting.
+ }
\label{pro:fig:fill_types}
\end{figure}
@@ -684,39 +707,40 @@ These functionalities are derived from \python{wt.Data.chop}, discussed further
\begin{figure}
\includegraphics[width=0.5\textwidth]{"processing/quick1D 000"}
\includepython{"processing/quick1D.py"}
- \caption[CAPTION TODO]{
- CAPTION TODO}
+ \caption[quick1D example.]{
+ Minimal example of the \python{quick1D} function.
+ }
\label{pro:fig:quick1D}
\end{figure}
\begin{figure}
\includegraphics[width=0.5\textwidth]{"processing/quick2D 000"}
\includepython{"processing/quick2D.py"}
- \caption[CAPTION TODO]{
- CAPTION TODO}
+ \caption[quick2D example.]{
+ Minimal example of the \python{quick2D} function.
+ }
\label{pro:fig:quick2D}
\end{figure}
% TODO: signed data (with and without dynamic_range=True)
-\subsection{API} % -------------------------------------------------------------------------------
+% \subsection{API} % -------------------------------------------------------------------------------
-The artists sub-package offers a thin wrapper on the default matplotlib object-oriented figure
-creation API. %
-The wrapper allows WrightTools to add the following capabilities on top of matplotlib:
-\begin{ditemize}
- \item More consistent multi-axes figure layout.
- \item Ability to plot data objects directly.
-\end{ditemize}
-Each of these is meant to lower the barrier to plotting data. %
-Without going into every detail of matplotlib figure generation capabilities, this section
-introduces the unique strategy that the WrightTools wrapper takes. %
+% The artists sub-package offers a thin wrapper on the default matplotlib object-oriented figure
+% creation API. %
+% The wrapper allows WrightTools to add the following capabilities on top of matplotlib:
+% \begin{ditemize}
+% \item More consistent multi-axes figure layout.
+% \item Ability to plot data objects directly.
+% \end{ditemize}
+% Each of these is meant to lower the barrier to plotting data. %
+% Without going into every detail of matplotlib figure generation capabilities, this section
+% introduces the unique strategy that the WrightTools wrapper takes. %
-% \subsection{Gotchas} % -------------------------------------------------------------------------
+% % \subsection{Gotchas} % -------------------------------------------------------------------------
% TODO: mention gotcha of apparently narrowing linewidths with wigners (how to READ colormaps)
-\clearpage
\section{Variables and channels} % ===============================================================
Data objects are made up of many component channels and variables, each array having the same
@@ -728,7 +752,7 @@ may not have even moved across some (or any) dimensions. %
To avoid unnecessarily large arrays, WrightTools allows Channels and Variables to have different
sizes than the parent data. %
As an example, consider the following object.
-\begin{codefragment}{bash} % TODO: need to use bash here because of box charachters :-(
+\begin{codefragment}{bash}
>>> import WrightTools as wt; from WrightTools import datasets
>>> data = wt.data.from_COLORS(datasets.COLORS.v2p1_MoS2_TrEE_movie)
>>> data.print_tree()
@@ -763,18 +787,34 @@ propagated. %
\section{Axes} \label{pro:sec:axes} % ============================================================
-The axes have the joint shape of their component variables. %
-Although not shown in this example, channels also may have axes with length 1.
+Axes are the primary interface to coordinate positions in WrightTools. %
+Axes are \emph{not} arrays, although they do behave like arrays. %
+They are merely \emph{interfaces} into the information stored in one or more variables. %
+
+Each axis has an expression, like \python{'w1'}, \python{'d1=d2'}, \python{2*w3} or
+\python{'w1+w2-wm'}. %
+These expressions describe an unambiguous mathematical operation involving one or more
+variables. %
+The axis computes these expressions on the fly when needed, giving users real flexability over how
+they would like to represent and process their results. %
Axes, variables, and channels are array-likes, so they support slicing operations. %
+The axes have the joint shape of their component variables. %
In addition, all three classes have \python{points} and \python{full} attributes that return the
squeezed and broadcasted array, respectively. %
+One of the coolest features enabled by this approach to axes is \emph{transformation}. %
+\autoref{pro:fig:transform} shows how easy it is to transform data from one coordinate system into
+another. %
+Transforming is essentially free, since axes compute coordinates on the fly. %
+
\begin{figure}
- \includegraphics[width=\textwidth]{"processing/fringes_transform"}
+ \includegraphics[width=0.8\textwidth]{"processing/fringes_transform"}
\includepython{"processing/fringes_transform.py"}
- \caption[CAPTION TODO]{
- CAPTION TODO}
+ \caption{
+ Example using transform.
+ }
+ \label{pro:fig:transform}
\end{figure}
\section{Math} % =================================================================================
@@ -783,8 +823,6 @@ Now that we know the basics of how the WrightTools \python{Data} class stores da
some data manipulation. %
Let's start with some elementary algebra. %
-% TODO: mention chunkwise strategy
-
\subsection{In-place operators} \label{pro:sec:in_place} % ---------------------------------------
In Python, operators are symbols that carry out some computation. %
@@ -847,10 +885,11 @@ Variables also support in-place operators. %
\subsection{Clip} % ------------------------------------------------------------------------------
Clip allows users to exclude values outside of a certain range. %
+Excluded values can be replaced with a user-specified value or with not-a-number. %
This can be particularly useful in cases like fitting. %
-See section ... for an example. % TODO: link to section
+See \autoref{pro:sec:fit} for an example. %
-It's also useful for when noise in a certain region of a spectrum obscures useful data...
+It's also useful for when noise in a certain region of a spectrum obscures useful data. %
Particularly true for normalized and signed data. %
\subsection{Symmetric root} % --------------------------------------------------------------------
@@ -885,7 +924,7 @@ keyword arguments. %
These may be slightly faster than \python{channel.log(base=10)} and
\python{channel.log(base=2)}. %
-\subsection{Level} % -----------------------------------------------------------------------------
+% \subsection{Level} % -----------------------------------------------------------------------------
% TODO: figure from wright.tools
@@ -894,6 +933,7 @@ These may be slightly faster than \python{channel.log(base=10)} and
Trim uses statistical treatment to find and remove outliers from a dataset. %
It is useful in cases where the naive strategy employed by \python{wt.Channel.clip} is not
sufficient, and when preparing for fitting. %
+See \autoref{pro:fig:outlier} for an example of outlier detection. %
Currently \python{trim} only supports one statistical treatment: the z-test. %
Z-testing compares each pixel to its multidimensional neighborhood of pixels. %
@@ -902,8 +942,6 @@ neighborhood standard deviation) it is either masked, replaced with \python{np.n
with the neighborhood mean. %
All outliers are found before any outliers are modified, so the algorithm is not directional. %
-% TODO: z-test citation
-
\python{wt.Channel.trim} can easily be enhanced with other statistical methods as needed. %
\subsection{Smooth} % ----------------------------------------------------------------------------
@@ -922,13 +960,20 @@ can confidently smooth collected data in post to achieve clean results. %
This strategy is similar to that accomplished in time domain CMDS where a low-pass filter is
applied on the very high resolution raw data. %
+\begin{figure}
+ \includegraphics[width=0.8\textwidth]{"processing/outlier"}
+ \caption{
+ Outlier detection using a z-test.
+ }
+ \label{pro:fig:outlier}
+\end{figure}
+
% TODO: figure: example of smoothed data
-\clearpage
\section{Dimensionality manipulation} % ==========================================================
WrightTools offers several strategies for reducing the dimensionality of a data object. %
-Also consider using the fit sub-package. % TODO: more info, link to section
+Also consider using the fit sub-package described in \autoref{pro:sec:fit}. %
\subsection{Chop} \label{pro:sec:chop} % ---------------------------------------------------------
@@ -941,7 +986,7 @@ This dataset can be chopped to it's component 2D \python{('wm'', 'w1')} spectra.
>>> import WrightTools as wt; from WrightTools import datasets
>>> data = wt.data.from_PyCMDS(datasets.PyCMDS.wm_w2_w1_000)
data created at /tmp/lzyjg4au.wt5::/
- axes ('wm', 'w2', 'w1')
+ axes ('wm'', 'w2', 'w1')
shape (35, 11, 11)
>>> chopped = data.chop('wm', 'w1')
chopped data into 11 piece(s) in ('wm', 'w1'')
@@ -977,8 +1022,8 @@ chopped data into 1 piece(s) in ('wm', 'w1')
>>> chopped.w2.points
array([1580.0])
\end{codefragment}
-Note the [0]... % TODO
-This same syntax used in artists... % TODO
+% Note the [0]... % TODO
+% This same syntax used in artists... % TODO
\subsection{Collapse} % --------------------------------------------------------------------------
@@ -1016,8 +1061,9 @@ To do this, the variable and channel names must agree. %
\begin{figure}
\includegraphics[width=\textwidth]{"processing/join_example"}
- \caption[CAPTION TODO]{
- CAPTION TODO}
+ \caption{
+ Join example.
+ }
\label{pro:fig:join_example}
\end{figure}
@@ -1032,7 +1078,7 @@ Rather than attempt to visualize \python{w1, w2} at some specific value of \pyth
powerful to instead consider the parameters (amplitude, offset, and time constant) of an
exponential fit at each \python{w1, w2} coordinate. %
On a more practical note, this kind of slice-by-slice dimensionality reduction via fitting can
-greatly simplify automated instrumental calibration (see ...) % TODO: link to opa chapter
+greatly simplify automated instrumental calibration (see \autoref{cha:opa}). %
WrightTools employs some simple tricks to enable these kind of fit operations, described here. %
% TODO: consider inserting figures that demonstrate this story (need to use wt2?)
@@ -1043,22 +1089,16 @@ One challenge of slice-by-slice fitting is making a good intial guess to optimiz
It is not tractable to ask the user to provide a guess for each slice, so some kind of reasonable
automated guessing must be used. %
WrightTools ``function'' objects are self contained describers of a particular function. %
-As an example, consider the \python{wt.fit.Expontial} class...
-It has parameters...
-Fit...
-Evaluate...
-Guess...
-
-Can be used directly...
-
-[USERS CAN WRITE THEIR OWN FUNCTION OBJECTS]
+As an example, consider the \python{wt.fit.Expontial} class.
+It can be used directly, as shown in \autoref{pro:fig:fit_function}
\begin{figure}
\includegraphics[width=0.5\textwidth]{"processing/fit_function"}
\includepython{"processing/fit_function.py"}
- \caption[CAPTION TODO]{
- CAPTION TODO
+ \caption{
+ Fitting a Gaussian.
}
+ \label{pro:fig:fit_function}
\end{figure}
\subsection{Fitter} % ----------------------------------------------------------------------------
@@ -1117,13 +1157,11 @@ the fit parameters, in this case amplitude, tau, and offset. Accordingly, outs i
than the original data object. model contains the fit evaluated at each coordinate of the original
dataset—it’s really useful for inspecting the quality of your fit procedure.
-[ALSO GOOD FOR WORKUP OF TUNING DATA: SEE SECTION ...]
-
\begin{figure}
\includegraphics[width=0.4\textwidth]{"processing/fit_amplitude"}
\includegraphics[width=0.4\textwidth]{"processing/fit_tau"}
- \caption[CAPTION TODO]{
- CAPTION TODO
+ \caption{
+ Fitting as dimensionality reduction.
}
\label{pro:fig:fitted_movie}
\end{figure}
@@ -1197,12 +1235,12 @@ returned to again and again. %
Developers can collaborate together to edit the software by making incremental changes that are
easy to review. %
-WrightTools uses git \cite{git} for source control, and the package is hosted on GitHub
-\cite{GitHub, WrightToolsGitHub}. %
+WrightTools uses git for source control, and the package is hosted on GitHub
+\cite{GitHub}. %
As of 2018-04-08, WrightTools has 1,018 commits from seven developers, as shown in
\autoref{pro:tab:commits}. %
-In addition to simply hosting the git repo, GitHub gives us issue tracking and automated
-integration with zenodo. %
+In addition to simply hosting the git repository, GitHub gives us issue tracking, continuous
+integration, and Zenodo support. %
The WrightTools package has a developer controlled version as well, following the semantic
versioning convention \cite{SemanticVersioning}. %
@@ -1223,9 +1261,8 @@ These attributes can be accessed through python: \python{wt.__version__} and
Daniel Kohler & 3 & 113 & 29 \\ \hline
Rachel Swedin & 1 & 5,197 & 0 \\ \hline
\end{tabular}
- \caption[CAPTION TODO]{
+ \caption[Commits to WrightTools.]{
Commits to WrightTools by individual, ordered by number of commits.
- Wright Stuff is...
Note that datasets...
}
\label{pro:tab:commits}
@@ -1260,7 +1297,8 @@ of the tests and receive a report on what failed and why. %
If a future programmer unintentionally breaks \python{convert}, the above test will fail and alert
her to the unexpected side effect of her modification. %
-[PARAGRAPH ABOUT CONTINUOUS INTEGRATION]
+WrightTools uses continuous integration services to run unit tests every time a pull request is
+made on GitHub. %
\subsection{Distribution} \label{pro:sec:distribution} % ------------------------------------------
@@ -1272,8 +1310,7 @@ Luckily for us, distribution is fairly simple within the Python ecosystem. %
The same tools that are used to distribute hugely popular packages like numpy are also available
for anyone else, including WrightTools. %
-The Python Package Index (PyPI), affectionately known as the cheese shop, is the official third
-party software repository for Python. %
+The Python Package Index (PyPI) is the official third party software repository for Python. %
It is community maintained, and supported by the Python Software Foundation and The Python
Packaging Group. %
As of 2018-04-08 PyPI hosts 134,758 Python packages, all for free. %
@@ -1291,10 +1328,9 @@ Since the scientific Python ecosystem has so many non-Python binary dependencies
popular choice---especially on Windows where the necessary compilers are not typically
pre-installed. %
Unlike pip, conda is not tied to a single repository. %
-There is the official repository, maintained by Anaconda, the company that develops conda.
-[CITE] %
+There is the official repository, maintained by Anaconda, the company that develops conda. %
There is also the popular conda-forge repository, which is maintained by the community via
-GitHub. [CITE] %
+GitHub. %
WrightTools is on conda-forge: %
\begin{codefragment}{bash}
conda config --add channels conda-forge
@@ -1308,13 +1344,11 @@ using such a thing immediately. %
To this end, it is important to have easy to use, searchable documentation with end-users in
mind. %
-wright.tools
-
-Built and hosted by readthedocs
-
-Both master and development versions...
+I have built a website for WrightTools documentation at \href{http://wright.tools/}. %
+The documentation is built and hosted by Read the Docs. \cite{ReadTheDocs}. %
+Both master and development versions are built, so users of both can get documentation that is up
+to date for them. %
-% \clearpage
% \section{Future directions} % ====================================================================
% Single variable decomposition. %
generated by cgit v1.2.3 (git 2.25.1) at 2024-09-28 22:18:46 +0000