path: root/acquisition/chapter.tex

\chapter{Acquistion} \label{cha:acq}

\begin{dquote}
  The question of software correctness ultimately boils down to, “Does it do what we have in our
  minds, even the things we have not gotten around to thinking about yet?”

  \dsignature{Alistair Cockburn}
\end{dquote}

\clearpage

MR-CMDS relies on a good number of instrumental capabilities.  %
Fundamentally, MR-CMDS is about delivering multiple pulses of light to a given sample.  %
The frequency and relative arrival time (delay) of each pulse must be scanned in the context of a
basic multidimensional experiment.  %
Scanning frequency requires using motors to change crystal angles and other optics within Optical
Parametric Amplifiers (OPAs).  %
Scanning delay typically involves moving mirrors very small distances such that the optical path
length of certain beam-lines changes.  %
In addition to these ``first-order'' controls, the power of MR-CMDS is enhanced with additional
control of pulse intensity and polarization.  % TODO: citations to motivate this 'enhancement'
Each of these is an optomechanical device.  %
An automated monchromator is typically used to spectrally resolve or isolate output signal.  %

An acquisition in MR-CMDS typically means going to a whole series of different positions.  %
As an example, a three-dimensional ``movie'' might be collected in the following way:
\begin{codefragment}{python, label=aqn:lst:psuedoaqn}
w1_points = [14300, 14400, 14500, 14600, 14700]  # wn
w2_points = [14100, 14200, 14300, 14400, 14500, 14600, 14700]  # wn
d2_points = [100, 75, 50, 25, 0, 50, 75, 100, 125, 150, 175, 200]  # fs
for w2 in w2_points:
    set_w2(w2)
    for w1 in w1_points:
        set_w1(w1)
        for d2 in d2_points:
            set_d2(d2)
            measure_signal()
\end{codefragment}
In this simple example, there are 5 \python{w1} destinations, 7 \phon{w2} destinations, and 12
\python{d2} destinations, so there are a total of $5\times7\times12=420$ pixels in the
three-dimensional scan.  %
The acquisition software must set the hardware to each of these points and acquire data at each of
them.  %
Each hardware motion and signal measurement takes roughly one second, so this acquisition would
take roughly 7 minutes to complete.  %
In practice, real scans are composed of $\sim$100 to $\sim$100,000 pixels, and take between 1
minute and one day to acquire.  %

Because of the highly specialized nature of these experiments, MR-CMDS instruments typically
require custom software to address all of simultaneous, repeated motor motion that a scan
requires.  %
Because MR-CMDS is really a family of techniques which require different kinds of motor motion,
this acqusition software should be flexable enough to meet the creativity of its users.  %
Furthermore, because MR-CMDS is a bleeding edge, rapidly evolving technique the instrumental
software must be built in an extendable way to accommodate the ever-changing hardware
configurations.  %
In this chapter I describe how I built such an acquisition software.  %

\section{Overview}  % =============================================================================

PyCMDS is a unified software for controlling hardware and collecting data in the Wright Group.  %
It is written almost entirely in Python, with a graphical user interface (GUI) made using Qt.  %
It is cross-platform, running on Linux, Windows and macOS.  %
It is open source, developed on GitHub.  % TODO: cite PyCMDS on github
PyCMDS is used to drive both of the MR-CMDS instruments maintained by the Wright Group: the ``fs
table'', focused on semiconductor photophysics, and the ``ps table'', focused on molecular
systems.  %
In the Wright Group, \gls{PyCMDS} replaces the old acquisition softwares `ps control', ritten by
Kent Meyer and `Control for Lots of Research in Spectroscopy' written by Schuyler Kain.

When PyCMDS starts up, the GUI is constructed out of modules depending on which hardware and
sensors the user has instructed the program to address.  %
A screenshot of the PyCMDS GUI, running on the fs table, is shown in
\autoref{aqn:fig:pycmds_screenshot}.  %
On the left hand there is a single column displaying the current positions for all loaded
hardware.  %
Users may enter new destinations and hit the ``SET'' button.  %
Positions have units, which are changeable through the pull-down menu next to the control and
display.  %
Each hardware knows its own limits, displayed in a tool tip when hovering over the control.  %
Users cannot type values outside of hardware limits into the controls.  %

The large right-hand section is tabbed...

\begin{landscape}
\begin{figure}
	\includegraphics[scale=0.5]{"acquisition/screenshots/005"}
  \caption{
    TODO (PyCMDS screenshot)
  }
  \label{aqn:fig:pycmds_screenshot}
\end{figure}
\end{landscape}

To perform an acquisition, a user goes to the SOMATIC tab and enters an item into the QUEUE...

During the acquisition...
Progress bar...
Cannot hit SET...
QUEUE states...

\begin{landscape}
\begin{figure}
  \includegraphics[width=9in]{"acquisition/screenshots/000"}
  \caption{
    TODO (PyCMDS screenshot while acquiring data)
  }
  \label{aqn:fig:pycmds_screenshot_during_scan}
\end{figure}
\end{landscape}

\section{Structure}  % ============================================================================

I think of PyCMDS as a central program with three kinds of modular ``plugins'' that can be extended
indefinitely:
\begin{ditemize}
  \item Hardware: things that can be set to a position (\autoref{aqn:sec:hardware}).
  \item Sensors: things that can be used to measure a signal (\autoref{aqn:sec:sensors}).
  \item Acquisition modules: things that can be used to define and carry out an acquisition, and
    associated post-processing (\autoref{aqn:sec:somatic}).
\end{ditemize}
The first design rule for PyCMDS is that these three things should be easy for the average
(motivated) user to add by herself.  %
Modularity and extensibility is important for all software projects, but it is of paramount
importance for acquisition software simply because the diversity of hardware and experimental
configurations is so great.  %
It is conceivable to imagine 90\% overlap between the data processing and simulation needs of one
spectroscopist to the next, but there is almost no overlap between hardware configurations even in
the two primary instruments maintained by the Wright Group.  %
Besides the extendable modular pieces, the rest of PyCMDS is a mostly-static code-base that accepts
modules and does the necessary things to handle display of information from, and communication
between them.  %

\subsection{Multithreading}  % --------------------------------------------------------------------

For the kinds of acquisitions that the Wright Group has done the acquisition software spends the
vast majority of its run-time waiting---waiting for user input through mouse clicks or keyboard
presses, waiting for hardware to finish moving or for sensors to finish reading and return
signals.  %
Despite all of this downtime, it is crucial that the software respond very quickly when
instructions or signals are recieved.  %
A multi-threaded implementation is necessary to achieve this.  %
The main thread handles the graphical user interface (GUI) and other top level things.  %
Everything else happens in child threads.  %
Each hardware instance (e.g. a delay stage) lives in its own thread, as does each sensor.  %
Since only one scan happens at a time, all acquisition modules share a single thread that handles
the orchestration of hardware motion, sensor operation, and data processing that the chosen
acquisition requires. %

Threads are powerful because they allow for ``semi-synchronous'' operation.  %
Imagine PyCMDS is in the middle of a 2D delay-delay scan, and the scan thread has just told each of
the two delay stages to head to their destinations.  %
PyCMDS must sit in a tight loop to keep track of the position as closely as possible during motor
motion.  %
In a single-threaded configuration, this tight loop would only run for one delay at a time, such
that PyCMDS would have to finish shepherding one delay stage before turning its attention to the
second.  %
In a multi-threaded configuration, each thread will run simultaniously, switching off CPU cycles at
a low level far faster than human comprehension.  %
This switching is handled in an OS and hardware specific way---luckily it is all abstracted through
platform-agnostic Qt threads.  %

Threads are dangerous because it is hard to pass information between them.  %
Without any special protection, two threads have no reason not to simultaneously edit and read the
same location in memory.  %
If a delay stage is writing its position to memory as a 64-bit double at the same time as the
acquisition thread reads that memory address, the acquisition thread will read in nonsense (or
worse), it will crash).  %
So some strategy is needed to ensure that threads respect each other.  %
The Mutex design allows threads to ``lock'' an object such that it cannot be modified by a
different thread.  %
This lock is like the ``talking stick'' employed my many early child educators.  %
When the talking stick is being used, only the child that holds the stick is allowed to speak.  %
The stick must be passed to another child (as directed by the teacher) before they can share their
thought.  %
PyCMDS makes heavy use of Mutexes, in particular the \bash{QMutex} class \cite{QMutex}.  %

Mutexes handle basic information transfer (two threads can both safely modify and read a particular
object), but what about sending instructions between threads?  %
Here the problem is deciding what happens when multiple instructions are given simultaneously, or
an instruction is given while another instruction is being carried out.  %
Again, this is a classic problem in computer science, and the classic answer is the queue.  %
Queues are just like lines at the coffee shop---each person (instruction) is served (carried out)
in the order that they joined the line.  %
Queues are commonly referred to as FIFO (First In First Out) for this reason.  %
PyCMDS uses queues for almost all instructions.  %

Finally, PyCMDS makes extensive use of the ``signals and slots'' construct, which is somewhat
unique (and certainly original) to Qt.  %
Signals and slots are powerful because they allow threads without instruction to go completely
silent, making them essentially free in terms of CPU usage.  %
Normally, a thread needs to sit in a loop merely listening for instructions.  %
Within the Qt framework, a thread can be ``woken'' by a signal without needing that thread to
explicitly ``listen''.  %
These concepts fit within the broader umbrella of ``event-driven programming'', a concept that has
been used in many languages and frameworks (notably high level LabVIEW tends to be very
event-driven).  %
The Qt signals and slots system massively simplifies programming within PyCMDS.  %

Note that multithreading is very different from multiprocessing.  %

\subsection{High level objects}  % ----------------------------------------------------------------

Towards the goal of stability and extensability, PyCMDS makes heavy use of abstraction and
inheritance.  %

Abstraction means that complex implementation details are hidden by simple interfaces.  %
For example, consider the simple case of setting an OPA to a particular color.  %
For OPAs in the Wright Group, this operation requires the following:
\begin{ditemize}
  \item Load the OPA tuning curve, find which motors must move.
  \item Interpolate the discrete tuning curve, and evaluate that interpolated curve at the desired
    destination.
  \item Send each motor towards their new destination.
  \item Wait for the motors to arrive, check that nothing has gone wrong.
\end{ditemize}
This is not even to mention the complexity of spawning and sending information between the main
thread and working threads.  %
Through abstraction, PyCMDS is able to wrap all this complexity into the \python{OPA} class and
it's \python{set_position} method---so doing all of the operations above is as simple as
\python{opa.set_position(1300, 'nm')}.  %
Importantly, abstraction does not magically get rid of the complexity.  %
It simply \emph{hides} the complexity so that it becomes tractable to write simple interfaces to
accomplish complex things.  %

PyCMDS implements abstraction through inheritance.  %
In object oriented programming, inheritance is when one class is based on another.  %
The \emph{child} class acquires all of the properties of the \emph{parent} class.  %
The child class, then, can modify or extend the properties that it needs, without needing to
re-implement the properties that it shares with the parent.  %
Let's consider PyCMDS hardware again.  %
Every single unique hardware in PyCMDS lives in it's own worker thread, so the basic problem of
information transfer through queues and Mutexes is shared between them.  %
A \python{Hardware} class which is parent to \emph{all} hardwares can define the methods and
attributes necessary to abstract this basic thread communication issue.  %
Every type of delay stage addressed by PyCMDS needs to handle the basic task of translating between
``natural'' units (femtosecond, picosecond, nanosecond) and ``native'' units (typically mm).  %
They all need an attribute \python{zero_position}, in native units, and a method to do the
conversion.  %
A class common to all delay stages can abstract away all of these conversion details.  %
In summary, an inheritance-based system for implementing delay stages might look like this:
\begin{codefragment}{bash}
Hardware  # implements basic thread control
└── Delay  # implements conversion between natural and native units
    ├── Homemade
    ├── Thorlabs
    └── Newport
\end{codefragment}
The powerful thing about this strategy is that the three driver-specific classes (...) need only
implement minimal driver-specific code, typically start, set close...
This means that code is more maintainable and less buggy...
Easier to change high-level behavior without redoing low-level code...
Only implemented once...

At it's most basic PyCMDS defines the following simple data types (derived from
\python{PyCMDS_object}):
\begin{ditemize}
  \item Bool
  \item Combo
  \item Filepath
  \item Number
  \item String
\end{ditemize}
These classes do multiple things.  %
First, they \emph{are} Mutexes, with thread-safe \python{read} and \python{write} methods.  %
Secondly, they support ``implicit'' storage in ini files.  %
Third, they know how to participate in the GUI.  %
They can display their value, and if modified they will propagate that modification to the internal
threads of outward...
Finally, they have special properties like units and limits etc...

Without getting into details, let's investigate the key ``signals and slots'' that hardware and
sensors have.  %
% TODO: elaborate

The following is the top-level hardware class, parent of all hardware and sensors.  %

\begin{figure}
	\includepython{"acquisition/parent_hardware.py"}
  \caption[Parent to hardware and sensors.]{
    Parent class of all hardware and sensors.  %
    For brevity, methods \python{close}, \python{update} and \python{wait_until_still} have been
    omitted.  %
  }
  \label{aqn:fig:parent_hardware_class}
\end{figure}

\begin{figure}
	\includepython{"acquisition/driver.py"}
  \caption[TODO]{
    TODO
  }
  \label{aqn:fig:driver}
\end{figure}

\subsection{Graphical user interface}  % ----------------------------------------------------------

Made up of widgets...

Table widget...

Use of qt plots...

pyqtgraph \cite{pyqtgraph}

\subsection{Scans}  % -----------------------------------------------------------------------------

The central loop of scans in PyCMDS.  %
\begin{codefragment}{python, label=aqn:lst:loop_simple}
for coordinates in list_of_coordinates:
  for hardware, coordinate in zip(hardwares, coordinates):
      hardware.set(coordinate)
  for hardware in hardwares:
      hardware.wait_until_still()
  for sensor in sensors:
      sensor.read()    
  for sensor in sensors:
      sensor.wait_until_done() 
\end{codefragment}

\subsection{Conditional validity}  % --------------------------------------------------------------

The central conceit of the PyCMDS modular hardware abstraction is that experiments can be boiled
down to a set of orthogonal axes that can be set separately from other hardware and sensor
status.  %
This requirement is loosened by the autonomic and expression systems, such that any experiment
could \emph{probably} be forced into PyCMDS, but still the conceit stands---PyCMDS is probably
\emph{not} the correct framework if your experiment cannot be reduced in this way.  %
From this we can see that it is useful to talk about the conditional validity of the modular
hardware abstraction.  %

The important axis is hardware complexity vs measurement complexity.

For hardware-complex problems, the challenge is coordination.  %
MR-CMDS is the perfect example of a hardware-complex problem.  %
MR-CMDS is composed of a collection (typically 5 to 10 members) of relatively simple hardware
components.  %
The challenge is that experiments involve complex ``dances'', including expressions, of the
component hardwares.  %

For measurement-complex problems, the challenge is, well, the measurement.  %
These are experiments where every little piece of the instrument is tied together into a complex
network of inseparable parts.  %
These are often time-domain or ``single shot'' measurements.  %
Consider work of (GRAPES INVENTOR).  %
Such instruments are typically much faster at data acquisition and more reliable.  %
This comes at a price of flexibility: often such instruments cannot be modified or enhanced without
touching everything.  %

From an acquisition software perspective, measurement-complex problems are not amenable to general
purpose modular software design.  %
The instrument is so custom that it certainly requires entirely custom software.  %

Measurements can be neither hardware-complex nor software-complex (simple) or both (expensive).  %
Conceptually, can imagine a 4 quadrant system.  %

Thus PyCMDS can be proud to try and generalize the hardware-complex part of acquisition software
because indeed that is all that can be generalized.  %

\begin{figure}
  \caption{
    CAPTION TODO: 4 QUADRANTS OF COMPLEXITY
  }
  \label{aqn:fig:complexity_quandrants}
\end{figure}
 
\section{Hardware} \label{aqn:sec:hardware}  % ====================================================

Hardware are things that 1) have a position, 2) can be set to a destination.  %
Typically they also have associated units and limits.  %
They sometimes have an offset, as specified by the autonomic system
(\autoref{aqn:sec:autonomic}).  %
Each hardware can be thought of as a dimension of the MR-CMDS experiment, and scans include a
specific traversal through this multidimensional space.  %

\subsection{Hardware inheritance}  % --------------------------------------------------------------

All hardware classes are children of the parent \python{Hardware} class
(\autoref{aqn:fig:hardware_class}), which is itself a child of the the global \python{Hardware}
class shown in \autoref{aqn:fig:parent_hardware_class}.  %
By inspecting \autoref{aqn:fig:hardware_class}, we can see that all hardware require the following
methods:
\begin{ditemize}
  \item \python{close}
  \item \python{get_destination}
  \item \python{get_position}
  \item \python{on_address_initialized}
  \item \python{poll}
  \item \python{set_offset}
  \item \python{set_position}
  \item \python{@property units}
\end{ditemize}

\begin{figure}
	\includepython{"acquisition/hardware.py"}
  \caption[Parent hardware class.]{
    Parent class of all hardware.  %
    For brevity, methods \python{close}, \python{get_destination}, \python{get_position},
    \python{is_valid}, \python{on_address_initialized}, \python{poll}, and \python{@property units}
    have been omitted.  %
  }
  \label{aqn:fig:hardware_class}
\end{figure}

\begin{figure}
	\includebash{"acquisition/hardware_inheritance"}
  \caption[Hardware inheritance.]{
    CAPTION TODO
  }
  \label{aqn:fig:hardware_inheritance}
\end{figure}

\subsection{Delays}  % ----------------------------------------------------------------------------

Delays are the kind of thing that have a position in absolute units, a zero position, and a
relative position measured in fs or ps.  %
They also are the kind of thing that is most commonly subject to offset by the Autonomic
system...  %

In addition to the inherited methods, delay hardware objects require the following:
\begin{ditemize}
  \item \python{set_motor_position}
\end{ditemize}

Delay driver objects require the following:
\begin{ditemize}
  \item \python{set_motor_position}
\end{ditemize}

The delay GUI, by default, offers
\begin{ditemize}
  \item \python{on_home}
  \item \python{on_set_motor}
  \item \python{on_set_zero}
\end{ditemize}

\begin{landscape}
\begin{figure}
 	\includegraphics[width=9in]{"acquisition/screenshots/006"}
  \caption[Representative delay stage advanced menu.]{
    CAPTION TODO
  }
  \label{aqn:fig:delay_advanced}
\end{figure}
\end{landscape}

\subsection{Spectrometers}  % ---------------------------------------------------------------------

Spectrometers are the kind of thing that can be set to a single color (typically native units are
in nm).  %
They often have turrets that allow for switching between gratings.  %
Other features are not currently supported as PyCMDS has only been asked to drive one kind of
monochromator so far.  %

Hardware:

Driver:

GUI:

\begin{landscape}
\begin{figure}
  \includegraphics[width=9in]{"acquisition/screenshots/007"}
  \caption[Representative spectrometer advanced menu.]{
    CAPTION TODO
  }
  \label{aqn:fig:spectrometer_advanced}
\end{figure}
\end{landscape}

\subsection{OPAs}  % ------------------------------------------------------------------------------

Hardware:

Driver:

GUI:

\subsubsection{Curves}

Recursive

United

Interpolation

\begin{landscape}
\begin{figure}
 	\includegraphics[width=9in]{"acquisition/screenshots/008"}
  \caption[Representative OPA advanced menu.]{
    CAPTION TODO
  }
  \label{aqn:fig:opa_advanced}
\end{figure}
\end{landscape}

\subsection{Filters}  % ---------------------------------------------------------------------------

Hardware:

Driver:

GUI:

\clearpage
\section{Sensors (devices)} \label{aqn:sec:sensors}  % ============================================

Sensors are...

For DAQ cards, shots and samples...

Power of digital processing...

Old boxcar: 300 ns window, ~10 micosecond delay. Onset of saturation ~2 V.

% TODO: screenshots

\subsection{Sensors as axes}  % -------------------------------------------------------------------

% TODO: equation from Skye

% TODO: poweramp tune test  (see darien slack)

\begin{figure}
  \includegraphics[scale=0.5]{"acquisition/tune_test"}
  \caption[Array detector serving as an axis.]{
    CAPTION TODO (2017-11-06 OPA2)
  }
  \label{aqn:fig:array_as_axis}
\end{figure}

\section{Autonomic} \label{aqn:sec:autonomic}  % ==================================================

\begin{dquote}
  The autonomic nervous system, which innervates primarily the smooth musculature of all organs,
  the heart and the glands, mediates the neuronal regulation of the internal milieu.  %
  The actions of this system, as its name implies, are in general not under direct voluntary
  control.  %
  These characteristics distinguish the autonomic nervous system from the somatic nervous system,
  which mediates afferent and efferent communication with the enviornment and, for the most part,
  is subject to voluntary control and accessible to consciousness.  %

  The autonomic and somatic systems operate hand in hand...

  The functions of the autnomic nervous system are to keep the internal milieu of the body constant
  (homeostasis...) or adjust it as required by changing circumstances (e.g., mechanical work, food
  intake, water deprivation, heat or cold).  %

  \dsignature{W. J\"{a}nig, Autonomic Nervous System (1989) \cite{JanigW1989a}}
\end{dquote}

% TODO: concept of additive offsets

\begin{figure}
  \caption[Representative spectral delay correction]{
    CAPTION TODO
  }
  \label{aqn:fig:sdc}
\end{figure}
  
\section{Somatic} \label{aqn:sec:somatic} % =======================================================

In contrast with the autonomic system (\autoref{aqn:sec:autonomic}), the somatic system is all
about voluntary, user specified motion.  %
This is where the fun stuff happens---the acquisitions!

\subsection{Acquisition modules}  % ---------------------------------------------------------------

Acquisition modules are defined interfaces which know how to assemble a scan.  %

\autoref{aqn:fig:aqn_file} shows an aqn file for an acquisition using SCAN.  %

\begin{figure}
  \includebash{"acquisition/example.aqn"}
  \caption[Example aqn file.]{
    CAPTION TODO
  }
  \label{aqn:fig:aqn_file}
\end{figure}

\subsubsection{SCAN}

SCAN is by far the most important acquisition module in PyCMDS---it handles acquisitions for almost
all of the CMDS experiments that PyCMDS has ever accomplished.  %
SCAN is capable of acquisitions of arbitrary dimensionality.  %
Users simply append as many axes as they want.  %
Acquistions are done with the trailing (highest index) axis as the innermost loop.  %
Arbitrary expressions (PyCMDS calls them ``constants'') are also possible, as can be seen in
\autoref{aqn:fig:aqn_file}.  %

\subsubsection{TUNE TEST}

The TUNE TEST module does a simple thing: it sets a chosen OPA to each of the points in it's tuning
curve and does a monochromator scan of set width about that setpoint.  %
In this way the tune (output color) agreement between the curve and the OPA can be determined.  %
As a convinience, a new point curve with remapped colors is automatically created.  %
% TODO: link to place in tuning chapter...

\subsubsection{MOTORTUNE}

Arbitrary tuning acquisitions.

\subsubsection{AUTOTUNE}

Automatically do appropriate scans and process as in chapter...

\subsubsection{POYNTING TUNE}

Dedicated to poynting (get content from Kyle S)...

z\subsection{Queue manager}  % ---------------------------------------------------------------------

\subsection{The central loop of PyCMDS}  % --------------------------------------------------------

\subsection{The data file}  % ---------------------------------------------------------------------

Does not have the same dimensionality restrictions as prior acquisition software.  %

Self describing (enabled by \python{tidy_headers}).

\subsection{Automatic processing}  % --------------------------------------------------------------

For scan module, just

\section{Integrations}  % =========================================================================

\begin{figure}
	\frame{\includegraphics[width=\textwidth]{"acquisition/slack"}}
	\caption[TODO]{
    TODO
  }
  \label{aqn:fig:slack}
\end{figure}

\section{Future directions}  % ====================================================================

\subsection{Spectral delay correction module}  % --------------------------------------------------

Currently spectral delay correction is done ``manually''.  %
Relevant Wigner scans are taken using the SCAN module, processed using WrightTools, and generated
coset files are manually applied in the autonomic menu.  % 
Ideally, ``check'' Wigners are then taken to verify the veracity of the corrections.  %
In the future, it would be preferable to have a dedicated SDC module that did all of these things
automatically.  %

It would start from existing corrections (for that table geometry) and iterate until the applied
Wigners are all correct.  %

\subsection{``Headless'' hardware, sensors}  % ----------------------------------------------------

The abstraction of hardware complexity that PyCMDS offers is really convienient, but currently
these convinient classes can only be used within PyCMDS itself.  %
Since code in PyCMDS is inseperable from the GUI, it is not possible to \python{import PyCMDS} and
use the abstract tools in other programs or scripts.  %
This design is not necessary, and doing the work to free hardware and sensor code from the GUI code
would allow for all kinds of creative experiments that are not currently possible within the
central conceit of PyCMDS.  %

\subsection{Ideal Axis Positions} \label{acq:sec:ideal_axis_positions}  % -------------------------

Frequency domain multidimensional spectroscopy is a time-intensive process.  %
A typical \gls{pixel} takes between one-half second and three seconds to acquire.  %
Depending on the exact hardware being scanned and signal being detected, this time may be mostly
due to hardware motion or signal collection.  %
Due to the \gls{curse of dimensionality}, a typical three-dimensional CMDS experiment contains
roughly 100,000 pixels.  %
CMDS hardware is transiently-reliable, so speeding up experiments is a crucial component of
unlocking ever larger dimensionalities and higher resolutions.  %

One obvious way to decrease the scan-time is to take fewer pixels.  %
Traditionally, multidimensional scans are done with linearly arranged points in each axis---this is
the simplest configuration to program into the acquisition software.  %
Because signal features are often sparse or slowly varying (especially so in high-dimensional
scans) linear stepping means that \emph{most of the collected pixels} are duplicates or simply
noise.  %
A more intelligent choice of axis points can capture the same nonlinear spectrum in a fraction of
the total pixel count.  %

An ideal distribution of pixels is linearized in \emph{signal}, not coordinate.  %
This means that every signal level (think of a contour in the N-dimensional case) has roughly the
same number of pixels defining it.  %
If some generic multidimensional signal goes between 0 and 1, one would want roughly 10\% of the
pixels to be between 0.9 and 1.0, 10\% between 0.8 and 0.9 and so on.  %
If the signal is sparse in the space explored (imagine a narrow two-dimensional Lorentzian in the
center of a large 2D-Frequency scan) this would place the majority of the pixels near the narrow
peak feature(s), with only a few of them defining the large (in axis space) low-signal floor.  %
In contrast linear stepping would allocate the vast majority of the pixels in the low-signal 0.0 to
0.1 region, with only a few being used to capture the narrow peak feature.  %
Of course, linearizing pixels in signal requires prior expectations about the shape of the
multidimensional signal---linear stepping is still an appropriate choice for low-resolution
``survey'' scans.  %

CMDS scans often posses correlated features in the multidimensional space.  %
In order to capture such features as cheaply as possible, one would want to define regions of
increased pixel density along the correlated (diagonal) lineshape.  %
As a concession to reasonable simplicity, our acquisition software (PyCMDS) assumes that all scans
constitute a regular array with-respect-to the scanned axes.  %
We can acquire arbitrary points along each axis, but not for the multidimensional scan.  %
This means that we cannot achieve strictly ideal pixel distributions for arbitrary datasets.  %
Still, we can do much better than linear spacing.
% TODO: refer to PyCMDS/WrightTools 'regularity' requirement when that section exists

Almost all CMDS lineshapes (in frequency and delay) can be described using just a few lineshape
functions:
\begin{ditemize}
	\item exponential
	\item Gaussian
	\item Lorentzian
	\item bimolecular
\end{ditemize}

Exponential and bimolecular dynamics fall out of simple first and second-order kinetics (I will
ignore higher-order kinetics here).  %
Gaussians come from our Gaussian pulse envelopes or from normally-distributed inhomogeneous
broadening.  %
The measured line-shapes are actually convolutions of the above.  %
I will ignore the convolution except for a few illustrative special cases.  %
More exotic lineshapes are possible in CMDS---quantum beating and breathing modes, for example---I
will also ignore these.  %
Derivations of the ideal pixel positions for each of these lineshapes appear below.
% TODO: cite Wright Group quantum beating paper, Kambempati breathing paper

\subsubsection{Exponential}

Simple exponential decays are typically used to describe population and coherence-level dynamics in
CMDS.  %
For some generic exponential signal $S$ with time constant $\tau$,
\begin{equation} \label{eq:simple_exponential_decay}
S(t) = \me^{-\frac{t}{\tau}}.
\end{equation}
We can write the conjugate equation to \ref{eq:simple_exponential_decay}, asking ``what $t$ do I
need to get a cerain signal level?'':
\begin{eqnarray}
\log{(S)} &=& -\frac{t}{\tau} \\
t &=& -\taulog{(S)}.
\end{eqnarray}
So to step linearly in $t$, my step size has to go as $-\tau\log{(S)}$.

We want to go linearly in signal, meaning that we want to divide $S$ into even sections.  %
If $S$ goes from 0 to 1 and we choose to acquire $N$ points,
\begin{eqnarray}
t_n &=& -\tau\log{\left(\frac{n}{N}\right)}.
\end{eqnarray}
Note that $t_n$ starts at long times and approaches zero delay.  %
So the first $t_1$ is the smallest signal and $t_N$ is the largest.  %

Now we can start to consider realistic cases, like where $\tau$ is not quite known and where some
other longer dynamics persist (manifested as a static offset).  %
Since these values are not separable in a general system, I'll keep $S$ normalized between 0 and
1.  %
\begin{eqnarray}
S &=& (1-c)\me^{-\frac{t}{\tau_{\mathrm{actual}}}} + c \\
S_n &=& (1-c)\me^{-\frac{-\tau_{\mathrm{step}}\log{\left(\frac{n}{N}\right)}}{\tau_{\mathrm{actual}}}} + c \\
S_n &=& (1-c)\me^{-\frac{\tau_{\mathrm{step}}}{\tau_{\mathrm{actual}}} \log{\left(\frac{N}{n}\right)}} + c \\
S_n &=& (1-c)\left(\frac{N}{n}\right)^{-\frac{\tau_{\mathrm{step}}}{\tau_{\mathrm{actual}}}} + c \\
S_n &=& (1-c)\left(\frac{n}{N}\right)^{\frac{\tau_{\mathrm{step}}}{\tau_{\mathrm{actual}}}} + c
\end{eqnarray}

\begin{figure}
	\includegraphics[scale=0.5]{"processing/PyCMDS/ideal axis positions/exponential"}
	\caption[TODO]{TODO}
  \label{aqn:fig:exponential_steps}
\end{figure}

\subsubsection{Gaussian}

\subsubsection{Lorentzian}

\subsubsection{Bimolecular}

\subsection{Simultanious acquisitions}  % ---------------------------------------------------------

Sometimes PyCMDS needs to do multiple scans that are completely orthogonal.  %
For example, certain OPA tuning operations only require signals from pyroelectric detectors that
are each permanently installed to monitor a single OPA.  %
In such cases, theoretically multiple OPAs could be tuned simultaneously.  %
The coordination of such operations would be more difficult than what currently exists, but could
be possible within PyCMDS' multithreaded design.  %

\subsection{Hotswappable hardware}  % -------------------------------------------------------------

One of the most important basic capabilities of PyCMDS is the ability to reconfigure itself
depending on what hardware is loaded.  %
However controlling the hardware configuration is currently more difficult than it should be.  %
Currently users need to shut down PyCMDS, manually edit INI files to configure which hardware are
included, and restart PyCMDS.  %
A better solution would allow users to ``hotswap''---load, or drop hardware and sensors
without shutting down PyCMDS.  %
The work of allowing this would also allow users to reload hardware, a great fallback option when
there are communication issues.  %

\subsection{wt5 savefile}  % ----------------------------------------------------------------------

The wt5 save format, introduced in WrightTools 3, is a huge improvement over the simple, plaintext
data format currently used by PyCMDS.  % TODO: link to discussion of wt5 format
wt5 files are smaller on disk, have richer metadata and self-descriptive properties, and multiple
scans can be stored in the same file using \python{Collections}.  % TODO: again, link
Because wt5 files are stored on disk but fully accessible through slicing, PyCMDS could have full
access to the scan arrays without risk of memory overflow, which promises to allow for new
potential visualizations during acquisition.  % TODO: presumably we talk about buffered file
                                % writing above?

Even more exciting, wt5 files can be instantiated as empty and \textit{then} filled.  %
This means that the wt5 file can be a self-describing set of destinations that actually defines
which pixels PyCMDS should visit.  %