Merge remote-tracking branch 'origin/master' into darien

2 files changed, 174 insertions, 117 deletions

diff --git a/acquisition/chapter.tex b/acquisition/chapter.tex
index 0436028..4446c0f 100644
--- a/acquisition/chapter.tex
+++ b/acquisition/chapter.tex
@@ -9,8 +9,10 @@
 

 \clearpage

 

-MR-CMDS relies on a good number of instrumental capabilities.  %

-Fundamentally, MR-CMDS is about delivering multiple pulses of light to a given sample.  %

+\section{Introduction}  % =========================================================================

+

+MR-CMDS instruments need to be capable of several different things.  %

+At its core, MR-CMDS is about delivering multiple pulses of light to a sample.  %

 The frequency and relative arrival time (delay) of each pulse must be scanned in the context of a

 basic multidimensional experiment.  %

 Scanning frequency requires using motors to change crystal angles and other optics within Optical

@@ -19,10 +21,11 @@ Scanning delay typically involves moving mirrors very small distances such that
 length of certain beam-lines changes.  %

 In addition to these ``first-order'' controls, the power of MR-CMDS is enhanced with additional

 control of pulse intensity and polarization.  % TODO: citations to motivate this 'enhancement'

-Each of these is an optomechanical device.  %

 An automated monchromator is typically used to spectrally resolve or isolate output signal.  %

+Each of these features is an optomechanical device: a piece of hardware that must be controlled in

+the context of an MR-CMDS experiment.  %

 

-An acquisition in MR-CMDS typically means going to a whole series of different positions.  %

+A scan in MR-CMDS typically means going to a whole series of different positions.  %

 As an example, a three-dimensional ``movie'' might be collected in the following way:

 \begin{codefragment}{python, label=aqn:lst:psuedoaqn}

 w1_points = [14300, 14400, 14500, 14600, 14700]  # wn

@@ -47,31 +50,58 @@ In practice, real scans are composed of $\sim$100 to $\sim$100,000 pixels, and t
 minute and one day to acquire.  %

 

 Because of the highly specialized nature of these experiments, MR-CMDS instruments typically

-require custom software to address all of simultaneous, repeated motor motion that a scan

+require custom software to address all of the simultaneous, repeated motor motion that a scan

 requires.  %

 Because MR-CMDS is really a family of techniques which require different kinds of motor motion,

 this acqusition software should be flexable enough to meet the creativity of its users.  %

 Furthermore, because MR-CMDS is a bleeding edge, rapidly evolving technique the instrumental

 software must be built in an extendable way to accommodate the ever-changing hardware

 configurations.  %

-In this chapter I describe how I built such an acquisition software.  %

-

-\section{Overview}  % =============================================================================

+In this chapter I describe how I built such an acquisition software, PyCMDS.  %

 

 PyCMDS is a unified software for controlling hardware and collecting data in the Wright Group.  %

-It is written almost entirely in Python, with a graphical user interface (GUI) made using Qt.  %

-It is cross-platform, running on Linux, Windows and macOS.  %

-It is open source, developed on GitHub.  % TODO: cite PyCMDS on github

-PyCMDS is used to drive both of the MR-CMDS instruments maintained by the Wright Group: the ``fs

-table'', focused on semiconductor photophysics, and the ``ps table'', focused on molecular

+It is written almost entirely in Python, with a graphical user interface (GUI) made using Qt.

+[CITE]  %

+It is cross-platform, with a core capable of running on Linux, Windows and macOS.  %

+It is open source, developed on GitHub. [CITE (GITHUB, PyCMDS)]  %

+In the Wright Group, PyCMDS replaces the old acquisition softwares `ps control', written by

+Kent Meyer, [CITE] and `Control for Lots of Research in Spectroscopy' (COLORS) written by Schuyler

+Kain [CITE].  %

+Today PyCMDS is used to drive both of the MR-CMDS instruments maintained by the Wright Group: the

+``fs table'', focused on semiconductor photophysics, and the ``ps table'', focused on molecular

 systems.  %

-In the Wright Group, PyCMDS replaces the old acquisition softwares `ps control', ritten by

-Kent Meyer and `Control for Lots of Research in Spectroscopy' written by Schuyler Kain.

+% BJT: consider giving more historical context re: COLORS, ps_control

+

+PyCMDS is best thought of as a central program with three kinds of modular ``plugins'' that can be

+extended indefinitely:

+\begin{ditemize}

+  \item Hardware: things that can be set to a position (\autoref{aqn:sec:hardware}).

+  \item Sensors: things that can be used to measure a signal (\autoref{aqn:sec:sensors}).

+  \item Acquisition modules: things that can be used to define and carry out an acquisition, and

+    associated post-processing (\autoref{aqn:sec:somatic}).

+\end{ditemize}

+The first design rule for PyCMDS is that these three things should be easy for the average

+(motivated) user to add by herself.  %

+Modularity and extensibility is important for all scientific software projects (see

+\autoref{cha:sof}), but it is of paramount importance for acquisition software simply because the

+diversity of hardware and experimental configurations is so great.  %

+It is conceivable to imagine 90\% overlap between the data processing and simulation needs of one

+spectroscopist to the next, but there is almost no overlap between hardware configurations even in

+the two primary instruments maintained by the Wright Group.  %

+Besides the extendable modular pieces, the rest of PyCMDS is a mostly-static code-base that accepts

+modules and does the necessary things to handle display of information from, and communication

+between them.  %

+

+% TODO: describe each of the sections of this chapter

+

+\clearpage

+\section{Graphical user interface}  % =============================================================

 

 When PyCMDS starts up, the GUI is constructed out of modules depending on which hardware and

 sensors the user has instructed the program to address.  %

 A screenshot of the PyCMDS GUI, running on the fs table, is shown in

 \autoref{aqn:fig:pycmds_screenshot}.  %

+

 On the left hand there is a single column displaying the current positions for all loaded

 hardware.  %

 Users may enter new destinations and hit the ``SET'' button.  %

@@ -79,68 +109,94 @@ Positions have units, which are changeable through the pull-down menu next to th
 display.  %

 Each hardware knows its own limits, displayed in a tool tip when hovering over the control.  %

 Users cannot type values outside of hardware limits into the controls.  %

-

-The large right-hand section is tabbed...

+Each hardware also has an ``ADVANCED'' button, which takes the user to a more extensive GUI to

+control lots more features (see section ....)  %

+At the very top, on the left hand side, is the ``SHUT DOWN'' button.  %

+

+On the right hand side there is a extensive set of nested tabs.  %

+The top level tabs are ``Program'', ``Hardware'', ``Devices'', ``Autonomic'', ``Somatic'' and

+``Plot''.  %

+Under each of these tabs is an entire separate set of display and control elements.  %

+Some of these elements are themselves tabbed, like the ``Somatic'' tab (active in

+\autoref{aqn:fig:pycmds_screenshot}), which has ``Queue'' and ``Scan'' sub-tabs.  %

+At the top of the right hand side there is a progress bar and queue status display, which I will

+discuss further in future sections.  %

 

 \begin{landscape}

 \begin{figure}

 	\includegraphics[scale=0.5]{"acquisition/screenshots/005"}

-  \caption{

-    TODO (PyCMDS screenshot)

+  \caption[PyCMDS at startup.]{

+    PyCMDS at startup, on the fs system.  %

   }

   \label{aqn:fig:pycmds_screenshot}

 \end{figure}

 \end{landscape}

 

-To perform an acquisition, a user goes to the SOMATIC tab and enters an item into the QUEUE...

-

-During the acquisition...

-Progress bar...

-Cannot hit SET...

-QUEUE states...

+When PyCMDS opens (\autoref{aqn:fig:pycmds_screenshot}), the user is first greeted with the

+``Somatic/Queue'' tab on the right hand side of the GUI.  %

+This is where the tells PyCMDS to do acquisitions.  %

+In PyCMDS, all acquisitions are done in a queue system.  %

+A queue is just a list of acquisitions, where the acquisitions are carried out one by one in

+order.  %

+

+To instruct PyCMDS to do an acquisition, the user must first create a queue by entering in a chosen

+name (default is ``queue''), and pressing the ``MAKE NEW QUEUE'' button.  %

+Alternatively the user may open an existing queue using ``OPEN QUEUE''.  %

+

+Next, the user must add the desired acquisition(s) to the queue.  %

+There are several acquisition modules, each with a different purpose.  %

+Choose an acquisition module using the drop down menu.  %

+Enter a name for your acquisition, and any additional info that you might want to keep track of.  %

+Finally, fill out any additional information that acquisition module might require, and press

+``APPEND TO QUEUE''.  %

+

+Once there are acquisitions in the queue, the user can press ``RUN QUEUE''.  %

+\autoref{aqn:fig:pycmds_screenshot_during_scan} is a screenshot of PyCMDS during a representative

+acquisition.  %

+This time, thee ``Somatic/Scan'' tab is chosen.  %

+On the left hand side, we can see that the ``SHUT DOWN'' and ``SET'' buttons are grayed out and

+unclickable, since the somatic system is in control of PyCMDS.  %

+At the moment when this screen shot was taken, PyCMDS is waiting for w3 (OPA-800CG) to finish

+moving, as the ``BUSY'' indicator shows.  %

+The progress bar, near the top of the GUI, is partially green to indicate the portion of the

+current scan that is finished.  %

+On the left hand side of the progress bar the time elapsed (fifteen minutes, thirty-seven seconds)

+is shown, and on the right hand shows the time remaining (three hours, eight minutes, sixteen

+seconds).  %

+In the center of the progress bar are the current scan parameters.  %

+

+The ``Somatic/Scan'' tab is built as an active display of currently acquiring data.  %

+A large graph and number show the current signal levels.  %

+On the far right-hand side, the device and channel to display can be chosen via drop-down menu.  %

+Under ``Status'', the loop time and scan index help users gauge the progress of their scan.  %

+In this case, the displayed pixel (index 6, 40) took 2.448 seconds to acquire.  %

 

 \begin{landscape}

 \begin{figure}

   \includegraphics[width=9in]{"acquisition/screenshots/000"}

-  \caption{

-    TODO (PyCMDS screenshot while acquiring data)

+  \caption[PyCMDS while scanning.]{

+    PyCMDS while acquiring data, on the ps system.  % 

   }

   \label{aqn:fig:pycmds_screenshot_during_scan}

 \end{figure}

 \end{landscape}

 

-\section{Structure}  % ============================================================================

+\section{Internal structure}  % ===================================================================

 

-I think of PyCMDS as a central program with three kinds of modular ``plugins'' that can be extended

-indefinitely:

-\begin{ditemize}

-  \item Hardware: things that can be set to a position (\autoref{aqn:sec:hardware}).

-  \item Sensors: things that can be used to measure a signal (\autoref{aqn:sec:sensors}).

-  \item Acquisition modules: things that can be used to define and carry out an acquisition, and

-    associated post-processing (\autoref{aqn:sec:somatic}).

-\end{ditemize}

-The first design rule for PyCMDS is that these three things should be easy for the average

-(motivated) user to add by herself.  %

-Modularity and extensibility is important for all software projects, but it is of paramount

-importance for acquisition software simply because the diversity of hardware and experimental

-configurations is so great.  %

-It is conceivable to imagine 90\% overlap between the data processing and simulation needs of one

-spectroscopist to the next, but there is almost no overlap between hardware configurations even in

-the two primary instruments maintained by the Wright Group.  %

-Besides the extendable modular pieces, the rest of PyCMDS is a mostly-static code-base that accepts

-modules and does the necessary things to handle display of information from, and communication

-between them.  %

+In this section I discuss the internal structure of PyCMDS.  %

+While there are a huge number of details not worthy of discussion (at time of writing, PyCMDS

+consists of 16,582 lines of source code), PyCMDS is made to be maintained and extended by future

+graduate students, so some insight into the internal structure is warranted.  %

 

 \subsection{Multithreading}  % --------------------------------------------------------------------

 

-For the kinds of acquisitions that the Wright Group has done the acquisition software spends the

-vast majority of its run-time waiting---waiting for user input through mouse clicks or keyboard

-presses, waiting for hardware to finish moving or for sensors to finish reading and return

-signals.  %

+PyCMDS spends the vast majority of its runtime waiting---waiting for user input through mouse

+clicks or keyboard presses, waiting for hardware to finish moving or for sensors to finish reading

+and return signals.  %

 Despite all of this downtime, it is crucial that the software respond very quickly when

-instructions or signals are recieved.  %

-A multi-threaded implementation is necessary to achieve this.  %

-The main thread handles the graphical user interface (GUI) and other top level things.  %

+instructions or signals are received.  %

+To achieve this, PyCMDS is designed using \emph{multithreading}.  %

+The main thread handles the graphical user interface (GUI) and other ``top level'' things.  %

 Everything else happens in child threads.  %

 Each hardware instance (e.g. a delay stage) lives in its own thread, as does each sensor.  %

 Since only one scan happens at a time, all acquisition modules share a single thread that handles

@@ -303,67 +359,6 @@ Use of qt plots...
 

 pyqtgraph \cite{pyqtgraph}

 

-\subsection{Scans}  % -----------------------------------------------------------------------------

-

-The central loop of scans in PyCMDS.  %

-\begin{codefragment}{python, label=aqn:lst:loop_simple}

-for coordinates in list_of_coordinates:

-  for hardware, coordinate in zip(hardwares, coordinates):

-      hardware.set(coordinate)

-  for hardware in hardwares:

-      hardware.wait_until_still()

-  for sensor in sensors:

-      sensor.read()    

-  for sensor in sensors:

-      sensor.wait_until_done() 

-\end{codefragment}

-

-\subsection{Conditional validity}  % --------------------------------------------------------------

-

-The central conceit of the PyCMDS modular hardware abstraction is that experiments can be boiled

-down to a set of orthogonal axes that can be set separately from other hardware and sensor

-status.  %

-This requirement is loosened by the autonomic and expression systems, such that any experiment

-could \emph{probably} be forced into PyCMDS, but still the conceit stands---PyCMDS is probably

-\emph{not} the correct framework if your experiment cannot be reduced in this way.  %

-From this we can see that it is useful to talk about the conditional validity of the modular

-hardware abstraction.  %

-

-The important axis is hardware complexity vs measurement complexity.

-

-For hardware-complex problems, the challenge is coordination.  %

-MR-CMDS is the perfect example of a hardware-complex problem.  %

-MR-CMDS is composed of a collection (typically 5 to 10 members) of relatively simple hardware

-components.  %

-The challenge is that experiments involve complex ``dances'', including expressions, of the

-component hardwares.  %

-

-For measurement-complex problems, the challenge is, well, the measurement.  %

-These are experiments where every little piece of the instrument is tied together into a complex

-network of inseparable parts.  %

-These are often time-domain or ``single shot'' measurements.  %

-Consider work of (GRAPES INVENTOR).  %

-Such instruments are typically much faster at data acquisition and more reliable.  %

-This comes at a price of flexibility: often such instruments cannot be modified or enhanced without

-touching everything.  %

-

-From an acquisition software perspective, measurement-complex problems are not amenable to general

-purpose modular software design.  %

-The instrument is so custom that it certainly requires entirely custom software.  %

-

-Measurements can be neither hardware-complex nor software-complex (simple) or both (expensive).  %

-Conceptually, can imagine a 4 quadrant system.  %

-

-Thus PyCMDS can be proud to try and generalize the hardware-complex part of acquisition software

-because indeed that is all that can be generalized.  %

-

-\begin{figure}

-  \caption{

-    CAPTION TODO: 4 QUADRANTS OF COMPLEXITY

-  }

-  \label{aqn:fig:complexity_quandrants}

-\end{figure}

- 

 \section{Hardware} \label{aqn:sec:hardware}  % ====================================================

 

 Hardware are things that 1) have a position, 2) can be set to a destination.  %

@@ -564,6 +559,21 @@ In contrast with the autonomic system (\autoref{aqn:sec:autonomic}), the somatic
 about voluntary, user specified motion.  %

 This is where the fun stuff happens---the acquisitions!

 

+\subsection{Scans}  % -----------------------------------------------------------------------------

+

+The central loop of scans in PyCMDS.  %

+\begin{codefragment}{python, label=aqn:lst:loop_simple}

+for coordinates in list_of_coordinates:

+  for hardware, coordinate in zip(hardwares, coordinates):

+      hardware.set(coordinate)

+  for hardware in hardwares:

+      hardware.wait_until_still()

+  for sensor in sensors:

+      sensor.read()    

+  for sensor in sensors:

+      sensor.wait_until_done() 

+\end{codefragment}

+

 \subsection{Acquisition modules}  % ---------------------------------------------------------------

 

 Acquisition modules are defined interfaces which know how to assemble a scan.  %

@@ -622,6 +632,53 @@ Self describing (enabled by \python{tidy_headers}).
 

 For scan module, just

 

+\clearpage

+\section{Conditional validity}  % =================================================================

+

+The central conceit of the PyCMDS modular hardware abstraction is that experiments can be boiled

+down to a set of orthogonal axes that can be set separately from other hardware and sensor

+status.  %

+This requirement is loosened by the autonomic and expression systems, such that any experiment

+could \emph{probably} be forced into PyCMDS, but still the conceit stands---PyCMDS is probably

+\emph{not} the correct framework if your experiment cannot be reduced in this way.  %

+From this we can see that it is useful to talk about the conditional validity of the modular

+hardware abstraction.  %

+

+The important axis is hardware complexity vs measurement complexity.

+

+For hardware-complex problems, the challenge is coordination.  %

+MR-CMDS is the perfect example of a hardware-complex problem.  %

+MR-CMDS is composed of a collection (typically 5 to 10 members) of relatively simple hardware

+components.  %

+The challenge is that experiments involve complex ``dances'', including expressions, of the

+component hardwares.  %

+

+For measurement-complex problems, the challenge is, well, the measurement.  %

+These are experiments where every little piece of the instrument is tied together into a complex

+network of inseparable parts.  %

+These are often time-domain or ``single shot'' measurements.  %

+Consider work of (GRAPES INVENTOR).  %

+Such instruments are typically much faster at data acquisition and more reliable.  %

+This comes at a price of flexibility: often such instruments cannot be modified or enhanced without

+touching everything.  %

+

+From an acquisition software perspective, measurement-complex problems are not amenable to general

+purpose modular software design.  %

+The instrument is so custom that it certainly requires entirely custom software.  %

+

+Measurements can be neither hardware-complex nor software-complex (simple) or both (expensive).  %

+Conceptually, can imagine a 4 quadrant system.  %

+

+Thus PyCMDS can be proud to try and generalize the hardware-complex part of acquisition software

+because indeed that is all that can be generalized.  %

+

+\begin{figure}

+  \caption{

+    CAPTION TODO: 4 QUADRANTS OF COMPLEXITY

+  }

+  \label{aqn:fig:complexity_quandrants}

+\end{figure}

+

 \section{Integrations}  % =========================================================================

 

 \begin{figure}

diff --git a/dissertation.tex b/dissertation.tex
index f3734bc..369e9a3 100644
--- a/dissertation.tex
+++ b/dissertation.tex
@@ -70,8 +70,8 @@ This dissertation is approved by the following members of the Final Oral Committ
 \part{Development} \label{prt:development}

 \include{processing/chapter}  % pro

 \include{acquisition/chapter}  % acq

-\include{active_correction/chapter}  % act

-\include{opa/chapter}  % opa

+%\include{active_correction/chapter}  % act

+%\include{opa/chapter}  % opa

 %\include{mixed_domain/chapter}

 

 \part{Applications} \label{prt:applications}