From 3dc46ef66670d4e465d7d662e5dd7175fb8fae2c Mon Sep 17 00:00:00 2001 From: Blaise Thompson Date: Mon, 9 Apr 2018 21:03:57 -0500 Subject: 2018-04-09 21:03 --- acquisition/chapter.tex | 287 +++++++++++++++++++++++++++++------------------- 1 file changed, 172 insertions(+), 115 deletions(-) (limited to 'acquisition') diff --git a/acquisition/chapter.tex b/acquisition/chapter.tex index 0436028..4446c0f 100644 --- a/acquisition/chapter.tex +++ b/acquisition/chapter.tex @@ -9,8 +9,10 @@ \clearpage -MR-CMDS relies on a good number of instrumental capabilities. % -Fundamentally, MR-CMDS is about delivering multiple pulses of light to a given sample. % +\section{Introduction} % ========================================================================= + +MR-CMDS instruments need to be capable of several different things. % +At its core, MR-CMDS is about delivering multiple pulses of light to a sample. % The frequency and relative arrival time (delay) of each pulse must be scanned in the context of a basic multidimensional experiment. % Scanning frequency requires using motors to change crystal angles and other optics within Optical @@ -19,10 +21,11 @@ Scanning delay typically involves moving mirrors very small distances such that length of certain beam-lines changes. % In addition to these ``first-order'' controls, the power of MR-CMDS is enhanced with additional control of pulse intensity and polarization. % TODO: citations to motivate this 'enhancement' -Each of these is an optomechanical device. % An automated monchromator is typically used to spectrally resolve or isolate output signal. % +Each of these features is an optomechanical device: a piece of hardware that must be controlled in +the context of an MR-CMDS experiment. % -An acquisition in MR-CMDS typically means going to a whole series of different positions. % +A scan in MR-CMDS typically means going to a whole series of different positions. % As an example, a three-dimensional ``movie'' might be collected in the following way: \begin{codefragment}{python, label=aqn:lst:psuedoaqn} w1_points = [14300, 14400, 14500, 14600, 14700] # wn @@ -47,31 +50,58 @@ In practice, real scans are composed of $\sim$100 to $\sim$100,000 pixels, and t minute and one day to acquire. % Because of the highly specialized nature of these experiments, MR-CMDS instruments typically -require custom software to address all of simultaneous, repeated motor motion that a scan +require custom software to address all of the simultaneous, repeated motor motion that a scan requires. % Because MR-CMDS is really a family of techniques which require different kinds of motor motion, this acqusition software should be flexable enough to meet the creativity of its users. % Furthermore, because MR-CMDS is a bleeding edge, rapidly evolving technique the instrumental software must be built in an extendable way to accommodate the ever-changing hardware configurations. % -In this chapter I describe how I built such an acquisition software. % - -\section{Overview} % ============================================================================= +In this chapter I describe how I built such an acquisition software, PyCMDS. % PyCMDS is a unified software for controlling hardware and collecting data in the Wright Group. % -It is written almost entirely in Python, with a graphical user interface (GUI) made using Qt. % -It is cross-platform, running on Linux, Windows and macOS. % -It is open source, developed on GitHub. % TODO: cite PyCMDS on github -PyCMDS is used to drive both of the MR-CMDS instruments maintained by the Wright Group: the ``fs -table'', focused on semiconductor photophysics, and the ``ps table'', focused on molecular +It is written almost entirely in Python, with a graphical user interface (GUI) made using Qt. +[CITE] % +It is cross-platform, with a core capable of running on Linux, Windows and macOS. % +It is open source, developed on GitHub. [CITE (GITHUB, PyCMDS)] % +In the Wright Group, PyCMDS replaces the old acquisition softwares `ps control', written by +Kent Meyer, [CITE] and `Control for Lots of Research in Spectroscopy' (COLORS) written by Schuyler +Kain [CITE]. % +Today PyCMDS is used to drive both of the MR-CMDS instruments maintained by the Wright Group: the +``fs table'', focused on semiconductor photophysics, and the ``ps table'', focused on molecular systems. % -In the Wright Group, PyCMDS replaces the old acquisition softwares `ps control', ritten by -Kent Meyer and `Control for Lots of Research in Spectroscopy' written by Schuyler Kain. +% BJT: consider giving more historical context re: COLORS, ps_control + +PyCMDS is best thought of as a central program with three kinds of modular ``plugins'' that can be +extended indefinitely: +\begin{ditemize} + \item Hardware: things that can be set to a position (\autoref{aqn:sec:hardware}). + \item Sensors: things that can be used to measure a signal (\autoref{aqn:sec:sensors}). + \item Acquisition modules: things that can be used to define and carry out an acquisition, and + associated post-processing (\autoref{aqn:sec:somatic}). +\end{ditemize} +The first design rule for PyCMDS is that these three things should be easy for the average +(motivated) user to add by herself. % +Modularity and extensibility is important for all scientific software projects (see +\autoref{cha:sof}), but it is of paramount importance for acquisition software simply because the +diversity of hardware and experimental configurations is so great. % +It is conceivable to imagine 90\% overlap between the data processing and simulation needs of one +spectroscopist to the next, but there is almost no overlap between hardware configurations even in +the two primary instruments maintained by the Wright Group. % +Besides the extendable modular pieces, the rest of PyCMDS is a mostly-static code-base that accepts +modules and does the necessary things to handle display of information from, and communication +between them. % + +% TODO: describe each of the sections of this chapter + +\clearpage +\section{Graphical user interface} % ============================================================= When PyCMDS starts up, the GUI is constructed out of modules depending on which hardware and sensors the user has instructed the program to address. % A screenshot of the PyCMDS GUI, running on the fs table, is shown in \autoref{aqn:fig:pycmds_screenshot}. % + On the left hand there is a single column displaying the current positions for all loaded hardware. % Users may enter new destinations and hit the ``SET'' button. % @@ -79,68 +109,94 @@ Positions have units, which are changeable through the pull-down menu next to th display. % Each hardware knows its own limits, displayed in a tool tip when hovering over the control. % Users cannot type values outside of hardware limits into the controls. % - -The large right-hand section is tabbed... +Each hardware also has an ``ADVANCED'' button, which takes the user to a more extensive GUI to +control lots more features (see section ....) % +At the very top, on the left hand side, is the ``SHUT DOWN'' button. % + +On the right hand side there is a extensive set of nested tabs. % +The top level tabs are ``Program'', ``Hardware'', ``Devices'', ``Autonomic'', ``Somatic'' and +``Plot''. % +Under each of these tabs is an entire separate set of display and control elements. % +Some of these elements are themselves tabbed, like the ``Somatic'' tab (active in +\autoref{aqn:fig:pycmds_screenshot}), which has ``Queue'' and ``Scan'' sub-tabs. % +At the top of the right hand side there is a progress bar and queue status display, which I will +discuss further in future sections. % \begin{landscape} \begin{figure} \includegraphics[scale=0.5]{"acquisition/screenshots/005"} - \caption{ - TODO (PyCMDS screenshot) + \caption[PyCMDS at startup.]{ + PyCMDS at startup, on the fs system. % } \label{aqn:fig:pycmds_screenshot} \end{figure} \end{landscape} -To perform an acquisition, a user goes to the SOMATIC tab and enters an item into the QUEUE... - -During the acquisition... -Progress bar... -Cannot hit SET... -QUEUE states... +When PyCMDS opens (\autoref{aqn:fig:pycmds_screenshot}), the user is first greeted with the +``Somatic/Queue'' tab on the right hand side of the GUI. % +This is where the tells PyCMDS to do acquisitions. % +In PyCMDS, all acquisitions are done in a queue system. % +A queue is just a list of acquisitions, where the acquisitions are carried out one by one in +order. % + +To instruct PyCMDS to do an acquisition, the user must first create a queue by entering in a chosen +name (default is ``queue''), and pressing the ``MAKE NEW QUEUE'' button. % +Alternatively the user may open an existing queue using ``OPEN QUEUE''. % + +Next, the user must add the desired acquisition(s) to the queue. % +There are several acquisition modules, each with a different purpose. % +Choose an acquisition module using the drop down menu. % +Enter a name for your acquisition, and any additional info that you might want to keep track of. % +Finally, fill out any additional information that acquisition module might require, and press +``APPEND TO QUEUE''. % + +Once there are acquisitions in the queue, the user can press ``RUN QUEUE''. % +\autoref{aqn:fig:pycmds_screenshot_during_scan} is a screenshot of PyCMDS during a representative +acquisition. % +This time, thee ``Somatic/Scan'' tab is chosen. % +On the left hand side, we can see that the ``SHUT DOWN'' and ``SET'' buttons are grayed out and +unclickable, since the somatic system is in control of PyCMDS. % +At the moment when this screen shot was taken, PyCMDS is waiting for w3 (OPA-800CG) to finish +moving, as the ``BUSY'' indicator shows. % +The progress bar, near the top of the GUI, is partially green to indicate the portion of the +current scan that is finished. % +On the left hand side of the progress bar the time elapsed (fifteen minutes, thirty-seven seconds) +is shown, and on the right hand shows the time remaining (three hours, eight minutes, sixteen +seconds). % +In the center of the progress bar are the current scan parameters. % + +The ``Somatic/Scan'' tab is built as an active display of currently acquiring data. % +A large graph and number show the current signal levels. % +On the far right-hand side, the device and channel to display can be chosen via drop-down menu. % +Under ``Status'', the loop time and scan index help users gauge the progress of their scan. % +In this case, the displayed pixel (index 6, 40) took 2.448 seconds to acquire. % \begin{landscape} \begin{figure} \includegraphics[width=9in]{"acquisition/screenshots/000"} - \caption{ - TODO (PyCMDS screenshot while acquiring data) + \caption[PyCMDS while scanning.]{ + PyCMDS while acquiring data, on the ps system. % } \label{aqn:fig:pycmds_screenshot_during_scan} \end{figure} \end{landscape} -\section{Structure} % ============================================================================ +\section{Internal structure} % =================================================================== -I think of PyCMDS as a central program with three kinds of modular ``plugins'' that can be extended -indefinitely: -\begin{ditemize} - \item Hardware: things that can be set to a position (\autoref{aqn:sec:hardware}). - \item Sensors: things that can be used to measure a signal (\autoref{aqn:sec:sensors}). - \item Acquisition modules: things that can be used to define and carry out an acquisition, and - associated post-processing (\autoref{aqn:sec:somatic}). -\end{ditemize} -The first design rule for PyCMDS is that these three things should be easy for the average -(motivated) user to add by herself. % -Modularity and extensibility is important for all software projects, but it is of paramount -importance for acquisition software simply because the diversity of hardware and experimental -configurations is so great. % -It is conceivable to imagine 90\% overlap between the data processing and simulation needs of one -spectroscopist to the next, but there is almost no overlap between hardware configurations even in -the two primary instruments maintained by the Wright Group. % -Besides the extendable modular pieces, the rest of PyCMDS is a mostly-static code-base that accepts -modules and does the necessary things to handle display of information from, and communication -between them. % +In this section I discuss the internal structure of PyCMDS. % +While there are a huge number of details not worthy of discussion (at time of writing, PyCMDS +consists of 16,582 lines of source code), PyCMDS is made to be maintained and extended by future +graduate students, so some insight into the internal structure is warranted. % \subsection{Multithreading} % -------------------------------------------------------------------- -For the kinds of acquisitions that the Wright Group has done the acquisition software spends the -vast majority of its run-time waiting---waiting for user input through mouse clicks or keyboard -presses, waiting for hardware to finish moving or for sensors to finish reading and return -signals. % +PyCMDS spends the vast majority of its runtime waiting---waiting for user input through mouse +clicks or keyboard presses, waiting for hardware to finish moving or for sensors to finish reading +and return signals. % Despite all of this downtime, it is crucial that the software respond very quickly when -instructions or signals are recieved. % -A multi-threaded implementation is necessary to achieve this. % -The main thread handles the graphical user interface (GUI) and other top level things. % +instructions or signals are received. % +To achieve this, PyCMDS is designed using \emph{multithreading}. % +The main thread handles the graphical user interface (GUI) and other ``top level'' things. % Everything else happens in child threads. % Each hardware instance (e.g. a delay stage) lives in its own thread, as does each sensor. % Since only one scan happens at a time, all acquisition modules share a single thread that handles @@ -303,67 +359,6 @@ Use of qt plots... pyqtgraph \cite{pyqtgraph} -\subsection{Scans} % ----------------------------------------------------------------------------- - -The central loop of scans in PyCMDS. % -\begin{codefragment}{python, label=aqn:lst:loop_simple} -for coordinates in list_of_coordinates: - for hardware, coordinate in zip(hardwares, coordinates): - hardware.set(coordinate) - for hardware in hardwares: - hardware.wait_until_still() - for sensor in sensors: - sensor.read() - for sensor in sensors: - sensor.wait_until_done() -\end{codefragment} - -\subsection{Conditional validity} % -------------------------------------------------------------- - -The central conceit of the PyCMDS modular hardware abstraction is that experiments can be boiled -down to a set of orthogonal axes that can be set separately from other hardware and sensor -status. % -This requirement is loosened by the autonomic and expression systems, such that any experiment -could \emph{probably} be forced into PyCMDS, but still the conceit stands---PyCMDS is probably -\emph{not} the correct framework if your experiment cannot be reduced in this way. % -From this we can see that it is useful to talk about the conditional validity of the modular -hardware abstraction. % - -The important axis is hardware complexity vs measurement complexity. - -For hardware-complex problems, the challenge is coordination. % -MR-CMDS is the perfect example of a hardware-complex problem. % -MR-CMDS is composed of a collection (typically 5 to 10 members) of relatively simple hardware -components. % -The challenge is that experiments involve complex ``dances'', including expressions, of the -component hardwares. % - -For measurement-complex problems, the challenge is, well, the measurement. % -These are experiments where every little piece of the instrument is tied together into a complex -network of inseparable parts. % -These are often time-domain or ``single shot'' measurements. % -Consider work of (GRAPES INVENTOR). % -Such instruments are typically much faster at data acquisition and more reliable. % -This comes at a price of flexibility: often such instruments cannot be modified or enhanced without -touching everything. % - -From an acquisition software perspective, measurement-complex problems are not amenable to general -purpose modular software design. % -The instrument is so custom that it certainly requires entirely custom software. % - -Measurements can be neither hardware-complex nor software-complex (simple) or both (expensive). % -Conceptually, can imagine a 4 quadrant system. % - -Thus PyCMDS can be proud to try and generalize the hardware-complex part of acquisition software -because indeed that is all that can be generalized. % - -\begin{figure} - \caption{ - CAPTION TODO: 4 QUADRANTS OF COMPLEXITY - } - \label{aqn:fig:complexity_quandrants} -\end{figure} - \section{Hardware} \label{aqn:sec:hardware} % ==================================================== Hardware are things that 1) have a position, 2) can be set to a destination. % @@ -564,6 +559,21 @@ In contrast with the autonomic system (\autoref{aqn:sec:autonomic}), the somatic about voluntary, user specified motion. % This is where the fun stuff happens---the acquisitions! +\subsection{Scans} % ----------------------------------------------------------------------------- + +The central loop of scans in PyCMDS. % +\begin{codefragment}{python, label=aqn:lst:loop_simple} +for coordinates in list_of_coordinates: + for hardware, coordinate in zip(hardwares, coordinates): + hardware.set(coordinate) + for hardware in hardwares: + hardware.wait_until_still() + for sensor in sensors: + sensor.read() + for sensor in sensors: + sensor.wait_until_done() +\end{codefragment} + \subsection{Acquisition modules} % --------------------------------------------------------------- Acquisition modules are defined interfaces which know how to assemble a scan. % @@ -622,6 +632,53 @@ Self describing (enabled by \python{tidy_headers}). For scan module, just +\clearpage +\section{Conditional validity} % ================================================================= + +The central conceit of the PyCMDS modular hardware abstraction is that experiments can be boiled +down to a set of orthogonal axes that can be set separately from other hardware and sensor +status. % +This requirement is loosened by the autonomic and expression systems, such that any experiment +could \emph{probably} be forced into PyCMDS, but still the conceit stands---PyCMDS is probably +\emph{not} the correct framework if your experiment cannot be reduced in this way. % +From this we can see that it is useful to talk about the conditional validity of the modular +hardware abstraction. % + +The important axis is hardware complexity vs measurement complexity. + +For hardware-complex problems, the challenge is coordination. % +MR-CMDS is the perfect example of a hardware-complex problem. % +MR-CMDS is composed of a collection (typically 5 to 10 members) of relatively simple hardware +components. % +The challenge is that experiments involve complex ``dances'', including expressions, of the +component hardwares. % + +For measurement-complex problems, the challenge is, well, the measurement. % +These are experiments where every little piece of the instrument is tied together into a complex +network of inseparable parts. % +These are often time-domain or ``single shot'' measurements. % +Consider work of (GRAPES INVENTOR). % +Such instruments are typically much faster at data acquisition and more reliable. % +This comes at a price of flexibility: often such instruments cannot be modified or enhanced without +touching everything. % + +From an acquisition software perspective, measurement-complex problems are not amenable to general +purpose modular software design. % +The instrument is so custom that it certainly requires entirely custom software. % + +Measurements can be neither hardware-complex nor software-complex (simple) or both (expensive). % +Conceptually, can imagine a 4 quadrant system. % + +Thus PyCMDS can be proud to try and generalize the hardware-complex part of acquisition software +because indeed that is all that can be generalized. % + +\begin{figure} + \caption{ + CAPTION TODO: 4 QUADRANTS OF COMPLEXITY + } + \label{aqn:fig:complexity_quandrants} +\end{figure} + \section{Integrations} % ========================================================================= \begin{figure} -- cgit v1.2.3