From 357568e1fb77afed9dfa203e62da237bf7ce51b3 Mon Sep 17 00:00:00 2001 From: Blaise Thompson Date: Mon, 9 Apr 2018 00:24:18 -0500 Subject: 2018-04-09 00:24 --- processing/chapter.tex | 172 ++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 134 insertions(+), 38 deletions(-) (limited to 'processing') diff --git a/processing/chapter.tex b/processing/chapter.tex index 1ec2c37..997b297 100644 --- a/processing/chapter.tex +++ b/processing/chapter.tex @@ -1084,44 +1084,31 @@ dataset—it’s really useful for inspecting the quality of your fit procedure. \section{Construction, maintenance, and distribution} % ========================================== While WrightTools has already been useful to the work done in the WrightGroup over the last 3 -years, the true potential of the package is in potential future uses. % -WrightTools is designed to be extended and continuously enhanced to serve an ever-wider set of -users and spectroscopies. % -Despite it's name, WrightTools is built to be used even by those outside of the Wright Group. % +years, the true potential of the package lies in its future. % +WrightTools is designed in a modular way so that it can be continuously enhanced to serve an +ever-wider set of users and spectroscopies. % +Despite its name, WrightTools is built to be used even by those outside of the Wright Group. % Currently WrightTools may be only 75\% of what a typical multidimensional spectroscopist needs, but if those scientists work to enhance the package with what \emph{they} need, they may also solve problems for others such that the usefulness of the software gradually increases. % In order for this dream to come true, WrightTools must be constructed and maintained by collaborative tools such that users feel comfortable contributing to future enhancements. % -It has not been easy to build this software as a group of 3 to 5 contributors, and the coordination -problems will be harder as more users and developers join. % -To this end, the following section discusses strategies for keeping WrightTools collaborative and -useful. % +All of the challenges to collaboration discussed in \autoref{cha:sof} certainly apply to +WrightTools, so it is important that we follow best practices now in order to make WrightTools as +maintainable and future-proof as possible. % +To this end, this section discusses strategies that I have employed in the construction, +maintenance, and distribution of WrightTools. % \subsection{Licensing} % ------------------------------------------------------------------------ -In order to even talk about collaborative development, one must at least have an open source -license. % -According to US copyright law, software is fully copyright protected by default. % TODO: cite +As discussed in \autoref{cha:sof}, open source licenses are an important part of scientific +software development. % Those writing software must explicitly license their project in order to ensure that users have basic rights to copy edit and distribute code. % -A whole set of licenses have been written to use for this purpose. % -Briefly, strong copyleft licenses do not allow for software to be modified or enhancements without -sharing those enhancements under the same licenses. % -These ``viral'' licenses are meant to force companies and individuals who would otherwise not open -source to share their code. % -The strongest copyleft licenses do not even allow others to link against the licensed software -without themselves being copyleft. % -This strategy has been moderately successful, but is becoming less popular. % TODO: cite -Open source licenses are less opinionated, merely granting users the right to copy, modify, -distribute, and publish code without restriction except perhaps credit to the source. % - -% TODO: link GPL, LGPL etc - -WrightTools is MIT licensed (otherwise called the Expat license). % +WrightTools is licensed under the hugely popular Expat/MIT license. % This license is incredibly permissive and puts as few restrictions as possible on the end users. % -Because the license is short, it is reproduced below. % +The following is the WrightTools license, reproduced in its entirety. % \begin{dquote} @@ -1154,30 +1141,139 @@ However many Python libraries end up being interfaces to compiled code that coul closed-source. % The Scientific Python Stack have MIT-compatible licenses, including BSD-like licenses. % -\subsection{Distribution} \label{pro:sec:distribution} % ------------------------------------------ +\subsection{Version control} % ------------------------------------------------------------------- + +As mentioned several times in \autoref{cha:sof}, having software under source control is probably +the most important recommendation in scientific software development. % +Source control allows developers to create ``checkpoints'' for their software package that can be +returned to again and again. % +Developers can collaborate together to edit the software by making incremental changes that are +easy to review. % + +WrightTools uses git \cite{git} for source control, and the package is hosted on GitHub +\cite{GitHub, WrightToolsGitHub}. % +As of 2018-04-08, WrightTools has 1,018 commits from seven developers, as shown in +\autoref{pro:tab:commits}. % +In addition to simply hosting the git repo, GitHub gives us issue tracking and automated +integration with zenodo. % + +The WrightTools package has a developer controlled version as well, following the semantic +versioning convention \cite{SemanticVersioning}. % +The current distributed version of WrightTools is \bash{3.0.1}, with \bash{3.0.2} under active +development. % +The wt5 file also has a semantic version, currently \bash{1.0.0}. % +These attributes can be accessed through python: \python{wt.__version__} and +\python{wt.__wt5_version__}. % -How does WrightTools get onto end-users machines? % -Distribution... +\begin{table} + \begin{tabular}{c | c | c | c} + person & number of commits & lines added & lines removed \\ \hline + Blaise Thompson & 478 & 621,918 & 507,938 \\ \hline + Kyle Sunden & 208 & 19,293 & 9,218 \\ \hline + Darien Morrow & 29 & 1,589 & 127 \\ \hline + Nathan Neff-Mallon & 20 & 2,880 & 910 \\ \hline + Kyle Czech & 5 & 1,150 & 25 \\ \hline + Daniel Kohler & 3 & 113 & 29 \\ \hline + Rachel Swedin & 1 & 5,197 & 0 \\ \hline + \end{tabular} + \caption[CAPTION TODO]{ + Commits to WrightTools by individual, ordered by number of commits. + Wright Stuff is... + Note that datasets... + } + \label{pro:tab:commits} +\end{table} -WrightTools is distributed on PyPI and conda-forge. % +\subsection{Unit tests} % ------------------------------------------------------------------------ -WrightTools uses semantic versioning. % +Maintainable code must be tested, so that future developers can use tests to ensure that they do +not break necessary functionality. % +Unit testing is a very simple testing paradigm in which small, separate tests are written to +address each ``unit'' of the software package. % +As an example, the following is one of WrightTools' tests: +\begin{codefragment}{python} +# part of WrightTools/tests/data/convert_data.py +def test_wigner(): + p = datasets.COLORS.v2p2_WL_wigner + data = wt.data.from_COLORS(p) + assert data.d1.units == 'fs' + data.convert('ns') + assert data.d1.units == 'ns' + assert data.wm.units == 'nm' + data.close() +\end{codefragment} +This test loads one of the distributed COLORS datasets and makes sure that the \python{convert} +method works as intended. % +To do this, it uses the \python{assert} statement which raises an exception when a condition is +\python{False}. % +This particular test is pretty humble, but there is strength in numbers: as of 2018-04-08 there are +224 unit tests within WrightTools. % +Using the built in \python{pytest} library (\bash{python setup.py test}), a programmer can run all +of the tests and receive a report on what failed and why. % +If a future programmer unintentionally breaks \python{convert}, the above test will fail and alert +her to the unexpected side effect of her modification. % + +[PARAGRAPH ABOUT CONTINUOUS INTEGRATION] -% TODO: citations to WrightTools on PyPI and conda-forge +\subsection{Distribution} \label{pro:sec:distribution} % ------------------------------------------ + +WrightTools is on GitHub, which is a fantastic way for developers to get software onto their +computers. % +But how does software get onto everybody elses machine? % +Developers call this process ``distribution''. % +Luckily for us, distribution is fairly simple within the Python ecosystem. % +The same tools that are used to distribute hugely popular packages like numpy are also available +for anyone else, including WrightTools. % + +The Python Package Index (PyPI), affectionately known as the cheese shop, is the official third +party software repository for Python. % +It is community maintained, and supported by the Python Software Foundation and The Python +Packaging Group. % +As of 2018-04-08 PyPI hosts 134,758 Python packages, all for free. % +WrightTools is also hosted on PyPI. % +Every time we change our version, we ``release'' by uploading the newest version to PyPI. % +pip (``pip installs packages'', ``pip installs python'', or ``preferred installer program'') [CITE] +can be used to install packages directly from PyPI: % +\begin{codefragment}{bash} +pip install WrightTools +\end{codefragment} + +Conda is a multilingual package manager that handles virtual environments and dependencies, even +binary dependencies, in a hassle-free way. % +Since the scientific Python ecosystem has so many non-Python binary dependencies, Conda is a +popular choice---especially on Windows where the necessary compilers are not typically +pre-installed. % +Unlike pip, conda is not tied to a single repository. % +There is the official repository, maintained by Anaconda, the company that develops conda. +[CITE] % +There is also the popular conda-forge repository, which is maintained by the community via +GitHub. [CITE] % +WrightTools is on conda-forge: % +\begin{codefragment}{bash} +conda config --add channels conda-forge +conda install WrightTools +\end{codefragment} -\subsection{Collaborative development} % --------------------------------------------------------- +\subsection{Documentation} % --------------------------------------------------------------------- -Git... +WrightTools is a piece of scripted software, and many spectroscopists many not be comfortable with +using such a thing immediately. % +To this end, it is important to have easy to use, searchable documentation with end-users in +mind. % -As of 2018-04-08, WrightTools has ........ commits from ..... developers +wright.tools -\subsection{Unit tests} % ------------------------------------------------------------------------ +Built and hosted by readthedocs -Unit testing... +Both master and development versions... \clearpage \section{Future directions} % ==================================================================== Single variable decomposition. % -Usage in next-generation simulation packages. % \ No newline at end of file +Usage in next-generation simulation packages. % + +More tests. % + +Usage by multiple groups. % \ No newline at end of file -- cgit v1.2.3