-
Notifications
You must be signed in to change notification settings - Fork 1
/
sb_icollection.html
298 lines (194 loc) · 13.2 KB
/
sb_icollection.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Overview of the bsui+xpdAcq environment — xpdAcq 0.5 documentation</title>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="top" title="xpdAcq 0.5 documentation" href="index.html"/>
<link rel="up" title="For XPD Users" href="xpdusers.html"/>
<link rel="next" title="Setting up your Beamtime" href="usb_Beamtime.html"/>
<link rel="prev" title="Overview of xpd acquisition environment" href="sb_overview.html"/>
<script src="_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search">
<a href="index.html" class="icon icon-home"> xpdAcq
</a>
<div class="version">
0.5
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<ul>
<li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quick Start</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="xpdusers.html">For XPD Users</a></li>
</ul>
<ul class="current">
<li class="toctree-l1 current"><a class="reference internal" href="beamlinestaff.html">For Beamline Staff</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="sb_cheat_sheet.html">Cheat Sheet</a></li>
<li class="toctree-l2"><a class="reference internal" href="sb_overview.html">Overview of xpd acquisition environment</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Overview of the bsui+xpdAcq environment</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#nsls-ii-data-philosophy">NSLS-II data philosophy</a></li>
<li class="toctree-l3"><a class="reference internal" href="#xpdacq-philosophy">xpdAcq philosophy</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="sb_newBeamtime.html">Initializing a new beamtime for a user</a></li>
<li class="toctree-l2"><a class="reference internal" href="sb_endBeamtime.html">Finalizing and ending a completed beamtime for a user</a></li>
</ul>
</li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="troubleshooting.html">Troubleshooting</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="release_note.html">Release Notes</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">xpdAcq</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html">Docs</a> »</li>
<li><a href="xpdusers.html">For XPD Users</a> »</li>
<li>Overview of the bsui+xpdAcq environment</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/sb_icollection.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="overview-of-the-bsui-xpdacq-environment">
<span id="sb-icollection"></span><h1>Overview of the bsui+xpdAcq environment<a class="headerlink" href="#overview-of-the-bsui-xpdacq-environment" title="Permalink to this headline">¶</a></h1>
<p>All data collection at NSLS-II is carried out through a powerful
Python-software-language based interface which runs in an ipython interactive
session. By typing <code class="docutils literal"><span class="pre">bsui</span></code> at the unix prompt (command line) in a terminal
window at the XPD computer you begin an ipython session and preload an ipython
profile (called <code class="docutils literal"><span class="pre">collection</span></code>) that contains all the software to run the NSLS-II
data collection software.</p>
<p>You can tell when you are in the <code class="docutils literal"><span class="pre">collection</span></code> environment
because you will see <code class="docutils literal"><span class="pre">(collection-yyQn.x)</span></code> at the beginning of the command-line.
<code class="docutils literal"><span class="pre">yy</span></code> stands for year and <code class="docutils literal"><span class="pre">Qn.x</span></code> tells detailed information about the
version. For example, <code class="docutils literal"><span class="pre">collection-17Q1.0</span></code> would mean the fist version of
quarter 1, in 2017.When you are not in the environment you won’t see that. There is also an analysis environment, which is currently called <code class="docutils literal"><span class="pre">analysis</span></code>,
that is very similar to the <code class="docutils literal"><span class="pre">collection</span></code> environment but it is for data
analysis functions and you cannot control any XPD hardware from it. This may
be run on a different computer than the control software, as long as it can see
the NSLS-II data database. This environment is activated by typing <code class="docutils literal"><span class="pre">ianalysis</span></code>
in a fresh terminal on one of the XPD linux computers.</p>
<p>Inside the <code class="docutils literal"><span class="pre">collection-yyQn.x</span></code> environment you can type commands that will control the
diffractometer, collect data, and also extract data from the NSLS-II database.
The NSLS-II data acquisition environment is powerful and flexible, but there is a
stiff learning curve to using it. We have therefore added a layer on top that
is a package of programs called <code class="docutils literal"><span class="pre">xpdAcq</span></code> that are designed to make the data
acquisition at XPD easier and more robust for most people. We strongly suggest
that you use the xpdacq functions as much as possible.</p>
<div class="section" id="nsls-ii-data-philosophy">
<h2>NSLS-II data philosophy<a class="headerlink" href="#nsls-ii-data-philosophy" title="Permalink to this headline">¶</a></h2>
<p>NSLS-II is committed to archiving all raw data, and making it available to
authorized users. For this to succeed it is important to be able to find the
data when it is desired. The basic philosophy at NSLS-II is therefore to
tag every dataset with unique-ID tags, and to save rich metadata with each saved
frame that can be searched on to find the data. The <code class="docutils literal"><span class="pre">xpdAcq</span></code> package is designed
to make the user inputs easier when collecting data, and to facilitate the saving
of rich and correct metadata with the saved data for easier retrieval later. It is
hoped that this will greatly speed up and facilitate users’ later data analysis
and modeling workflows.
You are reading the xpdAcq documentation. Documentation for the NSLS-II data acquisition
package, ‘Bluesky’ can be found <a class="reference external" href="http://nsls-ii.github.io/bluesky/">here</a>.</p>
</div>
<div class="section" id="xpdacq-philosophy">
<h2>xpdAcq philosophy<a class="headerlink" href="#xpdacq-philosophy" title="Permalink to this headline">¶</a></h2>
<p>When we are running the experiment we pretty much just want to be worrying about
setting up and running the scans, but to find the data later, and to help with
processing it (in the future the metadata will be passed directly to the data
reduction software) it is important that a complete stack of metadata is saved
with each scan. The philosophy of the xpdAcq software is to accomplish this.</p>
<p>Metadata comes from a number of sources: beamtime information, sample information, scan information, and
information coming from hardware during the experiment (such as sample temperature, for example).
With this in mind, <code class="docutils literal"><span class="pre">xpdAcq</span></code> assumes that you may be doing multiple scans on the same sample,
and so we define a scan as the union of a scan-plan (which contains all the parameters of the
scan you want to run) and a sample. Beamtime metadata consists of the SAF number, PI name,
list of experimenters and so on and is set up by the beamline scientist at the beginning of your
beamtime. There can be a wealth of information about your sample that you want to save. To make
this easier, we provide an excel spreadsheet that you can fill in before (or during) your experiment
that contains information about each separate sample that you will run.
When you run a scan using <code class="docutils literal"><span class="pre">xrun</span></code> and specifying a scan-plan and a sample,
<code class="docutils literal"><span class="pre">xpdAcq</span></code> will take all the beamtime metadata, the sample metadata, and all the parameters of the scan,
and save it in the scan header in the metadata store, linked to the actual dataset itself. You may also
specify more metadata to save when the scan is run by specifying it in <code class="docutils literal"><span class="pre">xrun</span></code>. Additional metadata
are also saved, such as a link to the <code class="docutils literal"><span class="pre">xpdAcq</span></code>‘s best guess at the right dark-collection to use for correcting
your scan data, calibration parameters for the experimental setup, and so on. These can be overridden during analysis
so don’t worry if they are incorrectly saved, but following the preferred <code class="docutils literal"><span class="pre">xpdAcq</span></code> workflow during data collection
can save you a lot of time later by making these guesses correct. Hardware generated metadata, such as sample temperature,
are also saved and can be retrieved later.</p>
<p>Later you will be able to use the scan metadata to search for your scans from the XPD database. For example, a search
on the PI-last name and a date-range will return all the datasets you collected in that date-range. If you add sample name
it will return all the scans done on that sample in that date-range and so on.</p>
<p>With this information in mind, please go ahead and start the step-by-step process
of setting up your beamtime in <a class="reference internal" href="usb_Beamtime.html#usb-beamtime"><span class="std std-ref">Setting up your Beamtime</span></a></p>
<p>return to <a class="reference internal" href="xpdusers.html#xpdu"><span class="std std-ref">For XPD Users</span></a></p>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="usb_Beamtime.html" class="btn btn-neutral float-right" title="Setting up your Beamtime" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="sb_overview.html" class="btn btn-neutral" title="Overview of xpd acquisition environment" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
© Copyright (c) 2016 trustees of Columbia University in the City of New York.
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'0.5',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.StickyNav.enable();
});
</script>
</body>
</html>