-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathusb_Running.html
773 lines (654 loc) · 44.2 KB
/
usb_Running.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Running scans — xpdAcq 0.5 documentation</title>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="top" title="xpdAcq 0.5 documentation" href="index.html"/>
<link rel="up" title="For XPD Users" href="xpdusers.html"/>
<link rel="next" title="Initial assessment of your data" href="usb_QuickAssess.html"/>
<link rel="prev" title="Where did all my Sample and Scan objects go?????" href="usb_Where.html"/>
<script src="_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search">
<a href="index.html" class="icon icon-home"> xpdAcq
</a>
<div class="version">
0.5
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<ul>
<li class="toctree-l1"><a class="reference internal" href="quickstart.html">Quick Start</a></li>
</ul>
<ul class="current">
<li class="toctree-l1 current"><a class="reference internal" href="xpdusers.html">For XPD Users</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="sb_overview.html">Overview of xpd acquisition environment</a></li>
<li class="toctree-l2"><a class="reference internal" href="sb_icollection.html">Overview of the bsui+xpdAcq environment</a></li>
<li class="toctree-l2"><a class="reference internal" href="usb_Beamtime.html">Setting up your Beamtime</a></li>
<li class="toctree-l2"><a class="reference internal" href="usb_Experiment.html">Setting up your XPD acuisition objects</a></li>
<li class="toctree-l2"><a class="reference internal" href="usb_Scan.html">ScanPlan objects</a></li>
<li class="toctree-l2"><a class="reference internal" href="usb_Where.html">Where did all my Sample and Scan objects go?????</a></li>
<li class="toctree-l2"><a class="reference internal" href="usb_Where.html#bt-list-is-your-friends">bt.list() is your friends</a></li>
<li class="toctree-l2"><a class="reference internal" href="usb_Where.html#bt-list-gotchas">bt.list() Gotchas</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Running scans</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#automated-dark-collection">Automated dark collection</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#nice-and-fresh"><strong>Nice and fresh</strong></a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#automated-calibration-capture">Automated calibration capture</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#quick-guide-of-calibration-steps-with-pyfai">Quick guide of calibration steps with pyFAI</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#sample-metadata-imported-from-spreadsheet">Sample metadata imported from spreadsheet</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#comma-separated-fields">comma separated fields</a></li>
<li class="toctree-l4"><a class="reference internal" href="#name-fields">name fields</a></li>
<li class="toctree-l4"><a class="reference internal" href="#phase-string">phase string</a></li>
<li class="toctree-l4"><a class="reference internal" href="#dictonary-like-fields">dictonary-like fields</a></li>
<li class="toctree-l4"><a class="reference internal" href="#sample-objects">Sample Objects</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#auto-masking">Auto-masking</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#applied-masks">Applied masks</a></li>
<li class="toctree-l4"><a class="reference internal" href="#using-the-auto-masker">Using the auto-masker</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="usb_QuickAssess.html">Initial assessment of your data</a></li>
<li class="toctree-l2"><a class="reference internal" href="usb_GlobalOptions.html">xpdAcq Configuration</a></li>
</ul>
</li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="beamlinestaff.html">For Beamline Staff</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="troubleshooting.html">Troubleshooting</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="release_note.html">Release Notes</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="index.html">xpdAcq</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="index.html">Docs</a> »</li>
<li><a href="xpdusers.html">For XPD Users</a> »</li>
<li>Running scans</li>
<li class="wy-breadcrumbs-aside">
<a href="_sources/usb_Running.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="running-scans">
<span id="usb-running"></span><h1>Running scans<a class="headerlink" href="#running-scans" title="Permalink to this headline">¶</a></h1>
<p>The hard work of the experimental setup is now done. It involved creating all the
rich metadata container and ScanPlan objects, but with this in the past it makes
it easy and low overhead to run the scans, allowing the experimenter to concentrate
on the science and not the experimental process.</p>
<p>To run scans there are just a few xpdAcq
run engines(functions) that you need. To run a scan you simply pick the run engine you want
and give it a predefined Sample object and
a predefined ScanPlan object, then hit return and sit back, your scan will be carried out.</p>
<p>The allowed scan types are:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">xrun</span><span class="p">(</span><span class="n">sample</span><span class="p">,</span> <span class="n">scanplan</span><span class="p">)</span>
</pre></div>
</div>
<p><code class="docutils literal"><span class="pre">xrun</span></code> stands for “XPD run” which is a our main run engine.</p>
<p>Here are some examples of a workflow. Assume a Ni_calibrant sample is loaded on the diffractometer
and the <code class="docutils literal"><span class="pre">Ni</span></code> Sample object is created as well as all the <code class="docutils literal"><span class="pre">ScanPlan</span></code> we need.
We will start by doing a scan with <code class="docutils literal"><span class="pre">Sample</span></code> object <code class="docutils literal"><span class="pre">Setup</span></code> on our <code class="docutils literal"><span class="pre">ct_1</span></code> ScanPlan.</p>
<p>Remember, always start with <code class="docutils literal"><span class="pre">bt.list()</span></code></p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">bt</span><span class="o">.</span><span class="n">list</span><span class="p">()</span>
<span class="go">ScanPlans:</span>
<span class="go">0: 'ct_5'</span>
<span class="go">1: 'ct_0.1'</span>
<span class="go">2: 'ct_1'</span>
<span class="go">3: 'ct_10'</span>
<span class="go">4: 'ct_30'</span>
<span class="go">5: 'ct_60'</span>
<span class="go">7: 'ct_1.5'</span>
<span class="go">8: 'ct_100.5'</span>
<span class="go">Samples:</span>
<span class="go">0: Setup</span>
<span class="go">1: Ni</span>
<span class="go">2: kapton_0.9mmOD</span>
<span class="go">3: kapton_1mmOD</span>
<span class="go">4: kapton_0.5mmOD</span>
<span class="go">5: activated_carbon_1</span>
<span class="go">6: activated_carbon_2</span>
<span class="go">7: activated_carbon_3</span>
<span class="gp">...</span>
</pre></div>
</div>
<p>The Sample object I want has list index 0 and the ScanPlan has list index 2.
Let’s try to make sure everything is ok.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">xrun</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="go">INFO: requested exposure time = 1 - > computed exposure time= 1.0</span>
<span class="go">INFO: No calibration file found in config_base. Scan will still keep going on</span>
<span class="go">INFO: no mask has been associated with current glbl</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| seq_num | time | pe1_image |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| 1 | 15:58:47.4 | 5.00 |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">generator count ['73dc71'] (scan num: 3)</span>
</pre></div>
</div>
<p>OK, it seems to work, let’s do some testing to see how much intensity we need.
We will do three setup scans with 1.5 seconds and 100.5 seconds exposure
and then compare them.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">xrun</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">7</span><span class="p">)</span> <span class="c1">#1.5 seconds</span>
<span class="go">INFO: requested exposure time = 1.5 - > computed exposure time= 1.5</span>
<span class="go">INFO: No calibration file found in config_base. Scan will still keep going on</span>
<span class="go">INFO: no mask has been associated with current glbl</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| seq_num | time | pe1_image |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| 1 | 16:01:37.3 | 5.00 |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">generator count ['728f2f'] (scan num: 5)</span>
<span class="gp">>>> </span><span class="n">setupscan</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">8</span><span class="p">)</span> <span class="c1">#100.5 seconds</span>
<span class="go">INFO: requested exposure time = 100.5 - > computed exposure time= 100.5</span>
<span class="go">INFO: No calibration file found in config_base. Scan will still keep going on</span>
<span class="go">INFO: no mask has been associated with current glbl</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| seq_num | time | pe1_image |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| 1 | 16:02:57.0 | 5.00 |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">generator count ['981e70'] (scan num: 6)</span>
</pre></div>
</div>
<p>It seems that the 2 second scans are the best, so let’s do[b] with desired <code class="docutils literal"><span class="pre">Sample</span></code>
to get the first data-set.</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="go">>>>In [13]: xrun(0, 8)</span>
<span class="go">INFO: requested exposure time = 100.5 - > computed exposure time= 100.5</span>
<span class="go">INFO: closing shutter...</span>
<span class="go">INFO: taking dark frame....</span>
<span class="go">INFO: No calibration file found in config_base. Scan will still keep going on</span>
<span class="go">INFO: no mask has been associated with current glbl</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| seq_num | time | pe1_image |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| 1 | 16:04:31.4 | 5.00 |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">generator count ['d770c7'] (scan num: 7)</span>
<span class="go">opening shutter...</span>
<span class="go">INFO: No calibration file found in config_base. Scan will still keep going on</span>
<span class="go">INFO: no mask has been associated with current glbl</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| seq_num | time | pe1_image |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">| 1 | 16:04:31.5 | 5.00 |</span>
<span class="go">+-----------+------------+------------+</span>
<span class="go">generator count ['0beaaf'] (scan num: 8)</span>
</pre></div>
</div>
<div class="section" id="automated-dark-collection">
<span id="auto-dark"></span><h2>Automated dark collection<a class="headerlink" href="#automated-dark-collection" title="Permalink to this headline">¶</a></h2>
<p>You might have found something weird when you ran the <code class="docutils literal"><span class="pre">xrun</span></code> command:</p>
<p><em>I only requested one ``xrun`` but the program runs two scans</em></p>
<p>So what happened?</p>
<p>That is a feature called auto-dark subtraction in <code class="docutils literal"><span class="pre">xpdAcq</span></code>.
When you are running your experiment, <code class="docutils literal"><span class="pre">xpdAcq</span></code> checks if you have
collected a <strong>fresh and appropriate</strong> dark frame every time it collects a scan.
The definition of <strong>fresh and appropriate</strong> is:</p>
<div class="section" id="nice-and-fresh">
<h3><strong>Nice and fresh</strong><a class="headerlink" href="#nice-and-fresh" title="Permalink to this headline">¶</a></h3>
<blockquote>
<div><div class="highlight-none"><div class="highlight"><pre><span></span>Given a certain period T (``dark window``), there exists a dark frame
with the same **total exposure time** and exactly the same **acquisition time**
as the light frame we are about collect.
</pre></div>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>At <strong>XPD</strong>, the area detector is running in <code class="docutils literal"><span class="pre">continuous</span> <span class="pre">acquisition</span></code> mode,
which means detector keeps <strong>reading</strong> but only <strong>saves</strong> images when <code class="docutils literal"><span class="pre">xpdAcq</span></code>
tells it to save, with the desired exposure time.</p>
<p>In short,</p>
<ul class="last simple">
<li>acquisition time is the collection time for a single frame from area detector.
This can take values between 0.1s to 5s.</li>
<li>exposure time is the user-defined total acquisition time</li>
</ul>
</div>
<p>Automated dark collection is enabled by default and it can be turned off by:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">glbl</span><span class="p">[</span><span class="s1">'auto_dark'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">False</span>
<span class="n">glbl</span><span class="p">[</span><span class="s1">'shutter_control'</span><span class="p">]</span> <span class="o">=</span> <span class="bp">False</span>
</pre></div>
</div>
<p>And the duration of your dark window can be modified by:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">glbl</span><span class="p">[</span><span class="s1">'dk_window'</span><span class="p">]</span> <span class="o">=</span> <span class="mi">200</span> <span class="c1"># in minutes. default is 3000 minutes</span>
</pre></div>
</div>
<p>Having <code class="docutils literal"><span class="pre">auto_dark</span></code> set to <code class="docutils literal"><span class="pre">True</span></code> is strongly recommended as this enables
<code class="docutils literal"><span class="pre">xpdAcq</span></code> to do automated dark frame subtraction when you pull out data from the
centralized <strong>NSLSL-II</strong> server.</p>
</div></blockquote>
</div>
</div>
<div class="section" id="automated-calibration-capture">
<span id="auto-calib"></span><h2>Automated calibration capture<a class="headerlink" href="#automated-calibration-capture" title="Permalink to this headline">¶</a></h2>
<p>Often times, keeping track of which calibration file is associated with a specific scan is very tiring. <code class="docutils literal"><span class="pre">xpdAcq</span></code> makes this easier or you. Before every
scan is being collected, the program goes to grab the most recent calibration
parameters in <code class="docutils literal"><span class="pre">xpdUser/config_base</span></code> and load them as part of the
metadata so that you can reference them whenever you want, and make in-situ data
reduction possible!</p>
<div class="section" id="quick-guide-of-calibration-steps-with-pyfai">
<span id="calib-manual"></span><h3>Quick guide of calibration steps with pyFAI<a class="headerlink" href="#quick-guide-of-calibration-steps-with-pyfai" title="Permalink to this headline">¶</a></h3>
<p>1. We assume that you have run <code class="docutils literal"><span class="pre">run_calibration()</span></code> in the <code class="docutils literal"><span class="pre">collection</span></code> window
with Ni as your calibrant. This will automatically expose the Ni, and after a
pause a 2D plot window should pop up on the acquisition computer, looking
something like this:</p>
<blockquote>
<div><a class="reference internal image-reference" href="_images/calib_05.png"><img alt="_images/calib_05.png" class="align-center" src="_images/calib_05.png" style="width: 400px; height: 300px;" /></a>
<p>That is the image we want to perform our calibration with. Use the <strong>magnify
tool</strong> at the tool bar to zoom in to something that looks like the figure below.
The magnifying tool is enabled by clicking on the button on the toolbar that
looks like a magnifying glass.</p>
<p>Now we will select five rings that PyFai will use to do the calibration. To do this
click on the magnifying glass button again to deselect the magnifying tool so the cursor
looks like an arrow. You will place the tip
of the arrow on the first ring and then <strong>RIGHT click</strong>. you will see dots going around the
ring you have selected. Then repeat this for the other four rings you will select.
<cite>For the highest accuracy, We recommend that you select the first, second, third and
6th ring, as shown in the figure.</cite> The 6th ring is weaker but well separated from its neighbor.</p>
<a class="reference internal image-reference" href="_images/calib_07.png"><img alt="_images/calib_07.png" class="align-center" src="_images/calib_07.png" style="width: 400px; height: 300px;" /></a>
</div></blockquote>
<ol class="arabic" start="2">
<li><p class="first">If you don’t like what you have selected, you can exit out using <code class="docutils literal"><span class="pre">CTL-C</span></code> and start again
by running <cite>run_calibration()</cite>. However, if you are happy with your selections accept them by first
making the collection terminal window active by clicking on it, then hitting <code class="docutils literal"><span class="pre"><enter></span></code>.</p>
<p>You will now follow the instructions coming from PyFai. It asks you to supply the
indices of the rings you have selected. Because PyFai was written by a computer
scientist and not a scientist, <strong>the first ring has number 0</strong>, the second has
number 1, the third has number 2 and the 6th has number 5.</p>
<p>If everything has gone well, after supplying all the indices, PyFai will pop
up a number of plots that can give an expert eye some indication of the quality
of the calibration. If the dashed lines look as if they are lining up well with
the peaks you have a good calibration. If not, <cite>CTL-C</cite> and start again. If yes,
activate the terminal window by clicking on it, hit <code class="docutils literal"><span class="pre"><enter></span></code> at the command prompt
then type <code class="docutils literal"><span class="pre">done</span></code>.</p>
</li>
</ol>
<blockquote>
<div><a class="reference internal image-reference" href="_images/calib_08.png"><img alt="_images/calib_08.png" class="align-center" src="_images/calib_08.png" style="width: 400px; height: 300px;" /></a>
<p>PyFai can be a bit finicky. If it hangs, type CTL-C and start over and make
sure you follow the instruction exactly.</p>
<p>You may find more information about calibration process from <a class="reference external" href="http://pyfai.readthedocs.io/en/latest/calibration.html">pyFAI documentation</a></p>
</div></blockquote>
<ol class="arabic" start="3">
<li><p class="first">You are done! <code class="docutils literal"><span class="pre">xpdAcq</span></code> has saved the calibration parameters and will store them will all subsequent scans until you
run another calibration.</p>
<p>To see the current calibration parameters, type <code class="docutils literal"><span class="pre">show_calib()</span></code>.</p>
<p>You can also find the calibration parameters in a file called
<code class="docutils literal"><span class="pre">pyFAI_calib.yml</span></code> in <code class="docutils literal"><span class="pre">.../xpdUser/config_base</span></code></p>
</li>
</ol>
<ol class="arabic simple" start="3">
<li>To clean up you can close all the PyFai windows, including
the 1D integration and 2D regrouping results that pop out (see below).
Return to the <code class="docutils literal"><span class="pre">Quick</span> <span class="pre">start</span> <span class="pre">tutorial</span></code> by hitting the browser back-arrow.</li>
</ol>
<blockquote>
<div><a class="reference internal image-reference" href="_images/calib_09.png"><img alt="_images/calib_09.png" class="align-center" src="_images/calib_09.png" style="width: 400px; height: 300px;" /></a>
</div></blockquote>
</div>
</div>
<div class="section" id="sample-metadata-imported-from-spreadsheet">
<span id="import-sample"></span><h2>Sample metadata imported from spreadsheet<a class="headerlink" href="#sample-metadata-imported-from-spreadsheet" title="Permalink to this headline">¶</a></h2>
<p>In order to facilitate retrospective operation on data, we suggest you to enter
as much information as you can and that is the main philosophy behind <code class="docutils literal"><span class="pre">xpdAcq</span></code>.</p>
<p>Typing in sample metadata during beamtime is always less efficient and it wastes
your time so a pre-populated excel sheet with all metadata entered beforehand
turns out to be the solution.</p>
<p>In order import sample metadata from spreadsheet, we would need you to have a
pre-filled spreadsheet with name <code class="docutils literal"><span class="pre"><saf_number>_sample.xls</span></code> sit in <code class="docutils literal"><span class="pre">xpduser/import</span></code>
directory. Then the import process is simply:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">import_sample_info</span><span class="p">()</span>
</pre></div>
</div>
<p><code class="docutils literal"><span class="pre">xpdAcq</span></code> will grab the <code class="docutils literal"><span class="pre">saf_number</span></code> and <code class="docutils literal"><span class="pre">bt</span></code> for current beamtime, so make sure you have your spreadsheet named with proper format. For example, if your SAF number is <code class="docutils literal"><span class="pre">300179</span></code>, then you should have your pre-populated spreadsheet with the name as <code class="docutils literal"><span class="pre">300179_sample.xls</span></code>, sit inside <code class="docutils literal"><span class="pre">xpdUser/import</span></code> directory.</p>
<p>To parse the information filled inside your spreadsheet, we have designed
several rules and here are the explantion to each of the rules.</p>
<div class="section" id="comma-separated-fields">
<h3>comma separated fields<a class="headerlink" href="#comma-separated-fields" title="Permalink to this headline">¶</a></h3>
<blockquote>
<div><p>Files with information entities are separated by a comma <code class="docutils literal"><span class="pre">,</span></code>.</p>
<p>Each separated by <code class="docutils literal"><span class="pre">,</span></code> will be individually searchable later.</p>
<p>Fields following this parsing rule are:</p>
<table border="1" class="docutils">
<colgroup>
<col width="28%" />
<col width="72%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><code class="docutils literal"><span class="pre">cif</span> <span class="pre">name</span></code></td>
<td>pointer of potential structures for your sample, if any.</td>
</tr>
<tr class="row-even"><td><code class="docutils literal"><span class="pre">User</span> <span class="pre">supplied</span> <span class="pre">tags</span></code></td>
<td>any comment you want to put on for this measurement.</td>
</tr>
</tbody>
</table>
<p>Example on <code class="docutils literal"><span class="pre">User</span> <span class="pre">supplied</span> <span class="pre">tags</span></code>:</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>background, standard --> background, standard
</pre></div>
</div>
<p>And a search on either <code class="docutils literal"><span class="pre">background</span></code> or``standard`` later on will include
this header.</p>
</div></blockquote>
</div>
<div class="section" id="name-fields">
<h3>name fields<a class="headerlink" href="#name-fields" title="Permalink to this headline">¶</a></h3>
<blockquote>
<div><p>Fields that are used to store a person’s name in <code class="docutils literal"><span class="pre">first</span> <span class="pre">name</span> <span class="pre">last</span> <span class="pre">name</span></code> format.</p>
<p>Each person’s first and last name will be searchable later on.</p>
<p>Fields following this parsing rule are:</p>
<table border="1" class="docutils">
<colgroup>
<col width="28%" />
<col width="72%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><code class="docutils literal"><span class="pre">Collaborators</span></code></td>
<td>name of your collaborators</td>
</tr>
<tr class="row-even"><td><code class="docutils literal"><span class="pre">Sample</span> <span class="pre">Maker</span></code></td>
<td>name of your sample maker</td>
</tr>
<tr class="row-odd"><td><code class="docutils literal"><span class="pre">Lead</span> <span class="pre">Experimenters</span></code></td>
<td>a person who is going to lead this experiment at beamline</td>
</tr>
</tbody>
</table>
<p>Example on name fields:</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>Maxwell Terban, Benjamin Frandsen ----> Maxwell, Terban, Benjamin, Frandsen
</pre></div>
</div>
<p>A search on either <code class="docutils literal"><span class="pre">Maxwell</span></code> or <code class="docutils literal"><span class="pre">Terban</span></code> or <code class="docutils literal"><span class="pre">Benjamin</span></code> or <code class="docutils literal"><span class="pre">Frandsen</span></code>
later will include this header.</p>
</div></blockquote>
</div>
<div class="section" id="phase-string">
<h3>phase string<a class="headerlink" href="#phase-string" title="Permalink to this headline">¶</a></h3>
<blockquote>
<div><p>Field used to specify the phase information and chemical composition of your
sample. It’s important to enter this field correctly so that we can have
accelerated data reduction workflow.</p>
<p>Fields follows this parsing rule are:</p>
<table border="1" class="docutils">
<colgroup>
<col width="18%" />
<col width="82%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><code class="docutils literal"><span class="pre">Phase</span> <span class="pre">Info</span></code></td>
<td>field to specify phase information and chemical composition of
your sample</td>
</tr>
</tbody>
</table>
<p>phase string will be expect to be enter in a form as
<code class="docutils literal"><span class="pre">phase_1:</span> <span class="pre">amount,</span> <span class="pre">phase_2:</span> <span class="pre">amount</span></code>.</p>
<p>An example of 0.9% sodium chloride solution will be:</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>Nacl: 0.09, H20: 0.91
</pre></div>
</div>
<p>This <code class="docutils literal"><span class="pre">Phase</span> <span class="pre">Info</span></code> will be parsed as:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="p">{</span><span class="s1">'sample_composition'</span><span class="p">:</span> <span class="p">{</span><span class="s1">'Na'</span><span class="p">:</span><span class="mf">0.09</span><span class="p">,</span> <span class="s1">'Cl'</span><span class="p">:</span><span class="mf">0.09</span><span class="p">,</span> <span class="sb">`H`</span><span class="p">:</span><span class="mf">1.82</span><span class="p">,</span> <span class="sb">`O`</span><span class="p">:</span><span class="mf">0.91</span><span class="p">},</span>
<span class="s1">'sample_phase'</span><span class="p">:</span> <span class="p">{</span><span class="s1">'NaCl'</span><span class="p">:</span><span class="mf">0.09</span><span class="p">,</span> <span class="s1">'H20'</span><span class="p">:</span><span class="mf">0.91</span><span class="p">},</span>
<span class="s1">'composition_string'</span><span class="p">:</span> <span class="s1">'Na0.09Cl0.09H1.82O0.91'</span><span class="p">}</span>
</pre></div>
</div>
<p><code class="docutils literal"><span class="pre">composition_string</span></code> is designed for data reduction software going to be
used. Under <code class="docutils literal"><span class="pre">xpdAcq</span></code> framework, we will assume
<a class="reference external" href="http://www.diffpy.org/products/pdfgetx3.html">pdfgetx3</a></p>
<p>As before, a search on <code class="docutils literal"><span class="pre">Na</span></code> or <code class="docutils literal"><span class="pre">Cl</span></code> or <code class="docutils literal"><span class="pre">H</span></code> or <code class="docutils literal"><span class="pre">O</span></code> will include this
header. Also a search on <code class="docutils literal"><span class="pre">Nacl=0.09</span></code> will include this header as well.</p>
</div></blockquote>
</div>
<div class="section" id="dictonary-like-fields">
<h3>dictonary-like fields<a class="headerlink" href="#dictonary-like-fields" title="Permalink to this headline">¶</a></h3>
<blockquote>
<div><p>Fields that are utilized to store information as <code class="docutils literal"><span class="pre">key-value</span> <span class="pre">pair</span></code> format.
Standard format of it is <code class="docutils literal"><span class="pre">key:</span> <span class="pre">value</span></code> and it also follows the comma-separate rule</p>
<p>Fields following this parsing rule are:</p>
<table border="1" class="docutils">
<colgroup>
<col width="51%" />
<col width="49%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><code class="docutils literal"><span class="pre">structrual</span> <span class="pre">database</span> <span class="pre">ID</span> <span class="pre">for</span> <span class="pre">phases</span></code></td>
<td>database name and the ID for
sample phases</td>
</tr>
</tbody>
</table>
<p>Example on dictionary-like fields:</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>ICSD:41120, CCDC:850926 ----> {'ICSD': '41120', 'CCDC': '850926'}
</pre></div>
</div>
</div></blockquote>
</div>
<div class="section" id="sample-objects">
<h3>Sample Objects<a class="headerlink" href="#sample-objects" title="Permalink to this headline">¶</a></h3>
<ul>
<li><p class="first"><strong>Sample</strong>:</p>
<p>Each row in your spreadsheet will be taken as one valid Sample and metadata
will be parsed based on the contents you type in with above parsing rule.</p>
<p>Generally, after successfully importing sample from spreadsheet, that is what
you would see:</p>
</li>
</ul>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="n">import_sample_info</span><span class="p">()</span>
<span class="o">***</span> <span class="n">End</span> <span class="n">of</span> <span class="kn">import</span> <span class="nn">Sample</span> <span class="nn">object</span> <span class="o">***</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="n">bt</span><span class="o">.</span><span class="n">list</span><span class="p">()</span>
<span class="n">ScanPlans</span><span class="p">:</span>
<span class="n">Samples</span><span class="p">:</span>
<span class="mi">0</span><span class="p">:</span> <span class="n">P2S</span>
<span class="mi">1</span><span class="p">:</span> <span class="n">Ni_calibrant</span>
<span class="mi">2</span><span class="p">:</span> <span class="n">activated_carbon_1</span>
<span class="mi">3</span><span class="p">:</span> <span class="n">activated_carbon_2</span>
<span class="mi">4</span><span class="p">:</span> <span class="n">activated_carbon_3</span>
<span class="mi">5</span><span class="p">:</span> <span class="n">activated_carbon_4</span>
<span class="mi">6</span><span class="p">:</span> <span class="n">activated_carbon_5</span>
<span class="mi">7</span><span class="p">:</span> <span class="n">activated_carbon_6</span>
<span class="mi">8</span><span class="p">:</span> <span class="n">FeF3</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span><span class="mi">4</span><span class="o">-</span><span class="n">bipyridyl</span><span class="p">)</span>
<span class="mi">9</span><span class="p">:</span> <span class="n">Zn_MOF</span>
<span class="o">...</span>
<span class="mi">41</span><span class="p">:</span> <span class="n">ITO_glass_noFilm</span>
<span class="mi">42</span><span class="p">:</span> <span class="n">ITO_glass_1hrHeatUpTo250C_1hrhold250C_airdry</span>
<span class="mi">43</span><span class="p">:</span> <span class="n">ITO_glass_1hrHeatUpTo450C_1hrhold450C_airdry</span>
<span class="mi">44</span><span class="p">:</span> <span class="n">ITO_glass_30minHeatUpTo150C_1</span><span class="o">.</span><span class="mi">5</span><span class="n">hrhold150C_airdry</span>
<span class="mi">45</span><span class="p">:</span> <span class="n">CeO2_film_calibrant</span>
<span class="mi">46</span><span class="p">:</span> <span class="n">bkg_1mm_OD_capillary</span>
<span class="mi">47</span><span class="p">:</span> <span class="n">bkg_0</span><span class="o">.</span><span class="mi">9</span><span class="n">mm_OD_capillary</span>
<span class="mi">48</span><span class="p">:</span> <span class="n">bkg_0</span><span class="o">.</span><span class="mi">5</span><span class="n">mm_OD_capillary</span>
<span class="mi">49</span><span class="p">:</span> <span class="n">bkg_film_on_substrate</span>
</pre></div>
</div>
<ul id="background-obj">
<li><p class="first"><strong>Background</strong>:</p>
<p>It is recommended to run a background scan before your sample so it is available for
the automated data reduction steps. It also allows you to see problems with the
experimental setup, for example, crystalline peaks due to the beam hitting a shutter.</p>
<p>You can associate a Sample as the background for the desired
Sample freely. Linking the background with the sample together also makes the
data-reduction workflow easier.</p>
<p>We specify this grouping by entering background sample name into the
<code class="docutils literal"><span class="pre">Sample-name</span> <span class="pre">of</span> <span class="pre">sample</span> <span class="pre">background</span></code> column in the spreadsheet. You can
fill in the <em>Sample Name of your background</em> to whichever sample you want to relate.</p>
<p>For example, in our <a class="reference external" href="https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/xpd-users/_6NSRWg_-l0">spreadsheet template</a> we created pure background
objects kapton_1mmOD, kapton_0.9mmOD and kapton_0.5mmOD
and we link Ni with background kapton_1mmOD by specifying it
at <code class="docutils literal"><span class="pre">Sample-name</span> <span class="pre">of</span> <span class="pre">sample</span> <span class="pre">background</span></code> column.</p>
<p>A proper linking between <strong>Sample</strong> and <strong>Background</strong> can be seen
from metadata stored inside the <code class="docutils literal"><span class="pre">Sample</span></code> object. As usual, let’s
interrogate the metadata:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">[]:</span> <span class="n">bt</span><span class="o">.</span><span class="n">samples</span><span class="o">.</span><span class="n">get_md</span><span class="p">(</span><span class="mi">15</span><span class="p">)</span> <span class="c1"># that's for example, index depends on</span>
<span class="c1"># your spreadsheet</span>
<span class="n">out</span><span class="p">[]:</span>
<span class="p">{</span><span class="s1">'bkgd_sample_name'</span><span class="p">:</span> <span class="s1">'kapton_0.9mmOD'</span><span class="p">,</span>
<span class="s1">'bt_piLast'</span><span class="p">:</span> <span class="s1">'Billinge'</span><span class="p">,</span>
<span class="o">...</span>
<span class="p">}</span>
</pre></div>
</div>
<p>The example above shows your <code class="docutils literal"><span class="pre">Sample</span></code> with index <code class="docutils literal"><span class="pre">15</span></code> has been
linked with background <code class="docutils literal"><span class="pre">kapton_0.9mmOD</span></code>. This can largely speeds up
the automated data-reduction workflow that we will have in the future!</p>
</li>
</ul>
</div>
</div>
<div class="section" id="auto-masking">
<span id="auto-mask"></span><h2>Auto-masking<a class="headerlink" href="#auto-masking" title="Permalink to this headline">¶</a></h2>
<p>Masking can be a tedious process, requiring long hours judging which pixels
are good and which need to be removed. The our automated masking software aims
to alleviate this by applying a set of masks in sequence to return better
quality data.</p>
<p>Masks can be created/used in two ways. The default procedure is a mask is
created for a low scattering sample (usually kapton). Then this mask is reused
for each subsequent image taken with the same detector position. The second
modality is that each image gets is own bespoke mask, potentially derived from
the low scattering mask.</p>
<div class="section" id="applied-masks">
<h3>Applied masks<a class="headerlink" href="#applied-masks" title="Permalink to this headline">¶</a></h3>
<ol class="arabic" start="0">
<li><dl class="first docutils">
<dt>Any mask passed in to the software:</dt>
<dd><p class="first last">If you have any preexisting masks, we will use those as a starting position
to add upon.</p>
</dd>
</dl>
</li>
<li><dl class="first docutils">
<dt>Edge mask:</dt>
<dd><p class="first last">A default of 30 pixels from the edge of the detector are masked out.
These pixels are usually faulty as the detector edge has lower than
expected intensity.</p>
</dd>
</dl>
</li>
<li><dl class="first docutils">
<dt>Lower threshold mask:</dt>
<dd><p class="first last">A lower threshold mask, which removes all pixels who’s intensities are
lower than a certain value is applied. The default threshold is 0.0</p>
</dd>
</dl>
</li>
<li><dl class="first docutils">
<dt>Upper threshold mask:</dt>
<dd><p class="first last">An upper threshold mask, which removes all pixels who’s intensities are
higher than a certain value is applied. This mask is not applied in the
default settings.</p>
</dd>
</dl>
</li>
<li><dl class="first docutils">
<dt>Beamstop holder mask:</dt>
<dd><p class="first last">A beamstop holder mask, which removes pixels underneath a straight
beamstop holder is applied. The beamstop holder is masked by finding the
beamcenter and drawing a pencil like shape from the beamcenter to the edge
of the detector. All the pixels within this polygon are masked. The default
settings are to mask out 30 pixels on either side of the line connecting
the beamcenter and the detector edge</p>
</dd>
</dl>
</li>
<li><dl class="first docutils">
<dt>Binned outlier mask:</dt>
<dd><p class="first last">Lastly a binned outlier mask is applied, removing pixels which are alpha
standard deviations away from the mean intensity as a function of Q. This
mask aims to remove many of the dead/hot pixels and streaks. The default
alpha is 3 standard deviations.</p>
</dd>
</dl>
</li>
</ol>
</div>
<div class="section" id="using-the-auto-masker">
<h3>Using the auto-masker<a class="headerlink" href="#using-the-auto-masker" title="Permalink to this headline">¶</a></h3>
<p>To use the auto-masker once, creating masks used for subsequent images,
just run the command:</p>
<blockquote>
<div><div class="highlight-python"><div class="highlight"><pre><span></span><span class="n">run_mask_builder</span><span class="p">()</span>
</pre></div>
</div>
</div></blockquote>
<p>This will take a shot and mask it. This mask will then be saved and loaded
into subsequent experiment <cite>run_headers</cite> allowing them to be used for the next
images.</p>
<p>Let’s <a class="reference internal" href="usb_QuickAssess.html#usb-quickassess"><span class="std std-ref">take a quick look at our data</span></a></p>
</div>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="usb_QuickAssess.html" class="btn btn-neutral float-right" title="Initial assessment of your data" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="usb_Where.html" class="btn btn-neutral" title="Where did all my Sample and Scan objects go?????" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
© Copyright (c) 2016 trustees of Columbia University in the City of New York.
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'0.5',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.StickyNav.enable();
});
</script>
</body>
</html>